What would you do if you could read out any SaaS application? [OSS]

[mathiasn]

Hey everyone,

imagine you could read out data of any SaaS application, what would it be? I’d like to understand use-cases you might have.

Here are some examples to get you going:

* Downloading invoices from Asana, Notion etc. * Get a list of projects from Asana as CSV * Get a list of users with their permission in Jira as CSV ...

What use-case for which application(s) would be interesting for you and why? Thank you.

[noduerme]

I'm trying but I can't think of an example that affects me personally. I'd say that if the SaaS isn't making this data easily accessible to you in whatever format you want, it's a shitty service and you shouldn't use it.

[mathiasn → noduerme]

Fair point. Sometimes, you want it quicker, easier and in another form. Anyway, thanks a lot for your comment!

[noduerme → mathiasn]

have you messed around with https://clickhouse.com/docs/en/intro at all? I've just been toying with it but it has a lot of promise for taking data from one place and patching it into another dataset without writing one-time code to get it done. That seems to be kind of their M.O. No association here but I feel like it's what I'd use next time if I were trying to tie some half-assed APIs or web scrapes together and extract the data into my own format.

[mathiasn → noduerme]

Hmm not at all. But isn't it just another persistence storage? I mean yes, it's a column instead of row-oriented one. But essentially it saves data.

The core problem I want to solve is basically getting data from various SaaS tools. Alone getting the data is a big challenge especially when it's usually not available in the right form or the API is not accessible. Another one is then consolidating data of several tools with complete different data models.

[noduerme → mathiasn]

FWIW it probably wouldn't solve your problem with any particular SaaS, but the CLI app is quite clever, doesn't have anything to do with their commercial offerings and just lets you query CSV files as if they were SQL, and join them to SQL queries... https://clickhouse.com/docs/en/interfaces/cli/

So to some extent, this helps with writing scripts for merging different data models, and different formats.

To be fair I've never used it for anything serious. I've written so many web scrapers and little one-time tools to match up data, I lose track. It's hard to think of a way to fix this as a global problem when every service wants to lock you in somehow... except to demand they make your data open for you. I suppose you could imagine a standard and try to enforce it with a badge or something. There's an incentive for services not to comply.

If I read your post right, you're talking about tackling this as a Big Problem that could be solved technically. In my experience - not to be jaded - you'd be chasing the endless dragon; and there's no money in it. You only live once, and spending hours reverse engineering non-existent APIs for fickle SaaS companies is probably not the ideal use of one's resources. Just speaking as an indie dev. Partly because any dev plugging into such a service would have to trust the service to keep up with whatever non-API they wanted to mangle, and in any single use case a dev is better off rolling their own. It'll always be fractured.

/sorry, I'm drunk and thinking out loud. But your idea isn't bad, just too glorious for reality ;)

[mathiasn → noduerme]

That's great advice actually. Thanks a lot for your open words, appreciate it!