My first job was in a FinTech and the way production access was managed scared me. This is my approach at streamlining the process.
Basically a PR review flow for SQL queries, enforcing the 4-eyes principle so you never accidentally can do a Delete * form users, forgetting the where clause.
Yes, I already thought about this. In theory all "production access", be it ssh, k8s commands, database access of any kind or even a ruby shell could go through such a a flow.
It's a bit of a pain to integrate well with everything though, but I have a lot of plans, will just take some time to mature :D
Thank you! This was initially a startup idea but it was very hard to sell a cyber security solution with a very bare bones product to companies large enough to have this problem. Because the required features of such large orgs are just too much for such an early stage team.
So I'm now doing it a bit as a side project but I hope to at some point also offer a paid version to fund development long term.
I mean I love postgres like the next guy. And I like simple solutions as long as they work.
I just wonder if this is truly simpler than using a redis or rabbitmq queue if you need Queues. If you're already using a cloud provider sqs is quite trivial as well.
I guess if you already have postgres and don't want to use the cloud provider's solution. You can use this to avoid hosting another piece of infra.
db-based gives you the ability to query against your queues, if you use case needs it. Other options tend to dispose the state once the job is finished.
But only if it could do it with reasonable accuracy.
The problem is that AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.
Surely this is still a massive problem for any real world enterprise use case unless you throw a human in the loop (which kills the productivity benefit) or you stamp a massive disclaimer on the output
Well, this thing I’m doing isn’t good enough for an audit or the like, but it’s good enough for sanity checking the budget and flagging things for further checking. And without the AI, you just wouldn’t do it at all, because it would take weeks to write a “parser” for these PDFs.
Actually, it doesn’t even need PDFs. It works just about as well if you just feed it PNGs of the pages. Crazy.
>AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.
Because this is what is used to deal with non-formal and unstructured data, if you build something that would be always accurate to the task, then you would have solved it formally.
I wholeheartedly agree with this, when I started working on Kviklet we were a team of 3 and one of us was a lot more perfectionist than the others. It took a lot of convincing to even put our (tbf shitty) first website version up. Much more to release our repo.
We lost that "co-founder" early on, but man I'm glad we released early and tried to sell.
It didn't work and we found no buyers but imagine we still were working on a product without knowing if anyone would ever want to pay for it, keeping our hopes up in the dark.
By now we went with the backup plan and open sourced and have a few cool users. Could maybe even say it's a small community: https://github.com/kviklet/kviklet
It's not the startup success story that I hoped for a year ago. But it's a lot better than still hoping for it and not being a bit more grounded.
Also open-source doesn't mean I can never sell support or a premium version and still make a few bucks right? For now it's just a fun side project though.
> Pull Request-like Review/Approval flow for database queries
Terrible description IMO. a query should not need approval. Should use mutation or edit or update or modification. Even if query is technically correct it just sounds wrong and confusing.
I'm not quite sure what you mean. Maybe statement instead of query would be more accurate but I think people get the gist of it just fine.
Also, I disagree a manual query like: "select * from credit_cards;" should probably go through an approval flow if you have a table like that in your prod env.
A guy I used to work with said he worked for a company where all queries had to get approved by one of two full time DBAs - apparently with good reason as someone tried to modify a query that would have joined with half the rows in some gigantic table.
I worked at a place where only certain teams with a dedicated DBA were trusted to write direct queries (based on past incidents). All other teams had to ask a central DBA team to build stored procedures for any interaction with the database. If you think that this would create a huge backlog, you are correct... Non critical updates also needed to be coordinated with a "release train" where the code had to be ready 2 weeks before deployment due to the amount of testing it required. It was one of the major drivers behind an initiative to create micro services with separate databases that each team could do what they wanted with.
We ended up with a huge number of micro services and special orchestrator services to handle distributed transactions. But I guess that in a company of that scale, there are no perfect solution. At least we were able to make changes within minutes/hours instead of weeks.
Paradoxically we also got more pressure to deliver. In the past it was acceptable to leave a healthy buffer at the end of the scrum, to avoid missing the release train. This meant that we often spent the remaining buffer on refactoring, fixing small bugs that we felt we had time for or experimenting with POCs.
No, that's just what everyone calls SQL statements. It's fine.
I like this idea. It sits between "developers have access to prod" (no! bad idea!) and requiring everything to be signed in triplicate. It provides a low-friction way to make reviewable changes to prod.
The project is now open-source and I'd like to continue maintaining it since it does have a few users and I still believe in the value it provides. But I also want to work in a larger team again on a more meaningful endeavor.
I have 4+ years of startup experience of various sizes and have done a lot of different jobs in those. Definitely a backend focus, but wouldn't mind learning anything new, I tend to pick things up quickly. I have some leadership and product management experience and wouldn't mind doing that again as long as I can keep somewhat hands-on.
Just like agile, DevOps has some good intentions. It's always about how it's executed and like anything in software engineering you will run into trade off situations where you have to find the best solution for your organization and product.
I really enjoy working in a deploy often and fast environment though and I firmly believe that fast feedback loops are one of the most important things for development speed. And this is what DevOps at its heart is about. How you achieve this and how reasonable it is for your situation is left for you to decide.
+1 for that book but it's a bit rough for a start I think. I'd try headspace, calm or any of the free guided meditations on YouTube for a bit before diving into that book. And yes the book reads quite spiritual despite the author being also a neurologist I think so it might not be for everyone.
multimodal llm would of course blow it all out the water, so some llama3-like model is probably SOTA in terms of what you can run yourself. something like https://huggingface.co/blog/idefics2
I wholeheartedly agree with this. I also am using an ad blocker, but aside from that the pages that are returned work perfectly fine for me for the most part.
Yes sometimes there is some useless blogs taking up top spots but then I tried the so hyped alternatives like kagi and they don't filter these out either so eh...
It works perfectly fine even on the Google android app that is unaffected by uBlock. I always hear people complain about search results but I can't remember the last time I even had to scroll down when searching. I do usually mostly search for ultra-specific programming stuff though.
You can kinda do this with Notion AI. It's not quite the same as slack ofcourse because you need to manually build up the knowledge base first but if it is well maintained you can use it to answer questions easily.
Probably there is some tools out there to create notion pages from slack history? Would be a useful tool at least.
My first job was in a FinTech and the way production access was managed scared me. This is my approach at streamlining the process. Basically a PR review flow for SQL queries, enforcing the 4-eyes principle so you never accidentally can do a Delete * form users, forgetting the where clause.
reply