Micro services show their benefits in a large organization.
It’s a tool to solve people issues. They can remove bureaucratic hurdles and allow devs to somewhat be autonomous again.
In a small startup, you really don’t gain much from them. Unless if the domain really necessitates them, eg. the company uses Elixir but all of the AI toolings are written in Python/Go.
If your application has different load or resource requirements, you should build separate services, even in a startup.
You can put most of your crud and domain logic in a monolith, but if you have a GPU workload or something that has very different requirements - that should be its own thing. That pattern shouldn't result in 100 services to maintain, but probably only a few boundaries.
Bias for monolith for everything, but know when you need to carve something out as its own.
microservices can also cause organizational dependencies and coordination that wouldn't otherwise be necessary. i've seen it create at least as many people issues as solve them. one seemingly innocuous example is the policy of 'everybody just uses whatever services they want', which can hugely increase the ongoing maintenance requirements and seems to require that everyone learn everything in order to be functional. which never happens, which means you're always chasing people down.
I probably just haven't checked these comment threads enough yet because I'm surprised I haven't seen this posted, but even though this is a bit old now, https://youtu.be/y8OnoxKotPQ, there is a reason it resonated with so many. It's spot on with the downsides microservices can inflict.
I've certainly seen microservices be a total disaster in large (and small) organizations. I think it's especially important that larger organizations have standards around cross-cutting concerns (e.g. authorization, logging, service-to-service communication, etc.) before they just should "OK, microservices, and go!"
I was starting from the assumption that Elixir does something they need, but yeah in most cases Golang would cover the same thing. Even then, you probably have separate Golang and Python, or just two separate Python services.
I tried three times, just for fun, to start a decent-sized projects from start to finish without interventions. All failed. The AI can't seem to have far enough horizon to see the projects to completions.
AI needs a super well defined scoped, for example: an IDE companion or logo generator or writing a specific Class implementation, to thrive.
It used to be such a torture (in my own mind) to constantly trying to come up with a new hustle.
But not anymore. As I got older and after fatherhood, I have learned that balance is everything. Including valueing my time in having fun doing productive things.
If the hacking side project gives me tremendous enjoyment, then it is a win already both from happiness and the “job training” aspect.
As it so happened, I just hang out with my little cousins who are into robotics and so glad that my side projects in the past helped me connect with them, and I was able to give valuable advice. That was a big win for me.
Also, I have learned the value of providing stability for my own family. Something that the childless me never appreciated. Throwing away stability to start a startup has a steep cost these days.
I initially stopped playing when I had kids. As they got older though we started gaming together on weekends while my wife has a break. We have an absolute blast together building a world or questing in Minecraft or squadding up and getting a victory royale in Fortnite. I know it probably won't last as they get into their teens but for now it's something both me and them look forward to every week.
1. How did he tell Claude to “update” based on the notebook entries?
2. Won’t he eventually ran out of context window?
3. Won’t this be expensive when using hosted solutions? For just personal hacking, why not simply use ollama + your favorite model?
4. If one were to build this locally, can Vector DB similarity search or a hybrid combined with fulltext search be used to achieve this?
I can totally imagine using pgai for the notebook logs feature and local ollama + deepseek for the inference.
The email idea mentioned by other commenters is brilliant. But I don’t think you need a new mailbox, just pull from Gmail and grep if sender and receiver is yourself (aka the self tag).
Thank you for sharing, OP’s project is something I have been thinking for a few months now.
The "memories" table has a date column which is used to record the data when the information is relevant. The prompt can then be fed just information for today and the next few days - which will always be tiny.
It's possible to save "memories" that are always included in the prompt, but even those will add up to not a lot of tokens over time.
> Won’t this be expensive when using hosted solutions?
You may be under-estimating how absurdly cheap hosted LLMs are these days. Most prompts against most models cost a fraction of a single cent, even for tens of thousands of tokens. Play around with my LLM pricing calculator for an illustration of that: https://tools.simonwillison.net/llm-prices
> If one were to build this locally, can Vector DB similarity search or a hybrid combined with fulltext search be used to achieve this?
Geoffrey's design is so simple it doesn't even need search - all it does is dump in context that's been stamped with a date, and there are so few tokens there's no need for FTS or vector search. If you wanted to build something more sophisticated you could absolutely use those. SQLite has surprisingly capable FTS built in and there are extensions like https://github.com/asg017/sqlite-vec for doing things with vectors.
You are asking why Canvas? The answer is simple: Canvas is vastly better performing when given a large amount of datapoints. The difference is not even on the same plane.
reply