On: Things we learned about LLMs in 2024

[theferalrobot]

Which models perform better than 4o or o1 for your use cases?

In my limited tests (primarily code) nothing from llama or Gemini have come close, Claude I’m not so sure about.

[torginus → theferalrobot]

How good is the best model of your choice at doing architecture work for complex and nontrivial apps?

I have been bashing my head against the wall over the course of the past few days trying to create my (quite complex) dream app.

Most of LLM coding I've done involved in writing code to interface with already existing libs or services and the LLMs are great at that.

I'm hung up on architecture questions that are unique to my app and definitely not something you can google.

[fullstackchris → torginus]

Don't wanna be that typical hackernews guy but I couldnt resist... if your app is "quite complex" there is probably a way or ways you can break it down into much simpler parts. Easier for you AND the LLM. It always comes back to architecture and composition ;)

[torginus → fullstackchris]

I don't want to be mean, but that bit of eastern wisdom you dispensed sounds incredibly like what a management consultant would say.