I'm a full stack developer and have looked into LangChain and am daunted by its surface area. I'm trying to understand where and how exactly does it add value.
It's bloated and ambitious and being built way too fast, and making very confusing design choices. But the alternatives I've found, such as Microsoft's Autogen, are arguably clunkier.
I feel the same! Just look at all the separate classes for different model providers for tackling minor input/output differences.
As for the alternatives, I think there are 3 things to think about:
1. LLM call itself
2. Orchestration of calls & monitoring/input/output
3. Integrations with external services
For #1, if we were to limit the discussion to model providers with REST APIs (OpenAI, Anthropic, Google, Cohere, Groq, Together, ElevenLabs all have them), I think it would be much simpler to build a simple wrapper directly on top of fetch or equivalent APIs and even skip the SDKs.
For #2, don't think LangChain is a better solution than alternatives like state charts.
#3 is where LangChain might offer some value, but it's pretty thin.
> Just look at all the separate classes for different model providers for tackling minor input/output differences.
It's bizarre: Standardize the API via individually wrapped endpoints. WTF?
Yes, after spending more time trying to decypher the LangChain docs than it would take to roll-my-own, everything I've done so far has involved rolling-my-own.
I still think I might need to work with LangChain however: I'm afraid it's going to be what employers will be looking for.
> It's bizarre: Standardize the API via individually wrapped endpoints. WTF?
Exactly!
> Yes, after spending more time trying to decypher the LangChain docs than it would take to roll-my-own, everything I've done so far has involved rolling-my-own.
Looks like quite a few are doing the same, that is, rolling their own. Wonder if there needs to be a way to do it where it's standardized but still feels like rolling your own.
> I still think I might need to work with LangChain however: I'm afraid it's going to be what employers will be looking for.
Yeah workplaces are so different than working on your own projects. Been there! Might be you can get started that way, and slowly help people "see the light"!
Please try out https://magentic.dev/ ! It is a light wrapper that is standard across LLM providers and handles the boilerplate code related to structured outputs and function calling. It doesn't include data/vector stores or integrations so it's a little lower-level than langchain, but for a lot of use cases it gives you the flexibility needed.
I've found making reusable software in the AI-ML space will absolutely drive you nuts. I've been there and done that. For instance at one job I developed a model trainer and never really got an answer that combined the values of "can train models that take >1 computer*day to train" and "has model selection features half as good as scikit-learn". (e.g. sorry Huggingface's model eval tools and data prep tools strike me as so wrongheaded they make me want to hurl, even though the actual training and inference stuff is fine... Trouble is training will get you a model that works sometime some of the time after a large number of tries, consistently getting models you can use takes model selection and discipline that people in that business don't have. The average data scientist is so hung up on the April sales report being a work of art that the idea of making it a science so you have a cron job that makes a monthly sales report is totally foreign)
Some of it is that you can't please anybody, some of it is inevitably you are going to introduce some abstractions that cost more than they benefit and very few people have the courage to undo that, particularly once you have systems that are (almost) working. Worse than that people are intimidated by the problem, particularly when they see Langchain, which makes them think they can't do it on their own, except that they can.
It's like the way Zapier puts a tax on the API economy. Programmers know that you can often write a script to access an API through Python that takes less time to write than a requirements doc. Unfortunately managers buy the hype that you need Zapier and don't start to see it's a problem until they realize they'd be making profits if they weren't paying for Zapier.
and maybe added Faiss or pinecone you could probably have a prototype running in the time you can spend contemplating WTF is wrong with LangChain so run don't walk.
I totally agree that it suffers from premature or even wrong abstractions. I guess it's to be expected because it hasn't had enough time to refine those yet -- it takes years! But the issue is that once it's out in production maintaining backward compatibility becomes top priority and hinders the refinement process.
I've had a very long term project (first thought about it in 2004) of developing workflow-based systems with classifiers such as
* an application tracking system for jobs I might want to apply to
* a smart RSS reader that picks out the top articles from a 3000 article/day feed
* a "bookmark manager", "web crawler" and "personal agent" that can import image galleries on a large scale and let me rate and tag them
* a tool I'm using to sort through 400 notes from a friend's Evernote notebook
These have been badly underengineered but highly effective. Talking about my ATS got me a job at an AI startup on my first interview. The trainer for the RSS reader is a crude set of python scripts but it makes a great model every time and is easy to make simple changes to.
Over time though I've learned a lot about the problem so the last two are running out of the same codebase and I'm probably going to get the RSS reader running in the same system. It took me literally years though to design components that were worth reusing.
It's bloated and ambitious and being built way too fast, and making very confusing design choices. But the alternatives I've found, such as Microsoft's Autogen, are arguably clunkier.