Wondering what the plan is to steward Eureka Labs, LLM101n, and whatever else was being cooked up. As a fellow educator, was very much looking forward to seeing how this would have evolved things.
I struggle to see the difference between "Let AI do that" and what a founder/executive is instinctively led to do also (i.e. delegate). Why does it have to be an ADHD thing? Yes, I see the risks of AI for someone with ADHD (described well in this article [0]), and for that reason I agree that ADHDers should be careful with these tools, as they present both a lot of promise and peril. But also... delegating functions to an 'agent' (whether human or AI) is just what people end up doing in life. Hard to tell these things apart...
From what I understand, the RAM shortage is more about AI inference than AI training. Yes, training created much of the early HBM crunch because frontier-model training clusters need tons of HBM near GPUs, but inference is what is keeping the pressure on now and into the future.
I’m surprised Karpathy thinks this is a viable solution for a quasi-continual learning system. Yes, it’s cool to experiment with these sort of ‘intermediate knowledge systems’, but the real goal within this current LLM paradigm remains clear: new information within a knowledge system should be manifest through updating the weights! Many efforts are taking a crack at this, which this really helpful talk [0] by Jack Morris goes into. I’m in full agreement with the other comments here: this ‘LLM Wiki’ merely results in “2nd-order information” that will only muddy the picture.
Furthermore, I’m left wondering why Karpathy jumped to a wiki-style approach versus something more akin to “version control but for LLM notebooks”.
I have no doubt that using LLMs as a notebook, which many of us have been doing now (and the best end-user application of which is Google’s NotebookLM), are a viable path forward for knowledge management. I find myself going back to certain LLM conversations as a ‘running log’ of a given project (akin to notebook-style thinking/building).
But what about merging concepts from the version control world (Git/SVN), rather than the wiki world? Karpathy should do more explanation about what took him down the wiki route vs that, in these early days of using LLMs as notebooks.
I'm surprised that neither this article nor any of the comments have thus far mentioned the fact that rarely do tombstones, eulogies, and the like mention or emphasize much a person's occupation. Most of the time it's "Here lay {name}, loving {relation}, {relation}, and {relation}. May he eternally do {hobby} in the skies forever."
And here's the crazy part: I'm still not convinced this is the right take! Of course we are our jobs and what we do. It literally shapes who you get to meet on the dating market, favors you can call in, and so forth. How naive it is to think otherwise. The Cracked article that another post linked to is pretty spot-on.
Yes, obviously being a 'person' who can be 'with' people (cf. 'for' people) is simultaneously also a thing. Both are true. But "You are not your job" is just cope, and the tombstones and eulogies fall for it too.
> This sounds like the Loss Disguised as a Win concept from gambling addiction. Consider the hundreds of lines of code, all the apps being created: some of these are genuinely useful, but much of this code is too complex to maintain or modify in the future, and it often contains hidden bugs.
This person needs to study Ronald Coase's Theory of the Firm in order to not make incorrect assertions like 'the only moat left is money'. There's many other criticisms to level at this piece, but this is the one I'll use my reply/comment on. If the author was right, then the entities with the most money, or ability to print money, a.k.a. the government, would be in charge of all economic activity. They are not. Empirically, what we observe about human economic activity is that it's not about who has the most money. Money helps, and certainly there's economic structures that are a tier below government but definitely far larger than a small business that can use money as a weapon to stay entrenched or gain other advantages in the market, etc, but it's not the only thing. Empirically it is just not. There has been, is, and will continue to be room for what other commenters have called 'creativity' broadly speaking, and that even includes creatively recruiting other people to your cause/company versus another's, etc, that 'the one with the most money' has not alone shown to be decisive.
reply