As a counterpoint, I also tried writing something with Claude last weekend: a Google docs clone[1]. I spent $170 on Anthropic API credits, and got something that did mostly what I asked but was basically useless. It seems that for simple interfaces for which there is an exact specification, like the recent compiler and web browser examples, it's possible to write bigger projects that "work" as a demo although not in a way they'd be viable alternatives. For anything that requires taste and judgment, we've still got a long way to go. There are lots of great demos out there but few if any real examples of vibe coded (or whatever you want to call it) software standing alone as an alternative to project people wrote.
So we are now at the stage that AI coding agents don’t work because you can’t create a good google docs alternative from scratch without dependencies with $170 in one weekend.
I think the conclusion is that Vibe oding is shit, and I agree with this, but having an AI assistant that can do some specific tasks that you can review is a good strategy.
I also have a story about Vibe Coding, I had the AI make for me a mqarkdown editor with some extra features, it worked fine but the problem is I have no idea how it works, I do not know if I can add feature X easily or it needs a rewrite from scratch, I have no ideas for improvements or new features, if there is a problem I have no clue what causes it since I I never looked at the code.
So I concluded that I (others can do whatever they want, they are free people with their own standards ) I will only use Vibe Coding for throw away personal shit, that I will not make public, like for example I made some bash scripts, some python scripts to automate some stuff.
Does "vibe coding" now mean the agent needs to produce a working product in one shot? Or not even just working, but meeting unspecified requirements? I thought in the original phrasing by Karpathy it meant that you don't care about the code, yet you may still iterate because you might care about the product. I tried out the "Impeccable" skill for Claude Code and found it to be very useful if you care about visual aspects and good UI/UX. (I vibe coded a web app for tracking personal finances where I used this skill, see: https://github.com/AdrianVollmer/Solvency. Not one-shotted, and not perfect, but personally I'm impressed with what is possible in a few weekends with a $90 subscription to Claude.)
> The result is OK. It has all the features I asked for, and includes document sharing, collaborative editing in real time, support for fonts and line spacing, etc. etc. I could not have paid a developer $170 and got this. The problem, of course, is that, while abstractly impressive, this is completely useless
Well, what would you expect from a few hours of running in a loop with these constraints?
> This project exists to build a document editor from the ground up. Violating these constraints defeats the entire purpose.
> FORBIDDEN dependencies (do NOT install or use these):
> Rich text editor frameworks: ProseMirror, Slate, Quill, TipTap, Draft.js, CKEditor, TinyMCE, Lexical, or any similar library
> CRDT/OT libraries: Yjs, Automerge, ShareDB, OT.js, or any similar library
> Full CSS frameworks: Bootstrap, Tailwind, Material UI (small utility libs for specific needs are OK)
> ORMs: Prisma, TypeORM, Sequelize (use raw SQL or a thin query builder)
I can't help but wonder what you thought you would achieve, and how getting "mostly what you asked for" is still disappointing to you.
> there is no taste being applied.
There are 0 lines in AGENT_PROMPT.md about "taste". You have instructed something/someone on how to build more than what to build.
Your goals are (from a quick skim):
- The goal of this project is ultimately to generate a working alternative to Google Docs with the same functionality.
- You are an autonomous software engineer building AltDocs, a from-scratch alternative to Google Docs.
I see a FEATURES.md file, but not clear if this is from you or expanded by the model. It seems pretty slim.
All in all, I don't get the "disappointment". It seems, from your blog post, that the "model" did most of the things you asked for. The disappointment might come from what you asked for, more than from the "model" being bad... To paraphrase a line from a sitcom: "Damn, Andrew, I can't control the weather!" :)
> For anything that requires taste and judgment, we've still got a long way to go. There are lots of great demos out there but few if any real examples of vibe coded (or whatever you want to call it) software standing alone as an alternative to project people wrote
Yeah, we still have exactly the same problem as before LLMs/agents, namely that we lack people with "Good Taste". I've outlined how I feel about before (https://emsh.cat/good-taste/) but the TLDR is basically that while LLMs can help you move faster, they won't suddenly mean you'll make better choices, probably the reverse is true, you'll move faster and make worse choices.
Having good taste and knowing how things actually should function is 80% of the work of building good software, and so far all these tools that are trying to replace human choices lead to worse software, and we need more tooling that puts the human and the LLM working together, instead of just outsourcing from the human to the LLM.
This weekend I tried what I'd call a medium scale agentic coding project[0], following what Anthropic demonstrated last week autonomously building a C-compiler [1]. Bottom line is, it's possible to make demos that look good, but it really doesn't work well enough to build software you would actually use. This naturally lends itself to the "everybody is taking about how great it is but nobody is building anything real with it" construct we're in right now. It is great, but also not really useful.
Related, this reminds me of the time Cursor spent millions of dollars worth of tokens to write a new browser with LLMs and ended up with a non-functioning wrapper of existing browser libraries.
Since we're sharing, I have a "claude" command that lets me get quick answers but also saves the conversation and outputs an identifier so in the rare case I want a follow-up, I can ask a question with the ID to continue the conversation.
Neat idea! Although as an identifier, instead of a hash, I'd probably ask it to summarize the conversation into 3 to 7 underscore-separated words and use that as the identifier (plus maybe a timestamp), since a list of them will more easily tell you which is relevant
[1] https://www.marble.onl/posts/this_cost_170.html