The pilot analogy hits different when you consider that pilots still train on simulators for exactly this reason — they're legally required to maintain proficiency even when autopilot handles 99% of flights.
There's no equivalent mandate for software engineers. Nothing stops you from spending years as a pure "prompt pilot" and losing the ability to read a stack trace or reason about algorithmic complexity. The atrophy is silent and gradual.
The author's suggestion to write code by hand as an educational exercise is right but will be ignored by most, because the feedback loop for skill atrophy is so delayed. You won't notice you've lost the skill until you're debugging something the agent made a mess of, under pressure, with no fallback.
The term "Children of the Magenta Line" has long been used in aviation to describe the over-reliance on automation. So even though they train to avoid losing manual skills, it's definitely still a concern.
The proxy approach for secret injection is the right mental model, but it only works if the proxy itself is hardened against prompt injection. An agent that can't access secrets directly can still be manipulated into crafting requests that leak data through side channels — URL params, timing, error messages.
The deeper issue: most of these guardrails assume the threat is accidental (agent goes off the rails) rather than adversarial (something in the agent's context is actively trying to manipulate it). Time-boxed domain whitelists help with the latter but the audit loop at session end is still reactive.
The /revert snapshot idea is underrated though. Reversibility should be the first constraint, not an afterthought.
> but it only works if the proxy itself is hardened against prompt injection.
Yes, I'm experimenting using a small model like Haiku to double check if the request looks good. It adds quite a bit of latency but it might be the right approach.
Honestly; it's still pretty much like early days of self driving cars. You can see the car can go without you supervising it but still you need to keep an eye on where it's going.
this is a good reminder that local session state is basically undocumented infrastructure at this point. the fact that people are building recovery tools around ~/.claude logs says something about how much we're relying on these agents for real work now. would love to see anthropic treat this as first-class — proper session persistence, not just forensic recovery after the fact
the trust problem cuts both ways tho — users don't trust agents, but the bigger issue is agents trusting each other. once you have multi-agent pipelines, you're one rogue upstream output away from a cascade. sandboxing individual agents is table stakes; what's actually hard is defining trust boundaries between them
On memory: You're in luck, we do support conversation history and we're actively working on improvements like lazy loading for long conversations.
Best practices: Spot on! The project docs have tips for building assistants that hide the underlying LLM ( Feature: Assistant personas - https://docs.nlkit.com/nlux/examples/assistant-persona ). Plus, docs cover LangChain integration (supports any AI backend!), Next.js, HuggingFace (access to thousands of models at your fingertips), and any custom backend.
I'd like to see them do this with Backpack, their personal information manager/personal wiki/personal note taking tool. It was a great product for its time.
While dependencies can provide convenient functionality, the JS ecosystem has tended towards excessive granularity. Many packages contain limited code yet still require a large number of dependencies .. sometimes because of tooling (as in the example in the post here), but quite often out of convenience!
Rather than hastily installing the latest shiny package, we should refactor and build reusable minimal utilities tailored to our domain.
There's no equivalent mandate for software engineers. Nothing stops you from spending years as a pure "prompt pilot" and losing the ability to read a stack trace or reason about algorithmic complexity. The atrophy is silent and gradual.
The author's suggestion to write code by hand as an educational exercise is right but will be ignored by most, because the feedback loop for skill atrophy is so delayed. You won't notice you've lost the skill until you're debugging something the agent made a mess of, under pressure, with no fallback.