All that keeps jumping out at me is how they've set it to refuse giving users thinking tokens and prompts for full reasoning in output. Just drives me further away; I may not stop using Claude completely for now, but I'll be moving even more of my primary workload to Chinese providers. That's where openness and freedom is now at.
> All that keeps jumping out at me is how they've set it to refuse giving users thinking tokens and prompts for full reasoning in output
I keep seeing comments added to code, which reads like reasoning output instead of meaningful words. I see this behavior for both OpenAI and Anthropic models (for several harnesses as well).
But this is a sample of one. And I may be in a situation where I'm more negative to the output from LLMs in general.
Yeah GPT 5.6 models did it a lot and Opus is absolutely awful on this. It's clearly encoding it's thinking/context into the comments. GPT-6 models seem to be better about it.
Are we talking, like, a ton of verbose comments? Because “what were you thinking here?” is kinda-sorta what I want in comments. As opposed to the old “telling me what I can already see.”
What Chinese models/providers are you using for this? I'm hitting Claude's weekly limits much sooner than I used to with roughly the same workload, so I'm interested in trying alternatives, especially ones with strong coding/agentic performance.
While it's a common tactic, and I'd vouch for it if you don't really know what you want to code in fact, but if you know what you want to get out of it, I haven't found anything I'd need opus 5.5 for instead of deepseek-flash (flash v4.1 hosted via platform.deepseek.com)
So you don't want the model to be better yet you continue to reuse it?... I find this kind of selfish behaviour increasing within us developers, like a fear of avoiding the inevitable
Do you use Mistral's models to help improve the European offering?
Otherwise if you don't care about your data being used in training you might as well use the best, which is the US models on a subscription plan. If you do care, use downloadable weight models served by reputable providers, or selfhost.
In my experience that tactic works well if the codebase is limited in size, or well maintained and separated. Otherwise I do notice a difference also letting fable do the execution, not just the planning for complex tasks.
In my experience it never works well on any real work. In fact, I'd go the opposite, plan with the dumb model and execute with the smart model because at least the model writing the code and solving the emergent problems is capable.
In my experience (and I've been trying this a bunch): smart planner + dumb executor produces worse code with higher spend than simply using the smart planner to do both.
It's easy to understand why:
- If the planner has truly thought the issue through, properly designed the solution, solved all of the emergent problems, then the final "write" of the code is just a few more output tokens.
- If the planner has NOT truly planned the issue completely, then you're letting a substantially dumber and less capable model make significant decisions, and trusting its problem solving, without having a better model check it.
If you're highly cost conscious (paying for your own tokens and not making any money) then you have no choice but to trade your time and effort for tricks like this to save money by lowering the quality of your output.
But if your employer is paying for tokens: just use the smarter model. You save your time preventing re-work and reducing code review, you save your employer money (primarily from the cost of your own labor and reduced rework), and you get a better output every time (Opus 5.5 mogs Deepseek 4.1 flash in every single way except cost).
The cost is 20-40x less for Deepseek Flash v4.1. If you are just comparing to Sonnet or you aren't paying (your case) then your advice makes perfect sense.
I also agree that its a big mistake to have a flash model implement without a strong model reviewing.
I have Opus plan, Deepseek implement the code, and then review with Opus [1]. In this workflow I am saving a lot of money by having Deepseek do the implementation. Note that the review back-and-forth is fully automated [2], so it doesn't take any extra attention from me.
> If you are just comparing to Sonnet or you aren't paying (your case) then your advice makes perfect sense.
Also if you aren't hitting capacity.
Off-work, I use LLMs regularly for both design/coding and non-technical work, but the volume is not enough to trip the weekly limits, and rarely enough to trip the daily limits. So I just go with whatever's current best SOTA available on my Claude & ChatGPT subscriptions and don't worry about limits. If I hit one, I do some household stuff or relax for a few hours (or just turn in for the day), and then the limit is refreshed.
Deepseek Flash v4.1 is only "40X cheaper" if you do not account for the time of the engineer reading the output. If Opus 5.5 high requires 1/2 of the actual engineer time, and the engineer costs $100-$200/hr, then Deepseek v4.1 is actually the more expensive model to use.
I have tried your workflow many times, and simply letting Opus do the implementation costs much less than wasting hundreds of millions of tokens letting deepseek and opus go back and forth and back and forth. And bonus, my project finishes in 5 minutes instead of 20.
I tried the experiment of reviewing vs. not reviewing with frontier models. I consistently found that reviewing by a model with independent context finds important issues when changes are non-trivial- certainly the definition of non-trivial is getting raise as the models get better.
I do have an /implement-simple workflow to skip the planning phase, but even that doesn't skip the review.
Are you doing your own intensive reviews of the model code? Can you share the prompts you are using as I have?
My bar for what models produce without human intervention is much lower defect than what a human would produce. The human interaction is mostly to guide the design and then the review burden is very low. I suspect your bar for what agents produce is lower- you are taking more of the review burden. I also suspect that you are measuring time more than actual cost since your employer is paying and that you are comparing to Sonnet rather than DeepSeek (DeepSeek 4.1 again is 20-40x cheaper than Sonnet). You mention hundreds of millions of tokens (my reviews don't use that much), but even that costs ~$1 on the DeepSeek side.
I think you are taking exactly the right approach at your employer given the cost is free and you only have access to Anthropic models.
I saw your response before it was deleted- that you are doing multi agent persona reviews and a very intensive review process. So having fewer review items saves you money.
One thing that I have found is that as the frontier models get better there is less need for agents with specialized personas. I actually don't don't use those anymore- I just use agents that have different models and reasoning levels. I have a generated CODING_STANDARDS.md document and a skill for architecture design and a skill for implementing testing [2] that are referenced by a single reviewer. I do implement a 2-pass review though [3].
I would be interested to know if you have found anything similar as models get better. It seems though that you are sharing a single exploration and then sharing the context across the specialized reviewers to dramatically reduce the cost of your approach. Does this have to be in the harness- that is if you write out the shared context to a file does that increase your costs a lot?
I also wonder how intensively are the models able to test their changes? The number one quality improvement I have found is not review but having the model properly test its code. I have a skill that is helping [4], but I also have to spend time to establish a pattern of testing with tools beyond just unit tests. The testing takes significant effort, and this is again where the cost savings of DeepSeek shine.
Bingo, DeepSeek (v4.1) is horribly overhyped. In all my personal benchmarks it sits below Glm5.3 Flash. Waaaay below Qwen3.8-Flash-Next a model less than half it's size.
No, the only open weight model that really makes sense for me is Qwen3.8-Flash-Next, but it is mainly because I can run it locally with reasonable speed (prefill between 650-1400t/s generation between 22-50t/s depending on number of slots/users I configure).
This is the first model that truly competes with Opus 4.8. I'd say it may be better than Opus 4.6 on programming.
But it is very verbose when it comes to reasoning tokens. The more difficult the task the more verbose it is. Certain very hard tasks that take opus 4.8 400k tokens take Qwen3.8-Flash-Next 2M tokens... But it finishes them.
And what you loose on the generation speed you get back on input caching you can keep on for weeks.
> If the planner has truly thought the issue through, properly designed the solution, solved all of the emergent problems, then the final "write" of the code is just a few more output tokens.
For me, what usually creates the high cost of implementation with a larger model is the validation process, not the writing of code.
I would get excited to see it finish writing code with so little usage, but then it would gobble up ten times as many tokens on validation.
I've tried to instruct it to keep the validation light with the intention of doing batches of deep validation after a few tasks, but it couldn't stop itself from doing heavy validation on each task no matter how I rephrased the instructions.
That's how the cycles happen. China realizes they could use a little more freedom and US realizes that they could do with a little less. The emerging/shrinking middle class of both countries also moves the sweet spot.
Doesn't make it any less amusing from the outside, to see the US struggle with their identity. (It's most always just a struggle when freedom becomes less)
Seat-based pricing just includes a certain amount of token usage at a discount for buying in "bulk" (and risking not using all your usage). You can still pay the API token-based rates if you really want to; they won't stop you from doing that.
they have started to squeeze, with gpt-6 i'm getting waaaay less value out of the subscription. Used to be thousands of dollars a reset and it's down to a few hundred
They may have better and more open weight models but they sure don't have our western understanding of individual freedom. Go try out their first and second amendment protections, or try the fifth? I'm sure we can find more but mostly, when the state needs the tech there won't be an Anthropic-like appeal against overstepping.
As a paying user, I expect to get what I'm paying for. I'm paying for thinking tokens, so I should be getting them, and in a way that I can actually read if/when I want without relying on any proprietary tools. I have no interest in being locked in.
Noobs are burning tokens like "make me an app that does this" whereas an experienced engineer would go with certain language, framework and architecture in mind.
How exactly does specifying the language, framework, and architecture in advance save a meaningful amount of tokens? I'd expect saying "make me an app" and "make me an app using Swift and SwiftUI" would be pretty close in terms of token usage. You save maybe one look up by the LLM for "what is the preferred language for writing an application for iOS?".
That's obviously pretty specific to iOS - or MacOS - where there's a single blessed path. Elsewhere, especially in web dev, unless you really don't care about what you get or are making something very simple, you better be ready to provide specifics.
Even still, the majority of things being built on the web perform largely the same if it's being built in Ruby or Rust or Node or Go. Only very niche things, that an LLM would probably fumble over anyway, really benefit from picking the perfect language/framework. The only real advantage to naming your language in the initial prompt is that you can be assured that you'll be able to understand the code when the GPU spins down and output is in front of you.
This should always be a goal. Doing anything major without being able to review manually is just asking for pain over time, or be ready to feed more and more tokens to the fire to reduce sloppiness.
Now this prompt has huge variety of implementation details. Language? PHP/Ruby/Python/Java/Typescript? In each of them then there are tons of frameworks, templating engines, ORMs, database servers, frontend tooling, bundler, frontend framework alone has several dozen candidates from React, Preact, Vue, Svelte and what not.
So if you really know your craft, you'll already be knowing what specific implementation you need so let us not discount the existing expertise here.
