In my experience the biggest multiplier isn't any single variable it's the interaction between them. Fanout × retries × context growth compounds in ways that linear cost models completely miss.
The fix that worked for us: treat budget as a hard constraint, not a target. When you're approaching limit, degrade gracefully (shorter context, fewer tool calls, fallback to smaller model) rather than letting costs explode and cleaning up later.
Also worth tracking: the 90th percentile request often costs 10x the median. A handful of pathological queries can dominate your bill. Capping max tokens per request is crude but effective.
+1 on interaction terms + tails : fanout × retries × context growth is where linear token math dies.
One thing we do in enzu is make “budget as constraint” executable: we clamp `max_output_tokens` from the budget before the call, and in multi-step/RLM runs we adapt output caps downward as the budget depletes (so it naturally gets shorter/cheaper instead of spiraling). When token counting is unavailable we explicitly enter a “budget degraded” mode rather than pretending estimates are exact.
Also agree p90/p95 cost/run matters more than averages; max-output caps are crude but effective.
The phrase "uncontrolled human experiment" is doing interesting rhetorical work here. It frames the status quo as the experiment and regulation as the control—when historically it's been the reverse.
I’d say status quo is before social media as that’s where most childhoods have happened. Targeting children with social media is definitely a new thing and still an experiment since those poor souls that had their lives surveilled by Meta are just coming of age and we’re just learning about the damages.
The fact that you're the only one calling this out is quite frankly alarming.
It's one of the most authoritarian statements I've ever heard from a western government. And just because its the trendy moral panic of the day, everybody is cheering it on.
Anything where we allow people free will is by definition an "uncontrolled human experiment" and the basis of any free society.
Should we also end the "uncontrolled human experiment" of allowing people to have private money and make their own purchase decisions? Should we end the "uncontrolled human experiment" of allowing people to select their own romantic partners?
You mention free will, but it would be an interesting conversation to discuss how much free will there is when people fight, without knowing it, against psychological manipulation, addition creation, all crafted with plenty of money and research. Especially when this people are young ones.
Can the behavior still be called free will when you get hooked to mechanisms specifically designed to exploit humans in this way?
Your rebuttal is based on the assumption that social media is uniquely manipulative, addictive and designed by a secret evil cabal of geniuses with top hats.
This is a media narrative and a popular meme, but it is not reality when looking at the actual studies:
The idea that social media is uniquely bad is simply the same moral panic that comes up every time we encounter something new. The truth is every person, company and government engages in "psychological manipulation" when trying to get what they want, including the children themselves.
Since other children engage in psychological manipulation with each other at school, should we ban schools to protect kids from it? Since the government engages in psychological manipulation with its policies and agendas (all backed by billions of dollars), should we ban government policy-making for children?
The fix that worked for us: treat budget as a hard constraint, not a target. When you're approaching limit, degrade gracefully (shorter context, fewer tool calls, fallback to smaller model) rather than letting costs explode and cleaning up later.
Also worth tracking: the 90th percentile request often costs 10x the median. A handful of pathological queries can dominate your bill. Capping max tokens per request is crude but effective.