One interesting thing is that this makes closed source code even greater asset for the defenders. Attacker cannot spend tokens for it, but defenders can spend tokens for hardening based on source code, while attacker is stuck with blackbox testing.
You would be surprised how adept SOTA models are at reverse engineering with IDA/Ghidra or even plain old objdump. Opus basically knows IDAPython on the back of its hand.
They can be, but the most interesting parts (backend code, deployment confs) are not usually available. Reversing clients can help to understand a bit, but not with equal level.
The CANOS arxiv link says absolutely nothing about AlphaEvolve, Gemini, or LLMs. It seems to use purely traditional ML models. If AE did in fact write a quick script to test different configurations in order to optimize the results, they don't seem to have bothered to write about it.
I can't read the Nature paper about DeepConsensus, but from the summary, it doesn't really explain what role AE had in improving DC. It would be nice to be able to read about what role it actually played, and whether it used traditional or novel methods of performing it
I for one can't tell the difference between Claude and Gemini for coding. And the internal agent tooling is many times faster than Claude Code in my experience.
Lie? Gemini CLI is unuseable. The IF of gemini models is atrocious. Honestly, how often does your gemini CLI go insane in thought loops and you have to stop it?
The people who know what a "child process" is are under no false pretenses about the humanity of the underlying system.
The people who are writing op eds in major news publications about how their favorite chatbot is an "astonishing creature" and how it truly understands them are the ones who need this sort of law.
3.5 MB is pretty generous, actually! Some older TI-84 models had MicroPython running on a secondary ATSAMD21 processor with 32 KB of RAM - that was effectively unusable.
If you write an article shitting on a popular thing because it eclipses the popularity of your favorite thing (and essentially calling the people who use it ignorant sheep), chances are good you are part of a self-fulfilling prophecy, pushing people away from your community.
The article heavily quotes the "AI Security Institute" as a third-party analysis. It was the first I heard of them, so I looked up their about page, and it appears to be primarily people from the AI industry (former Deepmind/OpenAI staff, etc.), with no folks from the security industry mentioned. So while the security landscape is clearly evolving (cf. also Big Sleep and Project Zero), the conclusion of "to harden a system we need to spend more tokens" sounds like yet more AI boosting from a different angle. It raises the question of why no other alternatives (like formal verification) are mentioned in the article or the AISI report.
I wouldn't be surprised if NVIDIA picked up this talking point to sell more GPUs.
I would be interested in which notable security researchers you can find to take the other side of this argument. I don't know anything about the "AI Security Institute", but they're saying something broadly mirrored by security researchers. From what I can see, the "debate" in the actual practitioner community is whether frontier models are merely as big a deal as fuzzing was, or something signficantly bigger. Fuzzing was a profound shift in vulnerability research.
It's less that I think they would take the other side of the argument, than that they would lend some credence to the content of the analysis. For example, I would not particularly trust a bunch of AI researchers to come up with a representative set of CTF tasks, which seems to be the basis of this analysis.
Yeah, you might be right about this particular analysis! The sense I have from talking to people at the labs is that they're really just picking deliberately diverse and high-profile targets to see what the models are capable of.
> but they're saying something broadly mirrored by security researchers.
You might well be right, it is not an area I know much of or work in. But I'm a fan of reliable sources for claims. It is far to easy to make general statements on the internet that appear authorative.
reply