I mean just take a look at twitter to see all the crazy shit people come up with to try to trick models into bypassing the guardrails. Do you honestly believe that if they allowed defensive cybersecurity analysis people wouldnt figure out how to turn that into offensive cybersecurity analysis? I think they just really dont want to be blamed when someone releases malware written by AI
I've had zero issues with defensive secure coding with Grok. It's model is not as powerful as Opus 5.5 but honestly that's fine. I would rather have a decent secure solution than something shiny and insecure. All I can see is Anthropic trying to up-sell a product as if security was a value-add. I disagree with Anthropic, secure code is a mandatory part of software development.
As for people tricking the LLM, instead of blocking people I say empower them and just silently expose malicious behavior to Palantir and PRISM like everyone else does. Want to hack an airline or build a dirty-bomb? Give them 80%-90% of the steps they need and let someone kick down their door. The LLM should not be getting in the way to any of that. Google from day-1 has monitored user intentions and behavior rather than blocking it and I think that is the better path to take (monitoring exposing those that have malicious intentions).
Ok but to me this just means that someone is going to use grok to create malware. I do not think that means anthropic should also let people use their product to create malware. Monitoring interactions isnt exactly foolproof. Google accepts that some CSAM users on their platform will slip through the cracks. The user has already convinced the AI that its doing defensive cybersecurity here. If the claim is that anthropic is more interested in protecting their image than actually reducing cyber crime, sure I agree with that. But Im not sure thats a terrible thing.
Grok will refuse to create any offensive security related code which in my opinion is the right balance. I do not believe Anthropic is about protecting their image [1] but rather up-selling enterprise features.
Also there is no need to trick anything. There are a myriad of abliterated models that will not refuse anything and will help anyone write malicious things or learn to do malicious things.
Grok will try to refuse any offensive cybersecurity work but it’s too easy to disguise offensive work as defensive work. I don’t think it is possible to devise an algorithm that allows defensive work but not offensive work with high certainty.
I guess I don’t see why other models doing bad things means Anthropic should let Claude do bad things too. Anthropic will get blamed if Claude builds malware. They won’t get blamed if grok or deepseek does.
It's called Total Information Awareness. Not swatting, analyzing who is:
- radicalized
- has verifiable intentions
- clever/intelligent enough to pull it off
LLM's would be the perfect tool to filter down the millions of people requesting dumb things into a small group of actual threats. I will take this over federal agents radicalizing people, providing them access to munitions and showing them how to use them and driving them to a building.
LLM's would be the perfect tool to generate a whole bunch of false positives someone's gonna just skim over, click approve, and get innocents murdered because they were trying to get their plutonium processing working in the game Satisfactory.
Imagine if in 2022, we restricted educational cybersecurity resources because we were concerned that more people who knew how to program in this domain would lead to more malware.
I mean, I'm sure that's actually true in a sense, but it would also mean fewer people who could fix vulnerabilities.
