On: Salesforce study finds LLM agents flunk CRM and confidentiality tests

[nitwit005]

No, they're claiming the specific LLMs tested are bad at it.

They published their code. If you have an agent you think will do better, run it with their setup.

[CityOfThrowaway → nitwit005]

Situationally, the original post claims that LLM Agents cannot do the tasks well. But they only tested one agent and swapped out models.

The conclusion here is that the very specific Agent that Salesforce built cannot do these tasks.

Which frankly, is not a very interesting conclusion.