I’ve seen agents do well on structured evals but fall apart once the environment becomes less constrained (messy files, ambiguous tasks, partial failures, etc.).
I’ve seen agents do well on structured evals but fall apart once the environment becomes less constrained (messy files, ambiguous tasks, partial failures, etc.).