Consistency Amplifies: How Behavioral Variance Shapes Agent Accuracy
arXiv:2603.25764v1 Announce Type: new Abstract: As LLM-based agents are deployed in production systems, understanding their behavioral consistency (whether they produce similar action sequences when given identical tasks) becomes critical for reliability. We study consistency in the context of SWE-bench, a challenging software engineering benchmark requiring complex, multi-step reasoning. Comparing Claude~4.5~Sonnet, GPT-5, and Llama-3.1-70B across 50 runs each (10 tasks $times$ 5 runs), we find that across models, higher consistency aligns with higher accuracy: Claude achieves the lowest variance […]