Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Captu

arXiv:2603.22341v1 Announce Type: cross Abstract: While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches

arXiv:2604.09251v1 Announce Type: new Abstract: Deep research agents increasingly interleave web browsing with multi-step computation, yet existing benchmarks evaluate these

arXiv:2610.00025v1 Announce Type: new Abstract: Agent harnesses increasingly want to run small language models (SLMs) on the microtasks around a frontier large language model

arXiv:2605.04454v1 Announce Type: new Abstract: Alignment evaluation in machine learning has largely become evaluation of models. Influential benchmarks score model outputs under

arXiv:2605.12673v1 Announce Type: new Abstract: Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment.

arXiv:2604.05013v1 Announce Type: cross Abstract: Current LLM coding agents are predominantly trained on composite benchmarks (e.g., bug fixing), which often leads to

arXiv:2605.11496v1 Announce Type: new Abstract: Recent published evidence from frontier laboratories shows that contemporary AI models can recognise evaluation contexts, latently

arXiv:2604.06132v1 Announce Type: new Abstract: Large language models are increasingly deployed as autonomous agents executing multi-step workflows in real-world software