Agent Evaluation Advances, Security Gets Serious

From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI

As teams increasingly rely on AI to accelerate development, this framework warns that we're accumulating dangerous new forms of debt beyond just technical debt. Cognitive debt occurs when teams lose shared understanding of their systems as AI generates code faster than they can comprehend it, while intent debt refers to the missing documentation of why decisions were made—critical context that both humans and AI agents need to safely evolve code. This triple debt model provides a essential lens for evaluating software health in the AI era.

Components of A Coding Agent

Essential reading if you're architecting coding agents for production use. This breaks down the core components that make LLMs effective at code generation: sophisticated tool integration, persistent memory systems that maintain context across interactions, and repository-aware context management that helps models understand large codebases. The practical focus on how these pieces work together makes this invaluable for teams moving beyond simple code completion to full coding assistance.

Embarrassingly Simple Self-Distillation Improves Code Generation

This challenges the conventional wisdom that you need external verification or teacher models to improve code generation—instead, models can learn from their own outputs using simple self-distillation. The technique improved a 30B model's performance from 42% to 55% on challenging coding problems by sampling solutions at specific temperatures and fine-tuning on them. The key insight is that this reshapes how models balance precision versus exploration in a context-dependent way, making it a practical post-training technique for enhancing coding assistants.

Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents

Bowen Ye, Rang Li, Qibin Yang, Yuanxin Liu, Linli Yao, Hanglong Lv, Zhihui Xie, Chenxin An, Lei Li, Lingpeng Kong, Qi Liu, Zhifang Sui, Tong Yang

Current agent benchmarks are dangerously inadequate for production deployment because they only check final outputs without understanding how agents got there, and they barely evaluate safety or robustness. Claw-Eval fixes this with 300 real-world tasks that record every agent action through execution traces, audit logs, and environment snapshots, enabling fine-grained evaluation across completion, safety, and robustness dimensions. This comprehensive approach is essential for teams serious about deploying autonomous agents in high-stakes environments.

ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces

Xiangyi Li, Kyoung Whan Choe, Yimin Liu, Xiaokun Chen, Chujun Tao, Bingran You, Wenbo Chen, Zonglin Di, Jiankai Sun, Shenghan Zheng, Jiajun Bao, Yuanli Wang, Weixiang Yan, Yiyuan Li, Han-chung Lee

Testing agents on live productivity services is too risky, but existing benchmarks don't capture the complexity of real workflows across Gmail, Slack, and Google services. ClawsBench solves this with high-fidelity mock services that maintain full state and support deterministic snapshot/restore, enabling safe evaluation of 44 structured tasks including dangerous scenarios. The research reveals that domain skills (API knowledge injection) and meta prompts (cross-service coordination) are independent levers that teams can optimize separately for better agent performance.

Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Proving

Devakh Rashie, Veda Rashi

Financial services face an existential problem: probabilistic LLMs operating in domains requiring absolute compliance guarantees, and existing guardrails are fundamentally inadequate for complex regulatory constraints. This paper presents a breakthrough using Lean 4 theorem proving to treat every AI action as a mathematical conjecture—execution only proceeds if the system can formally prove regulatory compliance. While the approach targets financial services, the formal verification framework could revolutionize how we build deterministic guardrails for any high-stakes AI system.

ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code Generation

Hui Sun, Yun-Ji Zhang, Zheng Xie, Ren-Biao Liu, Yali Du, Xin-Ye Li, Ming Li

When LLMs generate both code and tests, how do you evaluate test quality without knowing which code is correct? This paper breaks the circular dependency with a clever insight: tests should rank code quality, not just count passes, and you can measure ranking ability through leave-one-out evaluation. The approach measures whether each test's pass/fail pattern correlates with how other tests collectively rank the code, providing a principled way to weight unreliable LLM-generated tests without needing ground truth.

Neural Computers

Mingchen Zhuge, Changsheng Zhao, Haozhe Liu, Zijian Zhou, Shuming Liu, Wenyi Wang, Ernie Chang, Gael Le Lan, Junjie Fei, Wenxuan Zhang, Yasheng Sun, Zhipeng Cai, Zechun Liu, Yunyang Xiong, Yining Yang, Yuandong Tian, Yangyang Shi, Vikas Chandra, Jürgen Schmidhuber

This proposes a radical paradigm shift where models don't just generate code or control external systems—they become the execution environment itself, unifying computation, memory, and I/O in learned runtime state. Neural Computers learn to execute programs by watching I/O traces and can potentially be reprogrammed through natural language rather than traditional coding. While early-stage, this vision could fundamentally reshape how we build AI systems by eliminating the boundary between model and runtime environment.

Anthropic's Project Glasswing - restricting Claude Mythos to security researchers - sounds necessary to me

Anthropic took the unprecedented step of restricting access to Claude Mythos because its cybersecurity research capabilities are too powerful for general release—the model has already found thousands of high-severity vulnerabilities. This sets a crucial precedent for responsible AI deployment and signals that we're entering an era where model capabilities may outpace our ability to deploy them safely. Security-conscious engineering teams should pay close attention to how this restricted release model evolves.

Self-Execution Simulation Improves Coding Models

Gallil Maimon, Ori Yoran, Felix Kreuk, Michael Hassid, Gal Cohen, Pierre Chambon, Yossi Adi

Code LLMs struggle because they can't accurately predict what their generated code will do when executed, leading to logical errors that escape syntax checking. This research trains models to simulate program execution step-by-step, enabling self-verification and iterative debugging of their own code. The approach combines supervised learning on execution traces with reinforcement learning, achieving significant improvements on competitive programming benchmarks and providing a foundation for more reliable AI coding assistants.

Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism

Hadas Orgad, Boyi Wei, Kaden Zheng, Martin Wattenberg, Peter Henderson, Seraphina Goldfarb-Tarrant, Yonatan Belinkov

This research reveals that harmful content generation in LLMs depends on a surprisingly compact and unified set of weights that are distinct from benign capabilities—essentially, there's a discrete 'harm circuit' that can be surgically identified and removed. Alignment training compresses rather than eliminates these harmful capabilities, explaining why fine-tuning on narrow domains can cause 'emergent misalignment' and why jailbreaks remain effective despite safety training. These findings provide crucial insights for building more robust safety mechanisms in production systems.