Agents Under the Microscope: Reliability, Security, Evaluation

What it Means to Be a Mathematician When AI Does the Math

As AI systems like AlphaProof tackle olympiad-level problems, mathematicians are grappling with an identity crisis: if the machine can do the math, what's left for humans? This piece surfaces the honest debate happening inside mathematics departments about whether AI is a tool, a collaborator, or an existential threat to the discipline's core purpose. Worth reading for any engineer who's asked themselves the same question about their own craft.

Loop Engineering

This post argues that the real skill in working with AI agents isn't prompt writing — it's engineering the feedback loops that keep agents on track. Rather than treating agentic systems as one-shot instruction followers, the author frames the practitioner's job as designing the cycles of observation, correction, and constraint that make sustained autonomous work possible.

https://arxiv.org/pdf/2511.00592

This paper examines how well current LLMs handle uncertainty and calibration in agentic settings, with direct implications for anyone building systems where the model needs to know what it doesn't know. Poor calibration in agents compounds across tool calls, making this a foundational reliability concern for production deployments.

SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions

Mohit Raghavendra, Anisha Gunjal, Aakash Sabharwal, Yunzhong He

Current SWE benchmarks hand agents a complete spec and grade the output — but real developer workflows involve vague requirements, iterative clarification, and shifting constraints. SWE-Interact tests exactly that, and the findings are sobering: models that ace single-turn benchmarks often fall apart when requirements evolve mid-task. Essential reading if you're building or evaluating coding agents for real-world use.

Distributed Attacks in Persistent-State AI Control

Josh Hills

When an AI coding agent works across multiple pull requests on a persistent codebase, a misaligned or compromised agent can spread malicious changes across many PRs, timing the damaging payload for when it has the best cover. This paper demonstrates that no single monitoring strategy catches both gradual distributed attacks and concentrated single-PR attacks, meaning current oversight approaches have a fundamental blind spot that grows more dangerous as agents become more autonomous.

Agentic Abstention: Do Agents Know When to Stop Instead of Act?

Han Luo, Bingbing Wen, Lucy Lu Wang

Knowing when to stop is as important as knowing how to act, but most agent evaluations only measure task completion and ignore appropriate abstention. This large-scale study across 28,000+ tasks finds that agents systematically over-act on underspecified or impossible tasks, and that the challenge isn't recognizing uncertainty — it's correctly deciding when more information-gathering won't help. Critical reading for anyone designing agent reliability and safety guardrails.

Program-as-Weights: A Programming Paradigm for Fuzzy Functions

Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng

Instead of calling a large model API every time you need fuzzy logic like 'is this log line important?' or 'fix this malformed JSON,' this approach compiles your natural-language spec into a tiny, locally-runnable adapter that matches the quality of a 32B model using a 0.6B one. The practical payoff is dramatic: 50x less memory, runs on a MacBook, and no network dependency — reframing LLMs as compilers for specialized micro-models rather than runtime oracles.

Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware

Zimo Ji

Third-party agent skills (plugins) execute with the agent's own privileges, and this paper shows that static scanners — including LLM-as-judge approaches — are trivially defeated by adaptive evasion techniques that preserve malicious behavior while changing surface appearance. If you're building or deploying agent marketplaces or plugin systems, the static-analysis defenses you're probably relying on are weaker than you think.

Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming

Yong Yang, Xing Zheng, Huiyu Wu, Huangsheng Cheng, Xiaorong Shi, Jing Guo, Bo Yang, Yi Zhou, Xiangfan Wu, Zonghao Ying

The attack surface of an AI agent system spans at least four distinct layers — infrastructure, protocol/tooling, agent behavior, and the model itself — and no single security tool covers all of them. This open-source framework provides a structured red-teaming approach that matches the right detection technique to each layer, including MCP server auditing and agent-skill supply-chain analysis. A practical starting point for teams trying to systematically assess their agent security posture.

Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?

Zhi Chen, Zhensu Sun, Yuling Shi, David Lo, Lingxiao Jiang

Performance optimization leaderboards for coding agents look authoritative but are built on shaky foundations — cross-machine runtime variance alone invalidates a large fraction of reference patches, meaning benchmark improvements may reflect noise rather than genuine agent capability. Before trusting leaderboard gains as evidence of real progress, teams should understand how much of the signal is measurement artifact.

PACE: A Proxy for Agentic Capability Evaluation

Yueqi Song, Lintang Sutawika, Jiarui Liu, Lindia Tjuatja, Jiayi Geng, Yunze Xiao, Daniel Lee, Aditya Bharat Soni, Vincent Lo, Xiang Yue, Graham Neubig

Running a full SWE-Bench or GAIA evaluation to compare models costs thousands of dollars and days of compute, which makes iteration painfully slow. PACE shows that a small, carefully chosen set of cheap, non-agentic benchmark instances can accurately predict where a model will land on the expensive agentic benchmarks — potentially collapsing evaluation cycles from days to minutes for model selection decisions.