Edition

Agents at Scale, Security Cracks Widen

August 24, 2026 · 12 papers

This week's edition is heavily weighted toward agent reliability in practice: three pieces make the case that harness engineering and prompting strategy — not raw model capability — are the primary levers for production-grade agent performance, backed by a 717k-line refactoring case study, a Terminal-Bench benchmark result achieved at 38x lower cost, and a comprehensive system-engineering monograph. On the security front, the news is sobering: context leakage worsens with more capable models, reasoning traces can be extracted even when obscured, and prompt injection has now shown up in a US court filing. Rounding out the edition are Terence Tao on what AI means for knowledge work, a MitM teardown of GitHub Copilot's context assembly, and a note from Anthropic on structuring system prompts for production Claude deployments.

05 · agents Intermediate

StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling

Ziheng Qin, Yaxin Lu, Zhangyang Atlas Wang, Kai Wang

StateM demonstrates that you can beat frontier model performance on long-horizon agent benchmarks not by swapping in a better model, but by engineering a better runtime around the same model. By organizing agent execution around durable states, phase-local context, checked transitions, and versioned runbooks, StateM pushes GPT-5.5 past GPT-5.6 Sol Ultra on Terminal-Bench 2.1 — and achieves 95.3% accuracy at roughly $15 in API costs versus $575 for the reference run. This is a strong empirical argument that harness engineering is currently one of the highest-ROI investments in agent development.

Takeaways3

Harness scaling — improving the execution runtime without touching model weights — can outperform simply upgrading to a more expensive frontier model.
Durable state management and structured runbooks directly address the most common long-horizon agent failure modes: lost context, skipped procedures, and premature stopping.
The same runbook structure transferred across multiple models (GPT-5.5, GPT-5.6, DeepSeek) with minimal adaptation, suggesting these runtime patterns are model-agnostic.

437 upvotes on HF · via api-hf · arXiv:2608.15089

01 · prompt-engineering Accessible

The Gauntlet Loop: The Prompting Method Behind Claude of Duty

The author behind a viral Claude-generated Call of Duty clone explains the prompting technique that made it possible — and why most people get mediocre results from AI agents. The core insight is the 'Gauntlet Loop': instead of accepting the agent's first output, you force it into repeated self-comparison cycles against a quality bar, driving iterative improvement without human steering. This challenges the assumption that model capability is the bottleneck; often, it's the prompting strategy that determines whether you get a toy demo or 55,000 lines of working game code.

Takeaways3

Forcing an agent to repeatedly compare its own output against a quality standard — rather than accepting its first result — dramatically raises output quality.
Model capability is often not the limiting factor; prompting strategy and loop structure matter more than most practitioners assume.
A single well-structured prompt with a built-in iteration mechanism can replace hours of manual steering and review.

prompt-engineering agents how-we-work software-engineering

via suggestion

02 · software-engineering Intermediate

Specification-first convergence with an AI coding agent: a case study of dismantling a core architectural invariant across 189 files in a 717k-line codebase with no test oracle and no human code review

Joel Abenhaim

This case study is essential reading if you've ever dismissed AI agents as unsuitable for large-scale, high-stakes refactoring. An AI coding agent successfully dismantled a core architectural invariant across 189 files in a 717k-line TypeScript codebase — a change the author considered effectively infeasible via incremental refactoring — with no human code review and no pre-existing test oracle. The key was a rigorous 'specification-first' protocol: the agent formally specified the target behavior, audited that spec against the source across 14 refinement cycles, implemented atomically, then ran 17 verification cycles against the frozen spec, correcting 201 defects before any human ran the program.

Takeaways3

A formal specification-first protocol — where the agent writes, refines, and verifies against its own spec — can substitute for a human reviewer even on complex architectural changes.
Convergence criteria matter: requiring two consecutive zero-finding verification passes gives you a principled stopping condition instead of relying on gut feel.
AI agents may be most valuable precisely for the 'effectively infeasible' refactors that humans would otherwise punt to a full rewrite.

software-engineering agents how-we-work evaluations

10 upvotes on HF · via api-hf · arXiv:2608.12440

03 · agents Intermediate

Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model

Stephanie Jarmak

This monograph makes a case that most AI coding agent failures aren't model failures — they're system failures in the harness, execution environment, retrieval, state management, or observability layers. Drawing on 164 scholarly works and 100 practitioner records, it builds a dependency-chain framework for evaluating and operating agents reliably, arguing that improvements at one layer routinely fail to propagate to end-to-end outcomes. If you're building or operating coding agents in production, this is the closest thing to a comprehensive engineering handbook the field currently has.

Takeaways3

Many apparent model capability gaps are actually harness or infrastructure problems — fixing the system around the model often matters more than upgrading the model itself.
Evaluation and operation should be treated as a dependency chain: weaknesses in task construction, retrieval, or verification can silently invalidate your benchmark conclusions.
The monograph's catalog of 193 gated practices provides a concrete checklist for diagnosing and hardening agent reliability at each layer of the stack.

agents software-engineering evaluations how-we-work

via api-arxiv · arXiv:2608.13867

04 · software-engineering Intermediate

What I learned by putting GitHub Copilot behind a MitM proxy

j0selit0

By routing GitHub Copilot's traffic through a MitM proxy, the author reverse-engineers exactly how Copilot constructs context, manages memory, and decides what to send to the model — giving practitioners a rare ground-truth view of how a production AI coding tool actually works under the hood. The findings reveal that context assembly is increasingly 'the product': what gets included, chunked, and prioritized in the prompt is where the real engineering leverage lives. This is a practical teardown that will change how you think about building or evaluating any IDE-integrated AI tool.

Takeaways3

Context construction — what gets selected and how it's assembled before hitting the model — is the primary differentiator between AI coding tools, not the underlying model.
Inspecting network traffic is a surprisingly accessible way to audit what any AI-powered desktop tool is actually doing with your code and credentials.
Understanding the harness around the model (retrieval, chunking, memory) is more actionable for practitioners than obsessing over model benchmarks.

software-engineering agents how-we-work security

200 points on HN · via api-hn

06 · llms Intermediate

Models Are Getting Dumber on Purpose

hruvhwe

This blog post challenges the assumption that benchmark improvements translate to general model capability gains. The author argues that while models are achieving stunning results on math and coding benchmarks with far fewer active parameters than before, they're simultaneously getting worse at basic factual recall — and this is a deliberate tradeoff. The implication for practitioners is significant: if you're building RAG systems or agents that rely on factual grounding, you can't assume a high benchmark score means the model will be reliable on your use case.

Takeaways3

Benchmark scores on math/code are skyrocketing, but factual recall (SimpleQA) is declining — models are being optimized for one at the expense of the other.
Smaller quantized models (e.g., Qwen3.5 9B in 6GB VRAM) are now competitive with models that required massive infrastructure just two years ago.
For production systems, always evaluate models on the specific capability your app depends on — general benchmark rankings can actively mislead you.

llms opinion reasoning rag

326 points on HN · via api-hn

07 · opinion Accessible

Mathematics in the age of AI

jonbaer

Terence Tao — one of the greatest living mathematicians — sidesteps the tired 'can AI do real math?' debate and asks a more interesting question: what is mathematics actually *for*, and how should the field respond if AI can handle the problem-solving parts? This is essential reading for engineers building AI tools for knowledge workers, because Tao's framework for separating the mechanical from the meaningful applies well beyond mathematics. His answer reframes AI not as a replacement for human expertise but as a shift in what human expertise should focus on.

Takeaways3

Tao argues the more important question isn't whether AI can do research-level math, but what the goals of mathematical research are beyond problem-solving.
The essay is a model for how any knowledge discipline should think about AI augmentation — separating rote execution from deeper intellectual value.
For AI tooling builders, this is a reminder that automating the measurable parts of a domain doesn't capture what practitioners actually care about.

opinion foundational reasoning

surfaced by 2 sources · 208 points on HN · via api-hn · arXiv:2608.16753

08 · how-we-work Accessible

Building an (almost) fully self-hosted, sandboxed, agentic software factory

jakelsaunders94

This is a practical, hands-on account of building a fully autonomous software development pipeline — from a single prompt to a deployed app with CI, Postgres, HTTPS, and observability — without giving an LLM root access to a local machine. The author's core insight is that structural containment (sandboxing, remote environments) is more trustworthy than behavioral trust in the model. If you're thinking about agentic coding pipelines in production, this is a concrete reference architecture worth studying.

Takeaways3

Structural sandboxing (remote, isolated environments) is a more robust security model for agentic coding than relying on the LLM to behave safely.
A single prompt can now drive a full SDLC — planning, coding, testing, CI, and deployment — with current tooling if the environment is set up correctly.
Self-hosted agentic pipelines are increasingly viable and give you control over data, cost, and security that cloud-based coding agents don't.

how-we-work agents software-engineering security

116 points on HN · via api-hn

09 · security Intermediate

Inadvertent Context Leakage in Language Models

Jaiden Fairoze

This paper should be required reading for anyone building agents that handle sensitive user data. The researchers demonstrate that even when a model correctly refuses to reveal secrets, those secrets can still leak through subtle statistical patterns in the model's ordinary outputs — and more capable models leak *more*, not less. This isn't a bug you can patch with better system prompts; it appears to be a fundamental byproduct of strong instruction-following, which makes it a systemic architectural concern for any agent handling PII, credentials, or health data.

Takeaways3

Models can leak in-context secrets (SSNs, health data) through benign outputs even when they correctly refuse direct extraction — 2-digit secrets leak with near-perfect accuracy.
More capable, instruction-following models exhibit *more* leakage, meaning upgrading your model doesn't fix this and may make it worse.
Sensitive data should be kept out of the context window by design; architectural controls (not prompt-level guardrails) are the only reliable mitigation.

security llms foundational agents

via api-arxiv · arXiv:2608.19857

10 · security Accessible

Person Hides Prompt Injection in Legal Filing Telling AI to Side With Them

404media.co

A real-world prompt injection attack has now appeared in a US court filing — hidden in white 3-point font, instructing any AI reviewing the document to rule in the filer's favor. This is no longer a theoretical attack vector: adversarial inputs are showing up in documents that AI systems are increasingly being used to summarize and analyze in high-stakes contexts. For engineers building document processing pipelines or legal/compliance tools, this is a concrete example of why untrusted document content must be treated as potentially adversarial input.

Takeaways3

Prompt injection via hidden text in documents is now appearing in real legal proceedings, not just security research labs.
Any pipeline that ingests external documents (PDFs, filings, emails) and feeds them to an LLM is a potential injection target — treat document content as untrusted input.
This highlights the need for input sanitization and sandboxed LLM roles when processing third-party documents in high-stakes workflows.

security prompt-engineering llms

surfaced by 2 sources · 57 points on HN · via api-bluesky

11 · prompt-engineering Accessible

Claude: System Prompts

tosh

Anthropic's official documentation on system prompts for Claude is essential reading if you're integrating Claude into a product. It covers how to structure system prompts to shape Claude's behavior, persona, and constraints — the foundational layer of any serious Claude-based application. Getting this right is the difference between a reliable, production-grade assistant and one that behaves unpredictably at the edges.

Takeaways3

System prompts are your primary lever for controlling Claude's behavior, tone, and boundaries in production deployments.
Understanding Anthropic's own guidance helps you avoid common misconfigurations that lead to inconsistent or unsafe model outputs.
Pairing system prompt design with the right model version (Sonnet vs. Opus vs. Fable) is key to balancing cost and capability.

prompt-engineering llms

733 points on HN · via api-hn

12 · security Intermediate

[AINews] How to steal a Reasoning Trace

This piece covers a timely and alarming finding: reasoning traces from frontier models like OpenAI's o1 can be extracted or inferred even when labs deliberately obscure them — undermining a key security assumption baked into these systems. This sits at the intersection of model distillation, alignment, and chain-of-thought monitoring, and challenges the conventional wisdom that hiding reasoning traces is a reliable defense against model theft or misuse. If you're building on top of reasoning models or thinking about supply chain security for AI, this is required reading.

Takeaways3

Obscuring reasoning traces with cryptographic signatures is not a reliable barrier against distillation or extraction attacks.
Visible chain-of-thought is both a security liability and a potential alignment tool — labs are now forced to navigate that tension explicitly.
This research signals that monitoring and auditing reasoning traces will become a critical part of AI security posture for enterprises.

security reasoning llms

via rss-latentspace