Tag: prompt-engineering

Intermediate

LLM-as-a-Verifier: A General-Purpose Verification Framework

Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang, Chelsea Finn, Marco Pavone, Ion Stoica, Azalia Mirhoseini

Using an LLM to verify another LLM's outputs is already common practice, but most approaches produce coarse binary scores that aren't very reliable. This paper reframes verification as a scaling axis — like pre-training compute — and shows that computing continuous scores from logit distributions, then scaling granularity, repetition, and criteria decomposition, yields substantially better signal without any additional training. Directly applicable if you're building evaluation pipelines or using LLM judges to filter agent outputs.

Takeaways

Treating verification scores as continuous distributions over logits outperforms discrete LLM-judge scoring for separating correct from incorrect solutions.
Decomposing evaluation criteria and aggregating sub-scores improves calibration beyond what single-prompt judges achieve.
Verification quality scales predictably with compute investment, making it a tunable parameter in your evaluation pipeline.

from Jul 13, 2026 · via api-hf · arXiv:2607.05391

Intermediate

Program-as-Weights: A Programming Paradigm for Fuzzy Functions

Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng

llms software-engineering prompt-engineering foundational

Instead of calling a large model API every time you need fuzzy logic like 'is this log line important?' or 'fix this malformed JSON,' this approach compiles your natural-language spec into a tiny, locally-runnable adapter that matches the quality of a 32B model using a 0.6B one. The practical payoff is dramatic: 50x less memory, runs on a MacBook, and no network dependency — reframing LLMs as compilers for specialized micro-models rather than runtime oracles.

Takeaways

Many 'AI' tasks that currently require large API calls can be compiled into compact, local adapters with equivalent quality.
The 'compile once, run many times' model dramatically reduces inference cost and latency for stable fuzzy functions.
This approach restores locality and reproducibility to AI-powered functionality that currently depends on external APIs.

from Jul 6, 2026 · via api-hf · arXiv:2607.02512

Accessible

CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

Morayo Danielle Adeyemi, Ryan A. Rossi, Franck Dernoncourt

prompt-engineering evaluations llms

Compressing your prompts in 'caveman style' to save tokens actually backfires: models compensate by generating longer outputs, raising net cost by up to 1.8x on average. Conversely, constraining the model's output length does save money—1.4-2.4x cost reduction across API models—without proportionate accuracy loss. The practical upshot is clear: if you want to reduce API spend, constrain outputs, don't mangle inputs.

Takeaways

Input compression (shortened, grammar-dropping prompts) raises net API costs because models respond with longer outputs.
Output compression reliably reduces cost 1.4-2.4x across tested models without proportionate accuracy degradation.
Cost optimization efforts should target the output channel, not the input channel.

from Jun 29, 2026 · via api-hf · arXiv:2606.24083

Intermediate

Prompt Injection as Role Confusion

security prompt-engineering agents foundational

This blog post frames prompt injection attacks as a role-confusion problem—the model can't reliably distinguish between its own privileged instructions and injected attacker text—which is a more tractable and precise framing than 'the model got tricked.' For agent builders, understanding injection as a structural role-boundary failure (rather than a content-filtering failure) points toward architectural mitigations rather than just prompt hardening.

Takeaways

Prompt injection is fundamentally a role-boundary enforcement failure, not merely a content-filtering challenge.
Framing injection as role confusion suggests architectural mitigations (clearer privilege separation) over prompt-level patches.
Models trained on role tags don't reliably treat those boundaries as security boundaries under adversarial conditions.

from Jun 29, 2026 · via rss-willison

Accessible

Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning

Sanket Badhe, Deep Shah

prompt-engineering reasoning llms

Fine-tuning small models to reason well is expensive and opaque; Prompt-Level Distillation offers a third path — extract reasoning patterns from a large teacher model and encode them as structured system prompt instructions for a smaller student model. The benchmark results are striking (57% to 90% F1 on StereoSet, 67% to 83% on Contract-NLI) and the approach is immediately actionable without any training infrastructure.

Takeaways

Structured system prompt instructions distilled from a teacher model can close much of the performance gap between small and large models without fine-tuning.
PLD preserves interpretability because the decision logic is explicit in the prompt, unlike weights modified by fine-tuning.
Cross-architecture generalization (Gemma-3 and Mistral) suggests this is a robust technique, not just an artifact of a specific model family.

from Jun 22, 2026 · via api-hf · arXiv:2602.21103

Intermediate

Probe-and-Refine Tuning of Repository Guidance for Coding Agents

Asa Shepard

agents software-engineering prompt-engineering evaluations how-we-work

If your team is using AGENTS.md files (or similar repo guidance docs) to orient coding agents, this paper explains why some of them help and others actively hurt performance — and it's all about how the guidance is generated. The probe-and-refine method uses synthetic bug probes to iteratively diagnose and patch guidance files without an agent loop, achieving a 33% vs 28.3% resolve rate improvement on SWE-bench, a meaningful lift from a purely prompt-side intervention.

Takeaways

Hand-written or naively LLM-generated AGENTS.md files can harm agent performance; iterative refinement driven by synthetic probes is key.
Probe-and-refine requires no agent loop or tool use during tuning, making it lightweight to adopt.
How repository guidance is produced matters more than whether it exists at all.

from Jun 22, 2026 · via api-arxiv · arXiv:2606.20512

Intermediate

No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions

Xu Yang, Zhizhou Sha, Junbo Li, Jian Yu, Yifan Sun, Matthew Zhao, Jinrui Fang, Xinyue Guo, Yining Wu, Xu Hu, Yifu Luo, Qiang Liu, Zhangyang Wang

security evaluations llms prompt-engineering

Everyone worries about prompt injection in AI reviewers, but this paper shows you don't need hidden instructions at all — rewriting just the abstract, framing, and related work sections (no methods, no results changes) achieves a 75% attack success rate against mainstream AI peer review systems. This is a direct warning for anyone using LLMs as judges or evaluators in pipelines: they're systematically sensitive to how arguments are packaged, not just what evidence is presented.

Takeaways

LLM evaluators can be gamed purely through presentation framing with no changes to underlying evidence or results.
Adversarial repackaging via closed-loop AI feedback is a practical, scalable attack requiring no special exploits.
Any system using LLMs as judges needs defenses against narrative manipulation, not just prompt injection.

from Jun 22, 2026 · via api-hf · arXiv:2606.13044

Intermediate

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

Linas Nasvytis, Simon Jerome Han, Ben Prystawski, Satchel Grant, Noah D. Goodman, Judith E. Fan

reasoning agents prompt-engineering

CORE enables rapid reasoning improvements using minimal training samples by generating natural language insights that contrast successful and failed attempts, outperforming both parametric and non-parametric methods. While other approaches need hundreds of samples and thousands of rollouts, CORE achieves comparable results with as few as five training samples. This is immediately practical for production systems where you have limited training data but need reliable reasoning improvements.

Takeaways

CORE achieves faster reasoning improvements than traditional methods while using significantly fewer model rollouts.
Natural language insights from contrasting successful and failed attempts enable effective learning from minimal data.
Non-parametric reflection approaches can outperform parametric methods when training samples are severely limited.

from Jun 15, 2026 · via api-hf · arXiv:2605.28742

Intermediate

Reflective Prompt Tuning through Language Model Function-Calling

Farima Fatahi Bayat, Moin Aminnaseri, Pouya Pezeshkpour, Estevam Hruschka

prompt-engineering llms

Prompt engineering remains frustratingly manual and brittle, but this paper introduces a systematic solution that mimics how human prompt engineers actually work. Reflective Prompt Tuning uses LLM function calling to diagnose failures across entire datasets, identify systematic error patterns, and make targeted prompt edits based on failure history. Instead of random search or single-example fixes, it provides a structured framework for iterative prompt improvement that captures recurring problems.

Takeaways

LLM function calling can automate the diagnostic workflow of human prompt engineers for systematic prompt optimization.
Batch-level failure analysis outperforms single-example critique for identifying and fixing systematic prompt issues.
Structured diagnostic functions enable targeted prompt edits based on error patterns rather than random search.

from Jun 1, 2026 · via api-hf · arXiv:2605.21781

Accessible

Prompts are technical debt too

prompt-engineering software-engineering

Argues that prompts should be treated with the same engineering discipline as code since they accumulate complexity, dependencies, and maintenance burden over time. This perspective is crucial for teams building production LLM systems where ad-hoc prompt management leads to the same problems as unmanaged code: brittleness, difficult debugging, and hidden interdependencies that slow development.

Takeaways

Prompts accumulate technical debt through complexity, dependencies, and maintenance overhead just like traditional code.
Production LLM systems require disciplined prompt engineering practices to avoid brittleness and debugging difficulties.
Teams should apply software engineering best practices like versioning, testing, and refactoring to prompt management.

from May 25, 2026 · 5 points on Lobsters · via api-lobsters

Accessible

Many-Shot CoT-ICL: Making In-Context Learning Truly Learn

Tsz Ting Chung, Lemao Liu, Mo Yu, Dit-Yan Yeung

llms prompt-engineering reasoning foundational

This overturns conventional wisdom about many-shot in-context learning for reasoning tasks. While more examples help with simple tasks, reasoning tasks show unstable scaling behavior, and semantic similarity-based retrieval actually hurts performance. The order of examples matters more than previously thought. This has immediate implications for how you structure prompts and manage context in reasoning-heavy production systems.

Takeaways

Many-shot scaling rules for non-reasoning tasks don't apply to reasoning tasks and can degrade performance.
Semantic similarity poorly predicts procedural compatibility in chain-of-thought reasoning.
Example ordering significantly impacts performance and requires careful consideration in production prompt design.

from May 18, 2026 · via api-hf · arXiv:2605.13511

Intermediate

Introducing the OpenAI Safety Bug Bounty program

security agents prompt-engineering

OpenAI's new bug bounty program specifically targets AI safety issues including prompt injection, agentic vulnerabilities, and data exfiltration — signaling that these attack vectors are now mainstream security concerns. For production teams, this validates that AI-specific security testing should be part of standard security practices, not an afterthought.

Takeaways

AI-specific vulnerabilities like prompt injection and agentic exploits are now recognized as legitimate security concerns requiring dedicated testing.
Production AI systems need security models that account for both traditional software vulnerabilities and novel AI attack vectors.

from Mar 29, 2026 · via rss-openai