Taxonomy of Agent Failure Modes by Harness Layer

Agent failures aren't one thing, and treating them as one thing is the single most expensive mistake a team can make. A wrong output, a stalled run, and a task that quietly drifts off course can each come from the model, the prompt, a tool, a memory store, or the orchestration logic stitching it all together. Chalking every failure up to "the model got it wrong" is why so much debugging stalls out at symptoms instead of causes. The fix for a prompt problem looks nothing like the fix for a tool problem, and mixing them up burns real engineering time on the wrong target.

Consider a scenario documented in research on agent evaluation (arXiv:2607.28802): an agent working a rental-property task completes phase one correctly, then polls repeatedly for a reply that never comes. An automated judge scoring the trace blames the model, arguing it should have searched harder or tried a different channel. A human annotator looking at the same trace blames the environment, since the scripted reply the test setup was supposed to deliver never arrived. Same trace, same outcome, two different repair paths. One says retrain or re-prompt the model. The other says fix the test harness. Without knowing which layer owns the failure, a team can spend weeks tuning a model that was never the problem.

What the harness is, and why its layers are where most failures originate

The model itself is a stateless policy. Feeding it an observation returns an output, but it keeps no memory of the exchange unless something outside it stores and reinjects that context. The harness is everything wrapped around that: prompt templates, how tools get dispatched, how context gets trimmed, how memory gets written and read back, how the workflow gets sequenced, and what safety checks run before an action fires.

Calling this "prompt engineering" undersells it badly at this point. A production harness now involves workflow design, loop control, evaluation logic, permission gates, and persistent state, closer to systems engineering than to writing clever instructions. Split it into three layers. The first layer covers the model interface: prompt templates, parameter settings, how responses get parsed, how API errors get handled. Layer 2 is the runtime environment: tool definitions, memory stores, input checks, output guardrails, and how the context window gets managed as conversations grow. Layer 3 is orchestration: the agent loop itself, how tasks get broken into steps, conditional branches, approval checkpoints, and anything that runs in parallel.

The evidence for where the real leverage sits is stark, and anyone still spending most of their time polishing prompt wording is optimizing the wrong layer. Refining a prompt past a reasonable baseline moves output quality by less than 3%. Harness-level changes, adding retrieval, giving the agent tool access, adding structured validation on outputs, move quality by 28 to 47%. Prompt tuning is not where the work belongs. The gap is too wide to treat as a matter of taste.

How the research literature organizes agent failures by layer

Not every taxonomy of agent failure is built the same way, and the differences matter because some are far more actionable than others. MAST, built from more than 1,600 annotated execution traces across seven multi-agent frameworks, groups 14 failure modes into three clusters: system design issues, inter-agent misalignment, and task verification. Its conclusion is blunt: base model improvements alone can't cover this taxonomy. A better model doesn't fix a system design problem.

Scale AI's interaction-centric taxonomy takes a different unit of analysis. It maps 41 failure modes onto the edges between components, the connections between model, tool, and environment, and attaches a fault-side label to each one. That label is the real contribution, because it tells an engineer which side of an interaction to fix rather than just naming that something broke. Agreement between the strongest automated judge and human labels came in at Cohen's κ of 0.76, a solid signal that these categories track something real rather than noise.

AgentErrorTaxonomy works at the module level, splitting a rollout into memory, reflection, planning, action, and system-level operations, then pinning each failure on whichever module actually owns it across a large set of trajectories. The Vadlamudi four-dimensional taxonomy, drawing on more than 40 published works from 2020 through early 2025, organizes around reasoning and planning, tool use and action execution, memory and context management, and multi-agent orchestration, with formal definitions and the conditions that trigger each failure class.

A more practitioner-facing breakdown splits failures into architectural (roughly 20%), execution (roughly 25%), and data-layer (roughly 55%) categories. Over half of production failures trace back to data, not architecture or execution logic, and that number alone should reorder where most teams look first when something breaks. A silent-failure taxonomy, built from more than 40,000 controlled trials and over 100,000 production agent interactions, organizes failures by lifecycle layer and argues these aren't bugs to be patched out. They're structural consequences of running autonomous, language-based systems without deterministic guardrails.

Every one of these disagrees on unit of analysis and scope. But they converge on the same conclusion: an outcome label alone, "task failed," collapses distinct causes into one bucket and points repair in the wrong direction. What follows works through each layer in turn.

Prompt layer failures: ambiguity, cognitive overload, and the Mega-Prompt anti-pattern

Prompt failures aren't a vague "quality" issue. They have specific, recognizable shapes.

Ambiguity failures happen when a prompt doesn't give an agent enough structure to plan, break down a task, or reason step by step. The agent produces an illogical plan, skips decomposition entirely, or stalls at a decision point because nothing in the prompt tells it what to do next.

Then there's the Mega-Prompt anti-pattern: one giant system prompt trying to encode every rule, every edge case, every behavior the agent might need, all in a single block. Conflicting instructions buried in a huge block produce behavior nobody can predict, and adding one more requirement often causes earlier behavior to regress. Add one more requirement to that prompt and existing behavior often regresses, because nothing in a flat block of text protects earlier instructions from a later, contradictory one. Teams that keep reaching for "just add another paragraph to the system prompt" are the ones most likely to hit this wall, and they rarely see it coming until the agent starts contradicting itself mid-run.

A quieter but just as damaging failure: prompts that never specify what "done" looks like. No terminal state, no stopping condition. That absence is a root cause of reasoning loops further downstream.

Prompt-layer failures tend to repeat. Same input, same wrong decision, run after run. Tool or environment failures behave differently: they're intermittent, dependent on outside conditions. That consistency across runs is itself the diagnostic signal for a prompt-layer failure, and it's the fastest way to rule out the model as the cause.

Fixes at this layer are structural: modular guide documents loaded only at the workflow step that needs them, explicit terminal conditions written into the prompt, and decomposed instruction sets that swap in context-appropriate sub-prompts instead of one document trying to cover everything. The interaction-centric taxonomy draws a clean line here, attaching a fault-side label to each failure so an engineer can tell which side of an interaction owns the problem and where the fix belongs.

Tool layer failures: schema drift, silent malformation, and compounding error rates

Tool failures split into two very different categories, and only one of them is easy. The visible kind: network timeouts, rate limits, an upstream service going down. These throw an error, so someone notices. In production, individual tool calls fail somewhere between 3% and 15% of the time from causes like this alone.

The dangerous category is silent, and it deserves far more attention than it gets. A tool's schema changes, and instead of an error, the tool returns something malformed but plausible-looking, and it passes right through the pipeline until a human eventually notices the output is wrong.

The clearest real-world example comes from an n8n postmortem. Upgrading from n8n v2.4.7 to v2.6.3 introduced invalid tool schemas in tool calls, breaking both OpenAI and Anthropic integrations at the same time. The schema for tool arguments had changed between versions, and nothing surfaced that change to the harnesses relying on it. The failure sat right at the tool boundary, invisible until it wasn't.

The math on why this matters compounds fast. A 10-step workflow where each step has a 5% failure rate, which sounds tolerable in isolation, succeeds only about 60% of the time end to end. A per-step number that looks fine on its own turns into a serious problem once it's chained across a workflow, and most engineers underestimate this until they run the multiplication themselves.

The interaction-centric taxonomy's notation makes the distinction precise. TOOL-MODEL fault:MODEL means the model got an error signal and had the information to retry or reroute, but didn't. TOOL-MODEL fault:TOOL means the wrapper swallowed the error before the model ever saw it. Same surface failure. Completely different fix, and treating the two as one problem is how teams end up patching the wrong layer twice while the actual bug sits untouched.

Memory layer failures: staleness, context rot, and missed checkpoints after side effects

Production agents typically run on four memory tiers, namely in-context working memory, an external key-value store, episodic logs, and semantic retrieval over a vector store. Each fails in its own way, and none of the fixes are exotic.

Stale memory poisoning is one of the more insidious failures, and it's also one of the most preventable. The agent confidently recalls something from months ago that's no longer true, recommending a product that's since been discontinued, for instance. The fix is mechanical rather than clever: timestamp every memory entry, and apply staleness thresholds that penalize confidence in retrieval scoring as entries age.

Missing checkpoints after side effects cause a different kind of damage. An agent sends an email at step 8, crashes at step 9, and on resume rolls back to step 7, then sends that same email again. The fix here is an idempotency key attached to every step that produces a real-world side effect, so a resumed run can tell whether that action already happened.

Context rot is a separate phenomenon, and it undercuts the assumption that bigger context windows are always an improvement, an assumption plenty of teams still build around. A study from Chroma found performance degradation across 18 frontier models on simple tasks as prompt length grew, even when the task itself didn't get harder. Longer context isn't free. It degrades reliability, and the degradation is easy to miss because nothing throws an error, the answers just get quietly worse.

Cross-agent handoffs make this worse. A silent-failure taxonomy documented information preservation across relay experiments at a mean of 87.4%, with a range running from 70% to 100%. The same architecture can preserve everything in one run and lose nearly a third of the information in another. Whether agents can even detect their own stale memory is still an open research question: a 2026 paper, "STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?" (arXiv:2605.06527), takes this on directly, and the fact that it's still being asked as a research question means the failure class isn't solved by default in any current system.

Workflow layer failures: loops, compounding error, and the trajectory degeneration trap

Workflow failures are emergent. Every component involved can work exactly as designed, and the workflow still fails. These are harder to trace than a bad prompt or a broken tool call, because there's no single component to point at.

Reasoning loops are the clearest symptom, and they tend to trace back to three root causes: unclear goals, where the agent has no way to know the task is finished; ambiguous tool feedback, where a tool doesn't clearly signal success or failure; and the absence of any hard stopping criteria, no iteration cap, no time limit.

Trajectory degeneration, where an agent's behavior slowly drifts and decays over a long run, looks like a root cause but is usually a downstream symptom of something else, and mislabeling it is the single most common mistake in this layer. A priority-based annotation protocol described in arXiv:2605.22166 checks for action realization failures before environment contract mismatches, and only considers trajectory degeneration after ruling out earlier categories in the ordering. That ordering matters: if an agent writes a tool call as plain text that the environment never actually executes, the episode will eventually burn through its step budget and look, on the surface, like degeneration. The correct label is an action realization failure. Calling it degeneration instead means the real interface bug never gets fixed, and the team keeps chasing a symptom that was never the disease.

Compounding error is the workflow layer's core quantitative problem, and the numbers are unforgiving. At 85% accuracy per step, which counts as strong performance by most standards, a 10-step workflow only succeeds about 20% of the time (0.85 raised to the 10th power). That's not hypothetical: real-world data backs it up, with only about 24% of agent tasks completing successfully on the first attempt.

Multi-agent and orchestration layer failures: message loss, role conflict, and logical deadlocks

Once multiple agents start coordinating, an entirely new set of failure mechanisms appears, ones that don't exist in a single-agent system. Research cataloged in arXiv:2503.13657 identifies several ways inter-agent communication breaks down. One agent can withhold a message it should have passed along. Another can receive a message and simply ignore it, proceeding as though it never arrived.

These aren't edge cases confined to exotic architectures. Any system where multiple agents depend on each other's outputs to make a decision has to contend with the chance that a message gets dropped, misread, or acted on by the wrong agent at the wrong time. A bad tool call usually throws some kind of error, and someone eventually sees it. A withheld or ignored message often leaves no trace at all, which is what makes orchestration-layer failures some of the hardest in the entire taxonomy to catch before they reach production.