LLM Evaluation Metrics That Signal Real Agent Quality
Measure agent quality at each architectural layer, not just final answers.
Most agent evaluations still ask one question: did the final answer come out right? That's the wrong question, or at least a dangerously incomplete one. A passing score routinely hides a broken tool call, a wasted loop, or a memory lookup that quietly returned the wrong thing, and none of that appears in the one number most teams actually track. The metrics that signal agent quality live at specific architectural layers: planning, tool calling, reasoning, and memory. Each layer fails on its own, and usually without leaving a mark on the final output.
The standard workflow runs the agent against held-out tasks, checks the answer against expected output, and ships if the pass rate looks good. That number won't tell you the agent looped twice before landing on the right answer, called the wrong tool and got bailed out by luck, or burned three times the tokens a cleaner path would have needed. It won't tell you a user four turns into a conversation hit a dead end the benchmark never tested for, either. Worse, the mistakes compound: a wrong assumption formed early in a run rides along through every later step, so by the time something visibly breaks, the actual cause sits buried several turns upstream. A recurring gap in agent evaluation is that most frameworks hand you metrics, but no map connecting those metrics to the part of the system that produced the failure.
The category error: treating an agent as a black box with one output
Non-agentic LLM evaluation has a clean shape. One input goes in, one output comes out, a judge scores the pair, done. Most of the tooling built for LLMs assumes exactly that shape, because for years that shape was accurate.
Agents break the assumption. Before an agent produces an answer, it may draft a plan, call several tools, read the results back, revise its approach, retry a failed step, hand part of the job to a sub-agent, and carry state across turns that happened minutes or days earlier. Each of those is its own place where things go wrong, independent of the others.
An end-to-end failing score tells you only that something broke somewhere. It can't say whether the planner picked a bad route, the tool router grabbed the wrong function, the memory layer surfaced context that had gone stale, or a handoff between sub-agents dropped a piece of state on the floor. A single verdict can't carry that much information, no matter how carefully it's engineered. Serious evaluation splits into three layers instead of one. Final-answer evaluation still matters, but on its own it leaves too much room for undetected errors to slip through. Trajectory evaluation asks whether the path the agent took was efficient, logically sound, and within policy. Per-turn evaluation goes further, asking which exact step, tool call, or reasoning link is the one that actually broke.
Planning metrics: measuring the route before the agent takes it
The planning layer is where an agent decides, before it acts, what sequence of steps it intends to take, then revises that sequence as tool results come back. It sounds simple, and that simplicity is the trap. Modern agents narrate their own thinking in chain-of-thought traces that read as careful and deliberate even when the underlying route is circular, redundant, or flat wrong. The planning layer is the one most likely to get waved through during review because it sounds right.
Two separate metrics matter here, and treating them as one is a mistake teams make constantly. Plan quality asks whether the initial route made sense given the task and the tools on hand. Plan adherence asks something different: once tool outputs started arriving, did the agent stick to that plan, or did it drift off course for no defensible reason? An agent can score well on one and badly on the other. Teams that track only one of the two are flying half-blind, and they usually don't find out until the blind side is the one that mattered.
Long-horizon agents raise the stakes further. Some plan across dozens of steps, spin up sub-agents mid-run, and replan on the fly as conditions change. In that setting, the planning metric has to separate replanning that responds to genuinely new information from replanning that's just noise: an agent talking itself in circles and mistaking motion for progress.
Tool-calling metrics: correctness, argument validity, and selection accuracy
Tool calling splits into two questions that look similar but need entirely different scoring methods. Tool correctness asks whether the agent picked the right tool for the step at hand. Argument correctness asks whether it passed that tool the right inputs.
A machine can check the first question cleanly: did the agent call the function it should have called, at the point it should have called it. The second is murkier. Was this date formatted the way the downstream system expects? Is this the correct entity name, given everything the agent has seen so far? That's a judgment call, not a lookup, and it usually needs a semantic scorer rather than a deterministic one.
Schema drift in a tool call can produce a response that looks plausible enough to pass downstream checks, with the problem only surfacing when a person finally notices the output is wrong, sometimes days later. Add the ordinary mechanics of production systems on top: network timeouts, rate limits, upstream services blinking out for a second. Individual tool calls fail at a real, non-trivial rate even in carefully built systems. What separates a resilient agent from a fragile one isn't whether calls fail; it's whether the agent recovers cleanly or turns one failed call into three bad ones.
Reasoning metrics: evaluating the thinking between steps
Reasoning evaluation asks something more slippery than tool or plan checks. Is each intermediate decision actually relevant to the goal, and does the logic hold together from one step to the next, or does the agent quietly introduce an assumption that nothing in the prior steps or tool outputs actually supports?
Current-generation agents create a genuine blind spot here, and it's a bad one. Agents produce chain-of-thought traces that can appear coherent and well-structured, making it easy for a human skimming the trace to miss the point where the logic goes sideways. Confidence in the prose is not evidence of correctness in the reasoning, and that gap repeatedly slips past manual review, because the trace reads like the work of something that knows what it's doing.
Flawed reasoning tends to produce one of a few recognizable shapes downstream. A confident wrong answer is one: the agent reasons its way to a conclusion that sounds settled but rests on a bad assumption several steps back. Redundant tool calls are another, born from an agent that never actually folded the previous tool's output into its working state and goes back to ask again. A runaway loop is the third, where each step generates a new sub-question that the agent's own flawed reasoning can't resolve, so it keeps generating questions instead of an answer.
Scoring this well takes judgment. Reasoning quality depends on the specific chain of thought the agent produced, the task it was given, and the tool outputs in front of it at the time, none of which reduces cleanly to a pass or fail. That makes LLM-as-judge the right instrument for this layer, with the caveats that come with it, and those caveats are substantial (more on those shortly).
Memory metrics: detecting when state management breaks across turns
Memory covers durable state that persists across sessions, retrieval from a vector store, the running conversation history, and the policies that decide what actually gets included in each call to the model.
Memory failures rarely announce themselves as one broken turn. Instead they degrade relevance a little at a time: feeding the agent context that's gone stale, or dropping a fact the user established several turns earlier and never mentions again. The failure surface, in effect, is the gap between what the agent should know at a given point and what it actually manages to retrieve.
Anthropic's work on context engineering reframes the problem well. Instead of asking what belongs in the prompt, the sharper question is what configuration of context, across system prompts, tool definitions, MCP connections, and message history, actually produces the behavior a task needs. That's a bigger question than "what's in the prompt," and it treats context as something to actively manage rather than something that just piles up turn after turn.
The payoff appears in the numbers. Anthropic's context compaction approach cut token consumption by 84% in a 100-turn web search evaluation, while letting agents finish workflows that would otherwise have run into context limits and stalled out mid-task. Memory management is a variable practitioners can measure and improve directly, not a fixed cost of running an agent. It's a variable practitioners can measure and improve directly.
The four documented failure modes of LLM-as-judge scoring
LLM-as-judge has become the practical default for scoring plan quality, reasoning quality, faithfulness, and answer relevancy, and it shows up as the standard method across tools like LangSmith, Braintrust, Phoenix, and DeepEval. It earns that position honestly: most of these criteria genuinely require judgment, and no deterministic check substitutes for it.
It is not reliable in the way a unit test is reliable, and four failure modes occur consistently enough now to count as known hazards rather than edge cases. Length and verbosity bias means judges tend to rate longer, more detailed answers higher, whether or not the extra length adds anything correct. Position bias occurs in pairwise comparisons, where the answer placed first or last in the prompt gets an edge from position alone, to the point that swapping the order can flip which answer wins. Self-preference means a judge model tends to rate answers that sound like its own writing style, or come from its own model family, more favorably, regardless of whether they're actually better. Cost and non-determinism round it out: each judgment is a full model call, too expensive to run on every turn of every production trace, and the same exact trajectory can get scored differently across separate runs of the same judge.
The scale of the problem deserves to be named directly, not softened. On failure attribution datasets, GPT-5.1 reaches only 18.15% accuracy on failure attribution datasets using AGENTSCOPE. That is nowhere near what a reliable verdict would require, and it isn't a rounding error. It changes what LLM-as-judge actually is: not a scorer handing you a trustworthy verdict, but one signal among several that needs triangulation before anyone leans on it. Use deterministic checks for anything with an exact right answer, like tool correctness, and save LLM-as-judge for the criteria that genuinely can't be reduced to a lookup.
Root cause attribution beyond per-metric scores
A dropped planning score tells you the plan was bad. It does not tell you why. The planner model itself may have degraded, the system prompt's constraints may have been ambiguous enough to mislead it, or the tool definitions it was reasoning over may have quietly drifted out of sync with what the tools actually do. That's the attribution gap, and it's the reason a dashboard full of per-metric scores can still leave a team guessing at three in the morning with a production incident open.
Research on multi-agent failures has started building a taxonomy for exactly this problem. Specification flaws, misalignment between agents, and weak verification keep recurring as categories, and all three sit above any single metric layer. No planning score or tool-correctness score catches these alone, because they describe how the pieces interact rather than how any one piece performs in isolation.
Some newer tooling targets that gap directly, rather than adding another dashboard on top of the old one. AgentRx works by localizing the critical step in a trajectory and assigning it a root-cause category, using synthesized constraints and a validation log. Who&When Pro takes a different approach to the same problem: it expands the labeled data available for this kind of analysis with more than 12,000 labeled trajectories spanning different agent frameworks, domains, and modalities, the volume of ground truth needed to actually train and check attribution classifiers instead of guessing at them.
Production traces as the attribution signal offline evals cannot provide
Standard application monitoring was built for deterministic HTTP paths, and it shows its age fast against agents. A single agent request can trigger several LLM calls, multiple tool invocations, a vector store lookup, and a handoff to another agent, and every one of those is a place where something can go wrong that a normal APM tool never sees.
Agent observability has to capture a different set of things: which tool got selected and what arguments it was called with, what the model actually said at each step, what got read from and written to memory, which branch of a decision tree the agent took. Every one of those needs a timestamp and a parent-child link back to the request that started it, so a full trace can reconstruct the entire execution graph after the fact.
That's also why the signal types agents need don't exist anywhere in the final output. Intermediate reasoning steps, the outcome of each tool selection, the handoff points between agents, per-trace scoring for hallucination and faithfulness: none of it can be reconstructed after the run ends, because the final answer never contained it to begin with. Capture it while the run happens, or lose it for good.
That gap is a real barrier to adoption, not a hypothetical one. McKinsey's State of AI Trust report names security and risk concerns, governance gaps, and shortfalls in team knowledge and training as the top obstacles to scaling agentic AI in production. Observability has to get built into a working agent from the outset, not bolted on after something breaks. The infrastructure is what decides whether anyone trusts the agent enough to actually scale it.
