Extracting Implicit Feedback Signals From Live Agent Sessions

Most agent teams are flying blind because they're measuring the wrong thing, not because they lack data. It's that they're measuring the wrong thing. Users provide explicit feedback, thumbs-up, thumbs-down, a star rating, on somewhere between 1% and 3% of interactions; Liu, Zhang, and Choi's 2025 EMNLP study on the WildChat and LMSYS datasets found this. Soliciting more of it doesn't close the gap, because the structural rate at which users provide it remains stubbornly low regardless. So a production agent handling thousands of sessions a day gets a usable rating on maybe one in fifty of them, which isn't enough to tell anyone whether last week's deployment made things better or worse.

That gap isn't a collection problem. Asking more often makes it worse, not better. The mismatch is structural: agentic sessions generate a constant stream of behavioral evidence, actions, repetitions, escalations, abandonments, that never gets logged as "feedback" because nobody built the instrumentation to read it that way. The signal already exists. What's missing is the discipline to extract it from the trace and trust it as evidence.

What behavioral signals in session traces look like

Implicit feedback is inferred from what a user does after receiving a response, not what they report about it, and that distinction matters because behavior is harder to fake and far more frequent than a rating ever will be. It's what a user does after receiving a response, not what they report about it, and that distinction matters because behavior is harder to fake and far more frequent than a rating ever will be.

Failure signals tend to cluster around a handful of recognizable patterns. Request repetition, where a user rephrases or re-asks something they already asked, gets classified in the taxonomy Don-Yehiya and colleagues built in 2024 (and later adopted by Liu et al.) as "Rephrasing." It's a direct tell that the prior answer didn't land. A step up from that is "Make Aware without Correction," where the user flags the response as wrong but doesn't offer a fix, and "Make Aware with Correction," where they also supply the right answer themselves, a stronger and more specific failure signal because it tells you not just that something broke but what it should have said. "Ask for Clarification" Turns appear in the trace when the agent's output left out something the user needed. Escalation to a human operator is another prominent signal, and it, and escalation in particular is about as close to a confirmed failure as implicit signal gets. Conversation length itself is a rough proxy too: longer threads often mean the agent isn't closing the task, not that the user is enjoying the conversation.

Success leaves its own trace. When a user takes the agent's response and acts on it directly, that says the output was trusted enough to use as-is. Positive affirmation turns ("great job," "that's exactly right") register as positive feedback in the Don-Yehiya ontology, though Liu et al. flag a real caveat here: prompts that elicit this kind of praise skew toward lower-quality or more toxic inputs than a random sample, so a spike in "great job" turns isn't automatically a quality signal worth chasing. Sessions that close without follow-up correction turns round out the success side.

Agentic sessions also carry a texture of engagement signals that a simple chat log doesn't. Scroll depth and time-on-item proxy for whether an output actually got consumed rather than skimmed and discarded. Sentiment shift across turns, the delta in tone before and after a given response, is measurable straight from the log. Patterns in how the session unfolds after a tool-related exchange are another.

None of this requires the user to do anything extra. It's already sitting in the trace, whether or not anyone is reading it. Liu et al. densely annotated 443 user turns across 109 sessions and found that feedback turns make up more than half of user utterances in the later stretches of longer conversations. The longer a session runs, the richer the signal gets, which is exactly the opposite of what a 1-3% explicit response rate would suggest.

Why the same behavioral signal can mean different things depending on which harness layer produced it

A repeated request doesn't tell you why the agent failed. It could mean the prompt was ambiguous and the model genuinely misread the task. It could mean a tool schema drifted and the agent got back stale or malformed data it had no way to interpret correctly. It could mean the memory layer dropped context between turns, so the agent is re-asking something it was already told. Or it could mean a workflow loop routed the user back to the wrong step entirely.

Escalation to a human carries the same ambiguity. Perhaps the agent's recovery logic failed to catch an error it should have caught. Maybe a required tool was simply unavailable. Maybe an output quality gate let through something that looked fine on the surface but was actually wrong. The behavioral signal is real evidence, no argument there, but it sits upstream of the actual cause. Treat a repetition as proof of "model failure" when the real culprit was a tool-schema mismatch, and the fix that follows will target the wrong layer entirely.

This is a structural problem, not a labeling inconvenience. A survey out of TUM on LLM agent trajectory analysis makes the point directly: agent execution trajectories are long, dense with natural language, and non-deterministic, and the root cause is often buried in a step the behavioral signal never points to directly. The signal tells you something went wrong. It doesn't tell you where.

How production traces turn behavioral signals into attributable failure evidence

Standard application performance monitoring wasn't built for this. It can't see a tool call that returned malformed data, a context window that silently truncated, or a loop that kept re-running the same three steps; none of that becomes visible without instrumentation built specifically for agents. And that's exactly where most agent incidents actually originate, not in the model's reasoning but in the scaffolding around it.

A typed trace schema fixes the blind spot by capturing distinct spans: tool-call spans, reasoning spans, state spans, memory spans. Each one maps to a different layer of the harness, and once they're captured, a failure signal, say a repetition, can be cross-referenced against whatever span ran right before the user gave up and re-asked. That cross-reference is the whole trick. Without it, a team has a symptom with no case history.

The gap between wanting this and having it is wide. A research brief on enterprise AI adoption found that 73% of enterprises now require agent monitoring in production, yet 63.4% cite a lack of adequate observability tooling as a major obstacle to actually doing it. Most teams know they need the visibility. Most don't have it yet.

Teams that build this out typically mature through progressive stages, moving from basic trace collection and correlation to online scoring and regression alerting, and eventually to continuous integration of eval pipelines where every regression becomes a test case instead of a fire drill.

The payoff is that connecting a behavioral signal, a repetition, an abandonment, an escalation, to the exact span where the trajectory went sideways turns debugging from a one-off investigation into a repeatable workflow. OpenTelemetry's semantic conventions for generative AI give this a portable, vendor-neutral backbone that teams can carry across platforms. The tooling landscape around this has gotten crowded, and different platforms occupy genuinely different niches: some lean toward workflow-specific integrations, others toward open-source flexibility, others toward rigorous eval science as the core product. None of them solve how to attribute a failure to its source by itself. They give you the trace. Reading it is still the work.

The attribution problem: tracing a behavioral signal to the responsible harness layer

Get attribution wrong and the cost compounds fast. A two-year postmortem at a major retailer found a persistent misattribution rate hovering around 10%, where the diagnosis kept blaming a technology simply because it happened to be mentioned in the incident thread, not because it was actually responsible. That's what happens when attribution is done by pattern-matching co-occurrence instead of tracing an actual causal chain.

The difficulty is structural. Traditional software failures usually appear as a code defect you can point to. Agent failures are buried inside long natural-language trajectories, and the step actually responsible is frequently several turns removed from the point where the behavioral signal finally becomes visible.

Recent work on multi-turn diagnosis makes the contrast with single-pass scoring concrete. AgentDebugX's DeepDebug component does root-cause diagnosis across the full trajectory rather than scoring a single turn in isolation, combining global trajectory understanding with structure-guided investigation and cross-examination of competing hypotheses. On the Who&When benchmark, that approach hit 28.8% exact agent-and-step attribution accuracy on a 9B backbone, against 21.7% for the strongest single-pass baseline. That gap, roughly seven points, is the structural advantage of reasoning across an entire trajectory instead of scoring the moment where the symptom appeared.

The distinction between closed-loop and decoupled correction follows the same logic. Decoupled self-correction patches the failure at the point where it was observed, without ever grounding the fix in a diagnosed cause, which works fine when the cause and the symptom are the same step and fails badly when they aren't. On the GAIA benchmark, closed-loop recovery repaired more than twice as many failed tasks in a single rerun compared to decoupled baselines, because it fixes the actual upstream cause rather than patching the downstream symptom.

Other frameworks reinforce the same pattern from different angles. AgentRx, which pairs constraint synthesis with LLM-based adjudication for cross-domain failure diagnosis, delivered a 23.6% absolute improvement in failure localization and 22.9% in root-causing the failure once located. Separately, separate research found that incorporating code-level knowledge into root-cause analysis improved localization accuracy by 28.3% over the prior leading method.

A signal like "the user repeated this request three times" is a starting point, not a conclusion, regardless of which framework a team adopts. The actual work is reading backward through the trace to find which harness layer, prompt, tool, memory, or workflow, generated the condition that produced the signal in the first place.

Reading signals at the harness layer level: prompts, tools, memory, and workflows as distinct failure sources

The harness is everything wrapped around a fixed model at runtime: prompts, tool interfaces, context construction, state management, middleware, recovery logic, evaluation hooks. Two teams running the identical underlying model will get meaningfully different outcomes purely from how well each of these layers is built, and it means most quality problems aren't model problems at all.

At the prompt layer, a run of "Ask for Clarification" turns on the same task type usually means the prompt is underspecified for that class of request. A recurring "Make Aware without Correction" pattern tied to one task type suggests the prompt doesn't constrain the model's reasoning tightly enough there. It's tempting to treat this as purely a prompt-engineering fix, but the ceiling on prompt-only improvement is modest. Harness engineering research cited in industry work shows that evolving the full harness, not just the prompt, has been shown to produce an average relative improvement of 120% on Pass@1. Fixing the wording and stopping there leaves most of the available gain on the table.

The tool layer is where a large share of real incidents actually live. Tool-call failures, stale returns, malformed responses: these are a major source of agent incidents in production, often more so than errors in the model's own reasoning. Schema drift is a particularly stubborn version of this, where the tool's interface contract changes but the harness calling it never gets updated, producing a consistent failure pattern on that specific invocation that appears in traces as a cluster of repetition events. Worse, if the recovery logic doesn't catch the tool failure, it propagates forward and becomes visible several turns later as an unexplained bad response, with the trace showing an apparently healthy exchange sitting right before it.

Memory failures have their own signature. Re-asking for information the user already gave earlier in the same session is the clearest tell that context isn't persisting or retrieving correctly. Subtler failures appear as drift: semantic drift, where the agent slowly distorts facts through repeated summarization passes, and procedural drift, where it reinforces a suboptimal workflow pattern over the course of a session. Both degrade signal quality gradually rather than all at once, which makes them easy to miss in any single session and only visible once you're comparing early-session turns to late-session turns. Memory moved from an afterthought to a first-class architectural component over the past couple of years, and teams that still treat it as a detail tend to misattribute its failures to the model instead.

Workflow-layer failures tend to occur right after a decision point. Escalation to a human immediately following a routing decision usually means the workflow sent the session down the wrong branch. Abandonment right after a handoff to a different sub-agent often means the handoff lost context or exposed a capability gap the user had no way around. Runaway loops belong here too, and they're a category standard observability simply can't see without instrumentation built specifically to watch for agentic behavior.

The noise problem in implicit signals

Not every signal deserves the same weight, and pretending otherwise will send a team chasing fixes that don't help. Liu et al.'s 2025 study is the clearest warning on this front: folding implicit feedback content into model training helped on short, human-designed test questions from MTBench, but not on longer, more complex questions from WildBench. Implicit signal is genuinely useful for understanding what users are experiencing. It is noisier, and less reliable, as a direct lever for improvement than that first fact might suggest.

The contamination from positive-feedback bias compounds this. Liu et al. found that prompts which draw out positive feedback from users can actually be lower quality, and more toxic, than a random sample of prompts. So a system that blindly promotes any response that got a "great job" is at real risk of optimizing toward the wrong target entirely.

Attribution errors carry the same kind of hidden bias. That roughly 10% misattribution rate from the retailer postmortem wasn't scattered randomly across causes, it was systematically skewed toward blaming whichever technology happened to be mentioned alongside the incident. The same bias risk applies to behavioral signals: read them without checking the full trajectory, and the pattern that looks obvious is often just whatever was co-located with the failure, not what caused it.

There's a related failure mode: the "Continual Search" framework, built to address failure diagnosis in LLM-based systems, targets the tendency of a model to lock onto a plausible-looking cause before it's actually examined all the... The "Continual Search" framework, built to address failure diagnosis in LLM-based systems, targets the tendency of a model to lock onto a plausible-looking cause before it's actually examined all the available evidence. Its fix is to keep nudging the model across successive turns to revisit evidence it hasn't fully accounted for yet, rather than letting it settle on the first explanation that fits.

The practical fix for all of this is to stop reading signals one session at a time. A single repetition might just be a user who mistyped something. A single escalation might be a genuine edge case sitting outside what the agent was ever built to handle. The signal only becomes something worth acting on once it clusters across many sessions, correlates with a specific harness configuration, and stands out clearly against a baseline distribution of normal behavior. Treat every individual repetition as proof of a prompt failure, and the improvement backlog fills up with noise. The real discipline is spotting the pattern first, then attributing a cause, never the reverse.

Building a continuous feedback loop from behavioral signals to validated harness changes

Picture the team without any of this in place. A new prompt ships on a Friday. By Monday, escalations to human support are up, but nobody can say which slice of users is affected, because the dashboard tracks token usage and latency and nothing resembling quality. Every deployment is a coin flip, and every regression gets discovered by an angry customer instead of a test suite. That's not a hypothetical edge case; it's the default state for teams that haven't built this loop yet.

The loop itself runs in four stages, each one depending on the last. Signal collection comes first: instrument traces with typed schemas capturing tool-call, reasoning, state, and memory spans, and wire behavioral events, repetition, escalation, abandonment, directly to the span that preceded them. Pattern detection comes next, clustering failure signals by harness layer and task type, since a single event is noise but a pattern repeating across sessions is a genuine root-cause candidate. Fix generation follows: a prompt rewrite, a tool schema correction, a workflow reroute, a memory persistence fix, scoped narrowly to whichever layer the pattern actually implicated, not rolled out globally on a hunch. Replay validation closes the loop, running the proposed fix against historical traces that contain the original failure pattern. Golden-set replay checks for regressions against a fixed reference set, while production-side scoring on live traffic catches distribution shift the golden set never anticipated.

That last stage is where most teams currently fall down. Observability adoption has outpaced formal evaluation: 89% of teams have built some form of observability, but only 52% have built formal evals to go with it. That gap means most teams can see the logs perfectly well but have no reliable way to know whether a proposed fix actually resolves the pattern it's aimed at, or quietly introduces a regression somewhere else in the system.

Two complementary metrics matter once evals are in place: pass@k, which checks whether the agent succeeds in at least one of k attempts, and pass^k, the stricter bar requiring success on every single run. For a customer-facing agent, one inconsistent run does as much damage to trust as ten consistent failures, so pass^k is closer to the standard that actually matters. It's also the harder one to hit, which is exactly why most teams don't measure it yet, and exactly why the ones that do end up shipping with more confidence than the rest.