How Do You Use OpenTelemetry to Monitor AI Agents in 2026?

What OpenTelemetry Actually Gives AI Agent Teams

OpenTelemetry is the practical answer for teams that need one telemetry standard across AI agents, ordinary services, databases, and third-party model APIs. An agent may call a language model, execute code, query a vector database, invoke a payment API, and delegate work to another agent. Traditional application traces can capture the HTTP calls, but they often hide which prompt, model, retrieval result, or tool decision produced the final behavior. OpenTelemetry supplies a vendor-neutral way to create and export traces, metrics, and logs through a common instrumentation API. The project is hosted by the Cloud Native Computing Foundation, and its generative-AI semantic conventions provide a shared vocabulary for recording model interactions.

Also worth reading: How Do Teams Monitor AI Agents in Production Without Missing Failures? · How do OpenTelemetry sampling strategies work and which one should you implement? · What are agentic AI tracing frameworks and how do they monitor autonomous agent workflows?

That distinction matters because a production agent is not just a single endpoint receiving a prompt and returning text. It is a chain of decisions that can last 5 seconds or 20 minutes, consume 20,000 tokens, call 12 tools, and make an irreversible change. OpenTelemetry can connect the parent request to each child operation, including model inference, retrieval, tool execution, and handoffs. This makes it easier to answer whether latency came from the model, a search service, or a slow authorization check. It also gives platform teams data they can route to multiple backends instead of binding the agent application to one monitoring vendor.

OpenTelemetry does not, however, make an agent reliable by itself. Instrumentation records behavior; engineering teams still need appropriate quality targets, privacy rules, and evaluation criteria. It also does not automatically explain whether an answer was accurate, ethical, or useful. A trace can prove that an agent selected Tool A with 87% confidence, but it cannot prove that selecting Tool A was correct. For that reason, the strongest implementations combine OpenTelemetry traces with task-level evaluations, business outcomes, and periodic human review. As of September 2026, adoption is broadening across products such as Databricks, AWS, Oracle, New Relic, and Dynatrace, but the conventions and vendor support should still be checked against the versions used in a particular stack.

The Agent Operations Worth Tracing

A useful OpenTelemetry implementation begins by treating the agent as a sequence of observable operations rather than a magical model response. The root span should normally represent the user or system request, with child spans for planning, model generation, retrieval, tool calls, validation, and final response delivery. Each model call should record the provider, approved model name, operation type, input and output token counts, latency, finish reason, and relevant request identifiers. Token counts are especially important because they connect technical telemetry to a major variable cost; a trace that lacks them makes it difficult to determine why one successful run costs 0.03 dollars while another costs 0.90 dollars.

Retrieval deserves its own span and attributes. Record the query or search intent where privacy policy permits, the index or data source, the number of candidates returned, the number selected, latency, and a score distribution. A count of 10 retrieved documents does not mean all 10 reached the model, and a similarity score of 0.82 is not universally comparable across vector databases. Teams should therefore establish local thresholds based on evaluation data rather than assume that one score proves relevance. The same principle applies to agent memory: capture whether memory was read or written, its source, its age, and whether a later step used it.

Tool calls should preserve both the technical and the semantic meaning of the operation. Record the tool name, normalized arguments after redaction, execution result category, error type, duration, and whether the call changed external state. Never place credentials, raw secrets, unrestricted personal data, or complete sensitive prompts into span attributes by default. Business-specific attributes can include ticket status, transaction identifier hash, approved policy version, and the step's expected outcome. The objective is not to collect everything; collecting everything usually increases cost and weakens security. A 2026 production system might retain detailed traces for 7 days, sampled operational metrics for 30 to 90 days, and aggregated evaluation records for longer, but those durations are starting points rather than universal rules.

A Practical Implementation Workflow

Start with a single high-value workflow rather than attempting to instrument an entire agent platform in the first week. A customer-support agent that searches a knowledge base and creates a ticket is often a better pilot than a general coding agent with broad filesystem access. Define 8 to 12 questions before coding: Which model call was slowest? Which tool failed? How many tokens were consumed? Where did retrieval produce no useful context? How often did the workflow require a retry? How many completed tasks met the defined success criterion? These questions determine which spans and metrics are actually useful.

Then map the execution path and attach spans to real code boundaries. In a Python or TypeScript service, create a root span when the agent run begins and propagate trace context into HTTP requests, database clients, queues, and child agents. Use OpenTelemetry's auto-instrumentation where it reduces repetitive setup, but add manual instrumentation around prompts, model invocations, retrieval, and domain tools. Attach only the context that a child system can safely accept; some providers strip or rewrite trace headers, so verify propagation in a test rather than assuming every external call appears in the same trace. For agent-to-agent work, use a clear parent-child or link relationship rather than making every operation part of one misleading 40-minute trace.

Export to a collector, which is the central component that receives, processes, and forwards telemetry. Configure batching, memory limits, redaction processors, and retry behavior before moving to production. As a practical starting point, export batches of 512 spans when throughput and memory budgets allow, then adjust using observed payload size and delivery delay. A collector is not automatically a high-availability telemetry system; design for the possibility that the monitoring backend is unavailable without blocking the user's business operation. Sampling should preserve errors, unusually expensive runs, and a small statistical sample of successful requests. Exact token or character thresholds are workload-specific, so a run consuming 200,000 tokens deserves investigation even if it completes successfully.

Finally, validate the trace with known synthetic tests before trusting the dashboard. Create a run that makes 3 model calls, performs 2 retrievals, invokes 1 failing tool, and finishes in a controlled duration. Confirm that the expected spans, attributes, error status, and parent relationships appear. A dashboard showing 1,000 runs is not evidence of complete instrumentation if a tool's internal errors are collapsed into a generic model error. Record a trace schema version so later convention changes do not silently mix incompatible attribute names.

OpenTelemetry Tracing Compared with Agent-Specific Options

The main choice is usually between building telemetry directly around an agent framework, using a commercial AI-observability product, or adopting OpenTelemetry as the shared layer beneath one or more products. The options are not mutually exclusive, and many teams use more than one. The table compares their practical strengths rather than declaring a universal winner.

OpenTelemetry is particularly attractive when agents sit inside a distributed architecture that already uses Kubernetes, Kafka, PostgreSQL, and multiple cloud services. It lets an SRE team manage telemetry contracts across both AI and non-AI traffic. Agent-specific platforms can offer faster time to value, richer prompt analysis, and built-in evaluation workflows, but they introduce a second telemetry vocabulary and may create lock-in. Framework-native tracing is sensible for learning the boundaries of one library, yet it can leave gaps when the agent calls a model through a gateway, stores state in a proprietary memory service, or coordinates work through a queue.

A sensible decision rule is to require interoperability before requiring a richer interface. If two systems can consume the same normalized trace, teams retain portability and can compare vendors later. If an agent-specific product can export useful OpenTelemetry data, it can provide convenience without becoming the only place the information exists. Ask whether the tool supports model-call attributes, token accounting, retrieval, tool errors, handoffs, and custom business outcomes. Do not count a marketing phrase such as "supports OpenTelemetry" as proof that all those fields are populated.

Turning Traces Into SLOs and Useful Alerts

An AI agent should not be judged with a single average latency or error-rate metric. One workflow may have a 5-second target for classification and a 90-second target for a multi-step research task. Divide the agent into stages and assign a service-level indicator to each stage that has a meaningful user consequence. A model-call SLO could measure successful responses over time, a retrieval SLO could measure availability and freshness, and a workflow SLO could measure completed tasks that satisfy a validated outcome. Keep 3 to 5 primary indicators at first; adding 30 metrics often creates more dashboards but no better decisions.

Example targets should be based on a baseline rather than arbitrary numbers. If the median customer-support workflow is 7 seconds, a reasonable initial objective might be that 95% finish within 20 seconds and 99% of runs do not produce an unhandled error. For retrieval freshness, perhaps 98% of answers use documents no older than 30 days, while at least 90% of test cases meet a human-reviewed relevance threshold. These are illustrative governance choices, not OpenTelemetry standards. Measure how each target behaves during a 2-week baseline, review tail behavior by customer or model, and revise the target when the underlying service changes.

OpenTelemetry excels at producing evidence for these SLOs, but alert design still needs judgment. A 5% error increase may be serious in a payments agent and negligible in an asynchronous research queue. Route urgent pages for data loss, unauthorized tool use, sustained workflow failure, or cost anomalies; use ticket-level notifications for slow but eventually successful runs. Burn-rate alerts are useful for availability problems, while token-per-task alerts can identify a model change, runaway loops, or an unexpectedly large context. Do not alert on every 1-second latency fluctuation. A useful system distinguishes a 60-second incident from a week-long drift, and preserves enough trace context for an engineer to begin debugging immediately.

Quality evaluation belongs alongside the SLO. A sample of traces can feed offline judges, deterministic checks, or human review, but those evaluators have their own error rates. Keep the judge version, rubric, model version, and sample size with each evaluation result. Report confidence intervals when the sample is small; 5 favorable answers out of 5 is not a 100% success rate for all tasks. In one measurement exercise, the familiar idea of building 15 SLOs demonstrated how many distinct signals an agent can generate, but 15 SLOs do not automatically mean 15 useful service commitments. Prioritize indicators that an operator can change and a user can feel.

Common Instrumentation Mistakes

The first mistake is treating the final response as the only observable event. That approach misses failed retrievals, repeated tool calls, discarded plans, and retries that eventually produced an apparently good answer. Add spans at decision boundaries and record whether a step was attempted, selected, skipped, or rejected. A final answer can hide 3 unnecessary searches or a second agent call that was caused by a timeout. The trace should show the path taken, not merely the path that happened to finish.

The second mistake is dumping prompts, completions, and tool arguments into every span. This can expose protected data, inflate payloads, and make telemetry storage more expensive than the model API itself. Redact secrets before export, use sampling for successful verbose runs, and retain full interaction content only under an approved, short-lived policy. Hashing a customer identifier is not always sufficient if the hash can be reversed or combined with other attributes. Review what downstream dashboards and third-party SaaS vendors can access, and test the redaction processor with realistic edge cases.

The third mistake is assuming a trace is complete because it contains a model span. Check queue waits, asynchronous work, gateway calls, and agent handoffs, which often occur outside the process that starts the workflow. A child span missing from the trace is not proof that the child operation did not happen. Conversely, a vendor may report a successful call while the model returned a refusal or malformed structured output. Record domain-specific status separately from transport status. Also avoid hard-coding framework event names as permanent contracts; map them to your own stable schema and document the version.

Finally, do not compare model quality, cost, or latency without controlling for the workload. Temperature, context length, tool choice, retrieval results, and network location all change. When a new model appears to improve quality, evaluate a fixed set of representative tasks and record token use, latency percentiles, and failure modes. A 12% latency improvement is meaningless if success falls from 96% to 91%, just as a cheaper model is unattractive if it causes more manual rework. Version the agent prompt, policy, model, and tool configuration so a later comparison has something trustworthy to compare.

Cost, Storage, and Operational Trade-offs

OpenTelemetry itself is open-source software, so using the libraries and collector does not require a license fee. The real costs are engineering time, telemetry volume, storage, network transfer, and the observability platform that receives the data. A 20,000-token prompt may be small in application terms but substantial when stored and searched repeatedly. A simple production baseline might keep counts and durations in metrics, keep sampled detailed traces for 7 to 30 days, and keep only aggregate evaluation data for 90 days or longer. A regulated environment may require shorter retention for prompts, while a debugging environment may deliberately retain more under restricted access.

Sampling should be designed around questions rather than percentages alone. A rate of 10% may be reasonable for ordinary successful runs, but a fixed rate can hide a rare failure that affects a high-value customer. Retain all 100% of failed or unauthorized runs for a defined initial period if policy allows, and preserve a smaller sample of successful long or expensive runs. The goal is a bounded and explainable bill, not the cheapest possible export. Before enabling high-cardinality labels, check whether the backend can index them efficiently. Agent names, full prompt strings, and unbounded exception messages can otherwise create a storage problem disguised as observability.

Commercial agent-observability pricing is harder to generalize because providers change plans and measure different units. Some packages price by ingested spans, others by active agents, hosts, seats, events, or retained data. A 2026 purchase should therefore request a written formula and examples for an agent with 1 million monthly runs, an average of 8 spans per run, and a 20% failure rate. The same price may look inexpensive at 100,000 spans and expensive at 800,000. Compare 12-month retention, query and API limits, evaluation features, OpenTelemetry export, support, and egress charges. Do not infer a numeric market average from a vendor announcement.

When Teams Should Adopt It

Adopt OpenTelemetry when the agent is already part of a distributed production system, incidents span several services, or more than one monitoring destination may be required. It is especially useful after the first serious incident reveals that logs and model dashboards cannot reconstruct the run. A reasonable pilot can begin with 1 workflow, 4 core spans, 3 operational metrics, and 1 business outcome, then expand after engineers confirm that the data answers real questions. Teams should not wait for a perfect semantic-convention release; stable local attributes and a collector pipeline can be introduced while documenting compatibility gaps.

A small prototype may not need the full investment. A local coding script with 2 tools and 100 monthly runs can often use framework-native logs, bounded JSON files, and a simple trace viewer. The economics change when runs become asynchronous, prompts contain customer data, or several teams share ownership of the workflow. In that situation, redaction, sampling, retention, access control, and backend migration costs matter more than saving a few hours of setup. For open-source tutorials, demonstrate the smallest instrumented example first, then show how to export it to a collector and connect a dashboard. That sequence teaches the underlying method instead of presenting a vendor as the only route to visibility.

Keep evaluating alternatives as the ecosystem changes. OpenTelemetry conventions for generative AI have matured, but field names and support differ by library and provider. AgentCore, Databricks, Oracle, AWS, New Relic, and Dynatrace all describe OpenTelemetry-related workflows, yet their products solve different parts of the stack. Check current documentation, release notes, and data-export behavior before committing. A tutorial written in September 2026 should be treated as a current starting point, not a guarantee that every attribute or integration remains unchanged in 2027.

The Practical Recommendation

For most production teams, OpenTelemetry for AI agents is best used as the shared measurement foundation, not as the entire quality program. Instrument the root workflow, model calls, retrieval, tools, handoffs, errors, token usage, cost-relevant metadata, and validated task outcomes. Send telemetry through a collector, protect sensitive content before export, and retain representative traces long enough to investigate failures. Then build a small number of SLOs that reflect user-visible reliability and cost rather than simply the volume of data on a dashboard.

The key decision is whether the team's immediate priority is portability, speed, or specialized AI analysis. OpenTelemetry plus an existing backend usually wins when heterogeneous services and future migration matter. An agent-specific platform can win when rapid deployment, built-in evaluations, and prompt-oriented workflows matter more than a single telemetry standard. Framework-native tools remain useful for prototypes and tightly coupled applications. Combining them is often pragmatic: use the agent product for its analysis interface, but keep a documented OpenTelemetry path for platform-wide searching and vendor comparison. That approach makes the agent observable without pretending that instrumentation alone can judge the agent.