AI observability in multi-agent systems using OpenTelemetry

Multi-agent systems have the potential to do amazing things, but their complexity makes communication between agents a real challenge. Unlike traditional software, these systems are made up of many independent agents that interact in unpredictable ways. This makes it hard to see how information is shared, how decisions are made, and how the system works as a whole. 

When agents don't have enough context or miss important messages, they can make poor decisions or behave unexpectedly. 

Without robust observability into these communication pathways, organizations face obstacles in pinpointing performance issues, diagnosing failures, ensuring reliability, and building trust in AI-driven multi-agent applications.  

Observability must extend beyond traditional monitoring to include instrumentation of communication and reasoning layers, giving insight into message flows, interaction patterns, and the decision-making behind agent collaboration. This transparency is crucial for effective debugging, continuous improvement, and making these complex systems reliable and trustworthy. 

The AGNTCY project: Building trust in agent-to-agent communications  

AGNTCY, a Linux Foundation project, is building the complete infrastructure stack for the Internet of Agents, an open, interoperable internet for agent-to-agent collaboration. This is where multi-agent systems will be able to work with any other agent, regardless of who built it or where it runs, much like TCP/IP unified the early internet.  

It provides essential standards and components such as Agent Directory Service, Secure Low-latency Messaging (SLIM), Agent Identity, and Agent Observability. The project supports pivotal protocols like Agent-to-Agent (A2A) for structured messaging between agents and Model Context Protocol (MCP) for agents to access external tools, which are central to the complex interactions within these systems. 

By embedding observability standards into AGTNCY, we can enable consistent monitoring and tracing of agent interactions across multi-agent systems.  

OpenTelemetry: The standard for AI observability  

OpenTelemetry (OTel) is an open source, vendor-neutral framework designed to generate, collect, and export metrics, traces, and logs. 

The rationale for using OTel standards, SDKs, and APIs to enable observability in multi-agent systems is compelling: 

• Unified telemetry collection: OTel provides a single SDK to collect all three types of telemetry data—metrics, traces, and logs—from diverse components within an artificial intelligence (AI) agent system. This eliminates the need for multiple, potentially conflicting monitoring tools.
• Consistent instrumentation and standardization: Its consistent API and semantic conventions ensure that telemetry data is structured uniformly across various services and components, which is critical for understanding distributed, complex agent interactions. This standardization is key to avoiding vendor lock-in, allowing organizations to choose their preferred observability backends (e.g., Jaeger, Grafana, Datadog) without re-instrumenting their code.
• Distributed tracing for complex workflows: OTel’s distributed tracing capabilities are paramount for multi-agent systems. They provide an end-to-end visualization of how requests and tasks flow through the system, showing the interactions between different agents, large language models (LLM), and external tools. This is essential for understanding the non-deterministic behavior, complex decision paths, and debugging issues across agent boundaries.
• Context propagation with baggage: OTel's "baggage" feature is designed to carry request-specific metadata, such as user IDs or session IDs, across service boundaries. AGNTCY's Observe SDK uses this capability, particularly in A2A communications, to maintain session affinity and trace continuity across distributed agent invocations. By propagating a session_id using the traceparent key, context is preserved even as tasks are delegated and processed by different agents.
• Language and environment agnostic: With SDKs available for major programming languages used in AI development, OTel provides a flexible solution that adapts to various deployment environments, from on-premises to cloud and containers. 

Use case of automatic context propagation in multi-agentic systems 

Baggage in OTel propagates request-specific metadata—like user IDs, client IDs, or account IDs—across services, enabling richer observability (traces, metrics, logs) downstream.  

Context 

a2a.py instrumentation module from the agntcy/observe GitHub repository, which includes real-world usage of baggage in A2A communications.  

This A2A instrumentation in Observe SDK from AGNTCY enables propagation of session metadata across A2A communication.  

A tracing span is created for a2a.send_message, and header-level propagation ensures continuity of trace context and session state (via the session_id) across the client-to-server boundary.

Implementation details 

• Baggage key/value:
  • Key: "execution.{traceparent}"  
  • Value: session_id (e.g., "session-12345")
• Propagation mechanism:
  • Client side (A2AClient.send_message):
    • Gets the current traceparent (trace context). 
    • Retrieves a session_id from a key-value store using traceparent as identifier. 
    • Sets both as HTTP headers (traceparent, session_id). 
    • Uses baggage.set_baggage(...) to attach the session context to the baggage header.
• Server side (DefaultRequestHandler.on_message_send): 
  •  Extracts headers from the request (traceparent, session_id).
  • Uses the World Wide Web Consortium (W3C) trace context and baggage propagators to rebuild the span context and baggage data. 
  • Updates internal session mappings (e.g., storing session ID back into kv_store) with the current context to maintain continuity. 
• Protocol/SDK: This is built on top of the A2A Python SDK, with instrumentation using OTel’s span API, W3C trace context, and baggage propagators. 

Specification relationship and observations 

• W3C baggage spec alignment:
  • The keys adhere to the ASCII token format (execution.{traceparent}), and values are simple strings—fully conforming to W3C baggage requirements.
  • Baggage is used minimally—to carry a single piece of context metadata (session_id), which is a string.
• Advantages:
  • Facilitates session affinity across distributed agent invocations.
  • Ensures traceability and context continuity across agent boundaries—critical in multi-agent or orchestrated workflows.
  • Uses established standards (W3C trace context, baggage) for interoperability. 

Summary table: A2A baggage instrumentation example

Embracing observability in multi-agentic systems 

By using OpenTelemetry, projects like AGNTCY are laying the groundwork for a truly observable and trustworthy AI ecosystem. This commitment to open standards for observability is vital for the successful deployment and management of the next generation of AI-powered applications. This real-world example shows exactly how baggage can be leveraged in agent-based systems to maintain session and trace continuity. 

Get started and use the AGNTCY Observe SDK on GitHub for multi-agent observability now.