OpenTelemetry Integration

AI agents create unpredictable usage patterns and complex request flows that are hard to monitor with traditional methods. The Apollo MCP Server's OpenTelemetry integration provides the visibility you need to run a reliable service for AI agents.

What you can monitor

• Agent behavior: Which tools and operations are used most frequently

• Performance: Response times and bottlenecks across tool executions and GraphQL operations

• Reliability: Error rates, failed operations, and request success patterns

• Distributed request flows: Complete traces from agent request through your Apollo Router and subgraphs, with automatic trace context propagation

How it works

Apollo MCP Server exports metrics and traces using the OpenTelemetry Protocol (OTLP). Application logs use the configured logging output instead of an OTLP logs exporter.

Usage guide

Quick start: Local development

The fastest way to see Apollo MCP Server telemetry in action is with a local setup that requires only Docker.

5-minute setup

1. Start local observability stack:

   docker run -p 3000:3000 -p 4317:4317 -p 4318:4318 --rm -ti grafana/otel-lgtm

2. Add telemetry config to your config.yaml:

   YAML

   copy

   1telemetry:
   2  exporters:
   3    metrics:
   4      otlp:
   5        endpoint: "http://localhost:4318/v1/metrics"
   6        protocol: "http/protobuf"
   7    tracing:
   8      otlp:
   9        endpoint: "http://localhost:4318/v1/traces"
   10        protocol: "http/protobuf"

3. Restart your MCP server with the updated config

4. Open Grafana at http://localhost:3000 and explore your telemetry data. Default credentials are username admin with password admin.

Production deployment

For production environments, configure your MCP server to send metrics and traces to any OTLP-compatible backend. The Apollo MCP Server uses standard OpenTelemetry protocols, which enables compatibility with all major observability platforms.

Configuration example

1telemetry:
2  service_name: "mcp-server-prod" # Custom service name
3  exporters:
4    metrics:
5      otlp:
6        endpoint: "https://your-metrics-endpoint"
7        protocol: "http/protobuf" # or "grpc"
8    tracing:
9      otlp:
10        endpoint: "https://your-traces-endpoint"
11        protocol: "http/protobuf"

Observability platform integration

The MCP server works with any OTLP-compatible backend. Consult your provider's documentation for specific endpoint URLs and authentication:

• Datadog OTLP Integration - Native OTLP support

• New Relic OpenTelemetry - Direct OTLP ingestion

• AWS Observability - Via AWS Distro for OpenTelemetry

• Grafana Cloud - Hosted Grafana with OTLP

• Honeycomb - OpenTelemetry-native platform

• Jaeger - Self-hosted tracing

• OpenTelemetry Collector - Self-hosted with flexible routing

Pre-built dashboard templates

Apollo provides ready-to-use dashboard templates for monitoring your Apollo MCP Server. These templates are available in the apollographql/apm-templates repository and include visualizations for tool calls, HTTP server metrics, and request lifecycle events.

Import these templates into your observability platform to get started quickly with pre-configured graphs and alerts.

Production configuration best practices

Environment and security

1# Set via environment variable
2export ENVIRONMENT=production
3
4telemetry:
5  service_name: "apollo-mcp-server"
6  version: "1.0.0"                     # Version for correlation
7  exporters:
8    metrics:
9      otlp:
10        endpoint: "https://secure-endpoint"  # Always use HTTPS
11        protocol: "http/protobuf"           # Generally more reliable than gRPC

Performance considerations

• Protocol choice: http/protobuf is often more reliable through firewalls and load balancers than grpc

• Export interval: Metrics are exported every 30 seconds by default. Adjust this interval time using the export_interval config to balance freshness and network overhead.

• Batch export: OpenTelemetry automatically batches telemetry data for efficiency

• Network timeouts: Default timeouts are usually appropriate, but monitor for network issues

Resource correlation

• The ENVIRONMENT variable automatically tags all telemetry with deployment.environment.name

• Use consistent service_name across all your Apollo infrastructure (Router, subgraphs, MCP server)

• Set version to track releases and correlate issues with deployments

• Log lines emitted inside active spans include the span's trace_id, enabling trace-to-log correlation. Metrics use resource attributes and metric labels instead of per-request trace IDs.

Troubleshooting

Common issues

• Connection refused: Verify endpoint URL and network connectivity

• Authentication errors: Check if your provider requires API keys or special headers

• Missing data: Confirm your observability platform supports OTLP and is configured to receive data

• High memory usage: Monitor telemetry export frequency and consider sampling for high-volume environments

Verification

1# Check if telemetry is being exported (look for connection attempts)
2curl -v https://your-endpoint/v1/metrics
3
4# Monitor server logs for OpenTelemetry export errors
5./apollo-mcp-server config.yaml 2>&1 | grep -i "otel\|telemetry"

Configuration Reference

The OpenTelemetry integration is configured via the telemetry section of the configuration reference page.

Emitted Metrics

The server emits the following metrics, which are invaluable for monitoring and alerting. All duration metrics are in milliseconds.

In addition to these metrics, the server emits the standard HTTP server metrics for every request to the MCP endpoint, courtesy of the axum-otel-metrics library.

Stability is the status each metric carries in the OpenTelemetry semantic conventions. An experimental metric can be renamed by a future revision of the conventions, so pin a dashboard or an alert to one only if you're willing to update it.

Emitted Traces

Spans are generated for the following actions:

• Incoming HTTP Requests: The server creates a SERVER span for every HTTP request to the MCP endpoint. That span roots the trace unless the request carries a traceparent header. The server doesn't trace requests to the health check endpoint, so probes don't appear as service entry points.

• MCP Handler Methods: Nested spans are created for each of the main MCP protocol methods (initialize, call_tool, list_tools).

• Tool Execution: call_tool spans contain nested spans for the specific tool being executed (e.g., introspect, search, or a custom GraphQL operation).

• Downstream GraphQL Calls: The execute tool in Apollo MCP Server and custom operation tools create child spans for their outgoing reqwest HTTP calls, capturing the duration of the downstream request. The traceparent, tracestate, and baggage headers propagate automatically. Baggage is caller-controlled, request-scoped context, not identity. Don't put secrets or personally identifiable information in baggage, and never use it for authorization. If your GraphQL endpoint crosses a trust boundary, filter baggage before forwarding it. Apollo MCP Server doesn't automatically copy baggage onto exported span attributes.

Span Attributes

The inbound request span is named {method} {route}, for example POST /mcp. A request to a subpath of the MCP endpoint, such as /mcp/anything, matches no route, so the server names that span after its method alone. The span follows the OpenTelemetry semantic conventions for HTTP server spans:

Attributes whose source header is absent are omitted rather than recorded as empty. A request whose method falls outside the conventions' set is named HTTP {route}, so an arbitrary method can't multiply the span names a backend has to index.

The server sets the span status to error for a 5xx response and leaves it unset otherwise, which is what the conventions require. A POST span stays open until the response body finishes streaming, so its duration covers tool execution on a streamed response. A GET span instead ends at the response head, because that request is the session's standing server-to-client stream and timing it to the body would report session lifetime as request latency. Spans the server emits over that stream later aren't nested under it.

The call_tool span includes the following attributes:

For tools that execute a downstream GraphQL operation, the child execute span includes:

These attributes are populated only after the corresponding value is available and can be serialized. Early validation errors, network failures, and response-decoding failures can leave one or more of your attributes unset.

Sensitive data in traces

When you configure tracing export, the server exports populated span attributes as provided unless you omit them. Tool arguments, tool results, GraphQL queries, and GraphQL responses can contain identifiers, user-provided values, or other data that requires restricted handling.

For operations that use the MCP Apps @private directive, Apollo MCP Server records the LLM-visible GraphQL response and excludes the response metadata containing the full @private result from the tool_result span attribute. This protects those marked response fields but doesn't redact tool arguments, query text, or other response fields.

apollo.mcp.session_id is a session identifier rather than payload data, and the server records it only for a request it accepted, so a rejected caller can't write a value into it. With stateful_mode: false there are no sessions to look up, so the server records the header as the client sent it; omit the attribute if an unvalidated caller-supplied value matters to you. It's listed with the other apollo.mcp.* attributes and can be omitted the same way.

Exclude the four payload attributes from exported traces by configuring the tracing exporter:

1telemetry:
2  exporters:
3    tracing:
4      otlp:
5        endpoint: https://your-traces-endpoint/v1/traces
6        protocol: http/protobuf
7      omitted_attributes:
8        - tool_arguments
9        - tool_result
10        - graphql_query
11        - graphql_response

Omitting payload attributes removes payload-level detail while retaining useful trace structure, timing, status, and any attributes you don't omit. Select omissions according to your investigation needs and data-handling requirements.

Sampling and attribute filtering

High-cardinality metrics can occur in MCP Servers with large number of tools or when clients are allowed to generate freeform operations. To prevent performance issues and reduce costs, the Apollo MCP Server provides two mechanisms to control metric cardinality, trace sampling and attribute filtering.

Trace Sampling

Configure Apollo MCP Server to sample traces sent to your OpenTelemetry Collector using the sampler field in the telemetry.exporters.tracing configuration:

• always_on - Send every trace

• always_off - Disable trace collection entirely

• 0.0-1.0 - Send a specified percentage of traces

Attribute Filtering

Trace and metric exporters have independent omitted_attributes lists:

• telemetry.exporters.tracing.omitted_attributes removes the selected apollo.* attributes from spans exported by this server

• telemetry.exporters.metrics.omitted_attributes removes the selected supported labels from Apollo metrics

Use short attribute names such as tool_name, operation_id, tool_arguments, and graphql_response. For detailed configuration options, go to the telemetry configuration reference.

Audit and deployment responsibilities

Apollo MCP Server telemetry provides request traces, operational metrics, and correlated application logs that can support investigations. A complete audit trail also depends on the surrounding observability system. Configure the collector and backend for centralized storage, access controls, retention, integrity protection, alerting, and periodic review appropriate to your environment.

Visit OWASP MCP08: Lack of Audit and Telemetry for additional audit and monitoring considerations.