Instruments

An instrument in the router collects data and reports measurements to a metric backend. Supported instruments include standard instruments from OpenTelemetry, standard instruments for the router's request lifecycle, and custom instruments. Supported instrument kinds are counters and histograms.

You can configure instruments in router.yaml with telemetry.instrumentation.instruments.

OpenTelemetry standard instruments

OpenTelemetry specifies multiple standard metric instruments that are available in the router:

• http.server.active_requests - The number of active requests in flight.
• http.server.request.body.size - A histogram of request body sizes for requests handled by the router.
• http.server.request.duration - A histogram of request durations for requests handled by the router.
• http.server.response.body.size - A histogram of response body sizes for requests handled by the router.

These instruments are configurable in router.yaml:

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        http.server.active_requests: true # (default false)
6
        http.server.request.body.size: true # (default false)
7
        http.server.request.duration: true # (default false)
8
        http.server.response.body.size: true # (default false)

They can be customized by attaching or removing attributes. See attributes to learn more about configuring attributes.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      default_requirement_level: required
5
      router:
6
        http.server.active_requests: 
7
          attributes:
8
            http.request.method: true

Apollo standard instruments

To learn about standard metric instruments for the router's request lifecycle, see Apollo Router instruments.

Custom instruments

You can define custom instruments on the router, supergraph, and subgraph services in the router pipeline.

In the example configuration below, three custom instruments are defined (acme.request.duration, acme.graphql.requests, acme.graphql.subgraph.errors), one for each service of the router pipeline respectively (router, supergraph, subgraph):

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        http.server.active_requests: true
6
        acme.request.duration:
7
          value: duration
8
          type: counter
9
          unit: kb
10
          description: "my description"
11
          condition:
12
            eq:
13
              - 200
14
              - response_status: code
15
          attributes:
16
            http.response.status_code: true
17
            "my_attribute":
18
              response_header: "x-my-header"
19
  
20
      supergraph:
21
        acme.graphql.requests:
22
          value: unit
23
          type: counter
24
          unit: count
25
          description: "supergraph requests"
26
          
27
      subgraph:
28
        acme.graphql.subgraph.errors:
29
          value: unit
30
          type: counter
31
          unit: count
32
          description: "my description"

Instrument naming conventions

Make sure to follow naming conventions for instruments. For reference, the OpenTelemetry semantic conventions can help guide you to:

• Choose a good name for your instrument.
• See which standard attributes can be attached to your instrument.

Some particular guidelines to note:

• Don't include the unit name in the metric name. For example, size_kb should be size and the unit should be kb.
• Don't include _total as a suffix. For example, use http.server.active_requests, not http.server.active_requests_total.
• Use dot notation to separate namespaces in the metric name. For example, use http.server.active_requests, not http_server_active_requests.

Instrument configuration

default_requirement_level

The default_requirement_level option sets the default attributes to attach to default standard instruments, as defined by OpenTelemetry semantic conventions.

Valid values:

• required (default) - required attributes will be attached to standard instruments by default.
• recommended - recommended attributes will be attached to standard instruments by default.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      # Set the default requirement level
5
      default_requirement_level: required 

Attributes can be configured individually, so that required attributes can be overridden or disabled. For example, http.response.status_code is set individually to override the standard value:

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      # Set the default requirement level
5
      default_requirement_level: required
6
      router:
7
        # Standard metrics
8
        http.server.request.body.size:
9
          attributes:
10
            # Standard attributes
11
            http.response.status_code: false
12
            # Custom attribute
13
            "acme.my_attribute":
14
              response_header: "x-my-header"
15
        # Standard metrics
16
        http.server.active_requests:
17
          attributes:
18
            # Standard attributes, different than other ones provides in standard metrics, custom attributes are not available on this standard metric
19
            http.request.method: false
20
            server.address: true
21
            server.port: true
22
            url.scheme: true

Router request lifecycle services

The Apollo Router's request lifecycle has three major services:

• Router service - Handles an incoming request before it is parsed. Works within a context of opaque bytes.
• Supergraph service - Handles a request after it has been parsed and before it is sent to the subgraph. Works within a GraphQL context.
• Subgraph service - Handles a request after it has been sent to the subgraph. Works within a GraphQL context.

The router, supergraph and subgraph sections are used to define custom instruments for each service.

To define a custom instrument, add a new key to router.yaml as telemetry.instruments.<service>.<custom-instrument>. For example, add a custom instrument acme.request.duration:

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router: 
5
        acme.request.duration: # The name of your custom instrument/metric
6
          value: duration
7
          type: counter
8
          unit: s
9
          description: "my description"

value

The value of an instrument is the value which will be drawn from. This can be one of the following:

• duration - the duration of the pipeline service.
• unit - the number of times the pipeline service has been executed.
• custom - a custom value extracted from the pipeline service. See selectors for more information.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        acme.metric:
6
          # ...
7
          value: duration

Values of custom metrics can be extracted from the pipeline using custom attributes. For example, to sum the contents of a request header, create a counter with value set as the request header:

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        acme.metric:
6
          # ...
7
          type: counter
8
          value:
9
           request_header: "x-my-header"

type

Instruments come in two different types:

• counter - A monotonic counter. For example, requests served, tasks completed, or errors occurred.
• histogram - A histogram of values. For example, request durations or response body sizes.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        acme.metric: 
6
          # ...
7
          type: counter # counter, histogram

unit

A free format unit that is displayed in your APM.

A unit is recommended to use SI units and definitions from The Unified Code for Units of Measure.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        acme.metric: 
6
          # ...
7
          unit: s # seconds

description

A free format description of the instrument that will be displayed in your APM.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        acme.metric: 
6
          # ...
7
          description: "my description"

condition

You may only want to mutate an instrument under certain conditions. For example, you may only want to increment a counter if the response status code is 200.

To do this use a condition:

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        acme.metric:
6
          # ...
7
          condition:
8
            eq:
9
              - 200
10
              - response_status: code

attributes

Instruments may have attributes attached to them from the router pipeline. These attributes are used to filter and group metrics in your APM.

Attributes may be drawn from standard attributes or selectors except for the standard metric http.server.active_requests.

The attributes available depend on the service of the pipeline.

1
telemetry:
2
  instrumentation:
3
    instruments:
4
      router:
5
        # Standard metrics
6
        http.server.request.body.size:
7
          attributes:
8
            # Standard attributes
9
            http.response.status_code: false
10
            # Custom attribute
11
            "acme.my_attribute":
12
              response_header: "x-my-header"
13
        # Standard metrics
14
        http.server.active_requests:
15
          attributes:
16
            # Standard attributes, different than other ones provides in standard metrics, custom attributes are not available on this standard metric
17
            http.request.method: false
18
            server.address: true
19
            server.port: true
20
            url.scheme: true
21
        # Custom metric 
22
        acme.metric: 
23
          value: duration
24
          type: counter
25
          unit: s
26
          description: "my description"
27
          attributes:
28
            http.response.status_code: true
29
            "my_attribute":
30
              # ...
31
              response_header: "x-my-header"

Instrument configuration reference