Spans
Add router lifecycle context to traces
A span captures contextual information about requests and responses as they're processed through the router's request lifecycle (pipeline) . The information from spans can be used when displaying traces in your application performance monitors (APM).
Spans configuration
Router request lifecycle services
A router's request lifecycle has three major services:
Router service - Handles an incoming request before it is parsed. Works within a context of opaque bytes.
Supergraph service - Handles a request after it has been parsed and before it is sent to the subgraph. Works within a GraphQL context.
Subgraph service - Handles a request after it has been sent to the subgraph. Works within a GraphQL context.
The router
, supergraph
and subgraph
sections are used to define custom span configuration for each service:
1telemetry:
2 instrumentation:
3 spans:
4 router: # highlight-line
5 attributes: {}
6 # ...
7 supergraph: # highlight-line
8 attributes: {}
9 # ...
10 subgraph: # highlight-line
11 attributes: {}
12 # ...
attributes
Spans may have attributes attached to them from the router pipeline. These attributes are used to filter and group spans in your APM.
Attributes may be drawn from standard attributes or selectors .
The attributes that are available depend on the service of the pipeline.
1telemetry:
2 instrumentation:
3 spans:
4 router:
5 attributes:
6 # Standard attributes
7 http.response.status_code: true
8 # Custom attributes
9 "my_attribute":
10 response_header: "x-my-header"
You can also have conditions on custom attributes using selectors . You can only have conditions on a selector at the same execution level.
Example you can't have a condition on response_header
if you want to set an attribute from request_header
.
1telemetry:
2 instrumentation:
3 spans:
4 router:
5 attributes:
6 on_error:
7 response_status: reason
8 condition:
9 not:
10 eq:
11 - response_status: code
12 - 200
default_attribute_requirement_level
The default_attribute_requirement_level
option sets the default attributes to attach to spans, as defined by OpenTelemetry semantic conventions .
Valid values:
required
(default)recommended
1telemetry:
2 instrumentation:
3 spans:
4 # Set the default requirement level
5 default_attribute_requirement_level: required #highlight-line
Attributes can be configured individually, so that required
attributes can be overridden or disabled. For example, http.response.status_code
is set individually to override the standard value:
1telemetry:
2 instrumentation:
3 spans:
4 # Set the default requirement level
5 default_attribute_requirement_level: required
6 router:
7 attributes:
8 # Disable standard attribute
9 http.response.status_code: false
opt-in
must be configured individually.mode
The mode
option enables the router spans to either use legacy attributes in the router, or those defined in the OpenTelemetry specification.
Valid values:
spec_compliant
deprecated
(default)
spec_compliant
This mode follows the OpenTelemetry spec. Attributes that were previously added to spans that did not follow conventions are now removed.
You will likely gain a significant performance improvement by using this mode as it reduces the number of attributes that are added to spans.
1telemetry:
2 instrumentation:
3 spans:
4 mode: spec_compliant
For now this is not the default, however it will be in a future release.
deprecated
This mode is the default and follows the previous behavior if you have not configured the mode
option to spec_compliant
.
1telemetry:
2 instrumentation:
3 spans:
4 mode: deprecated
Attributes are added to spans that do not follow OpenTelemetry conventions.
mode
option will be defaulted to spec_compliant
in a future release, and eventually removed.Span status
By default spans are marked in error only if the http status code is different than 200. If you want to mark a span in error for other reason you can override the otel.status_code
attribute which is responsible to mark a span in error or not.
If it's in error then otel.status_code
= error
, if not it will be ok
.
Naming
By default, we will use a span naming convention that aligns with the current semantinc conventions for GraphQL server in OpenTelemetry which means the root span name
must be of format <graphql.operation.type> <graphql.operation.name>
provided that graphql.operation.type
and graphql.operation.name
are available.
If you want to change the name of spans we're creating for each services you can override this value by setting the otel.name
attribute using any selectors you want.
Here is an example if you want to mark the router
and supergraph
span in error if you have a graphql error in the payload and you want to enforce the router
span name to be graphql_router
.
1telemetry:
2 instrumentation:
3 spans:
4 router:
5 attributes:
6 otel.name:
7 static: graphql_router # Override the span name to graphql_router
8 otel.status_code:
9 static: error
10 condition:
11 eq:
12 - true
13 - on_graphql_error: true
14 supergraph:
15 attributes:
16 otel.status_code:
17 static: error
18 condition:
19 exists:
20 response_errors: $[0].extensions.code # Here is an example to get the first error code, `on_graphql_error` is also available for supergraph
Span configuration example
An example configuration of telemetry.spans
in router.yaml
sets both standard and custom attributes for the router service:
1telemetry:
2 instrumentation:
3 spans:
4 default_attribute_requirement_level: required
5 mode: spec_compliant
6 router:
7 attributes:
8 # Standard attributes (http)
9 dd.trace_id: false
10 http.request.body.size: false
11 http.response.body.size: false
12 http.request.method: false
13 # ...
14
15 # Conditional custom attribute
16 otel.status_description: # You can conditionally put a status description attribute on your span if it respect the condition
17 static: "there was an error"
18 condition: # http response status code != 200 or it contains a graphql error in the payload
19 any:
20 - not:
21 eq:
22 - response_status: code
23 - 200
24 - eq:
25 - on_graphql_error
26 - true
27 # Custom attributes
28 "acme.custom_1":
29 trace_id: datadog
30 "acme.custom_2":
31 response_header: "X-CUSTOM2"
32 default: "unknown"
33 "acme.custom_3":
34 env: "ENV_VAR"
35 "static_attribute":
36 static: "my_static_value"
37 # ...
38
39 supergraph:
40 attributes: {}
41 # ...
42 subgraph:
43 attributes: {}
44 # ...
Spans configuration reference
Option | Values | Default | Description |
---|---|---|---|
<attribute-name> | The name of the custom attribute. | ||
attributes | standard attributes |selectors | The attributes of the span. | |
condition | conditions | The condition for adding a custom attribute. | |
default_attribute_requirement_level | required |recommended | required | The default attribute requirement level. |
mode | spec_compliant | deprecated | deprecated | The attributes of the span. |