Router Logging to stdout

Configure logging to stdout

You can configure GraphOS Router or Apollo Router Core logging to be directed to stdout, and its output format can be set to text or JSON.

For general logging configuration, refer to Router Logging Configuration.

stdout configuration

`enabled`

The stdout logging output is disabled by default.

To enable stdout logging, set the enabled option to true:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true

`format`

You can configure the logging output format. The default format depends on how the router is run:

In an interactive shell, text is the default.
In a non-interactive shell, json is the default.

You can explicitly set the format in router.yaml with telemetry.exporters.logging.stdout.format:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: text

`tty_format`

You can configure the log format when you're running on an interactive shell. This is useful during development.

If both format and tty_format are configured then the output depends on the environment where the router is run:

In an interactive shell, tty_format will take precedence.
In a non-interactive shell, format will take precedence.

You can explicitly set the format in router.yaml with telemetry.exporters.logging.stdout.tty_format:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: json
7         tty_format: text

`rate_limit`

The rate at which log messages are produced can become too high, especially for request processing errors. To prevent the router from filling its logs with redundant messages, you can use the rate_limit option to set the logging rate limit.

You can set the logging rate limit for each log location, where different log messages are rate limited independently, and log lines with the same message but different sets of attributes are limited under the same rate. This rate limiting only applies to logging to stdout and doesn't affect events sent to trace exporters like OTLP, which have their own sampling configuration.

To enable rate limiting, set the rate_limit option:

YAML

router.yaml

1telemetry:
2  exporters:
3    logging:
4      stdout:
5        format: json
6        rate_limit:
7          capacity: 1  # number of allowed messages during the rate limiting interval
8          interval: 3s

For configuration options specific to each output format, see the text and json format references.

Configuration reference

Option	Values	Default	Description
`enabled`	`true`\|`false`	`false`	Enable or disable stdout logging.
`format`		`text`\|`json`	See the format documentation for details.
`tty_format`		`text`\|`json`	See the format documentation for details.

Logging output format

You can configure logging to be output in different formats:

Each format has its own specific settings.

`text`

The text format is human-readable and ideal for development and debugging. It is the default logging output format.

To use the text format, in router.yaml enable telemetry.exporters.logging.stdout and set the format as text:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: text # The default text format will be used

The text format can also be used as a key in YAML, telemetry.exporters.logging.stdout.format.text, to specify advanced configuration options:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format:
7           text:
8              ansi_escape_codes: true
9              display_filename: true
10              display_level: true
11              display_line_number: true
12              display_target: true
13              display_thread_id: true
14              display_thread_name: true
15              display_timestamp: true
16              display_resource: true
17              display_span_list: true
18              display_current_span: true
19              display_service_name: true
20              display_service_namespace: true
21              display_trace_id: true # true|false|open_telemetry|hexadecimal|decimal|datadog|uuid
22              display_span_id: true

Example text output:

JSON

12023-10-30T15:49:34.174435Z  INFO trace_id: bbafc3f048b6137375dd78c10df18f50 span_id: 40ede28c5df1b5cc main ThreadId(01) span_name{span_attr_1="span_attr_1" span_attr_2="span_attr_2"}: event_target: event_file.rs:32: event_attr_1="event_attr_1" event_attr_2="event_attr_2"

`text` configuration reference

Option	Values	Default	Description
`ansi_escape_codes`	`true`\|`false`	`true`	Use ansi terminal escape codes.
`display_filename`	`true`\|`false`	`false`	The filename where the log event was raised.
`display_level`	`true`\|`false`	`true`	The level of the log event, e.g. INFO, WARN, ERROR, TRACE.
`display_line_number`	`true`\|`false`	`false`	The line number where the event was raised.
`display_target`	`true`\|`false`	`false`	The module name where the event was raised.
`display_thread_id`	`true`\|`false`	`false`	The id of the thread where the event was raised.
`display_thread_name`	`true`\|`false`	`false`	The name of the thread where the event was raised.
`display_timestamp`	`true`\|`false`	`true`	The timestamp of when the event was raised.
`display_service_name`	`true`\|`false`	`false`	The service name as configured in metrics common.
`display_service_namespace`	`true`\|`false`	`false`	The service namespace as configured in metrics common.
`display_trace_id`	`true`\|`false`\|`open_telemetry`\|`hexadecimal`\|`decimal`\|`datadog`\|`uuid`	`false`	The trace id of the span in which the event was raised.
`display_span_id`	`true`\|`false`	`false`	The span ID of the span in which the event was raised.
`display_span_list`	`true`\|`false`	`true`	A list of all spans to root in which the event was raised and all of their attributes.
`display_current_span`	`true`\|`false`	`true`	The span in which the event was raised and all of its' attributes.

`json`

The json format is a machine-readable format ideal for consumption by application performance monitors (APMs).

The router supports structured JSON output provided by tracing-subscriber.

To use the json format, in router.yaml enable telemetry.exporters.logging.stdout and set the format as json:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: json

Each log entry will be a single well-formed JSON document that is ideal for processing in your APM tool of choice.

Example default json output:

JSON

stdout

1{
2  "timestamp": "2023-10-30T14:09:34.771388Z",
3  "level": "INFO",
4  "trace_id": "54ac7e5f0e8ab90ae67b822e95ffcbb8",
5  "span_id": "d52e3478c718b8a9",
6  "fields": {
7    "event_attr_1": "event_attr_1",
8    "event_attr_2": "event_attr_2"
9  },
10  "target": "event_target"
11}

You can configure which attributes are included in the JSON output by specifying telemetry.exporters.logging.stdout.format.json as a key in router.yaml:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: 
7           json:
8             display_filename: false
9             display_level: true
10             display_line_number: false
11             display_target: false
12             display_thread_id: false
13             display_thread_name: false
14             display_timestamp: true
15             display_current_span: true
16             display_span_list: true
17             display_resource: true
18             display_trace_id: true # true|false|open_telemetry|hexadecimal|decimal|datadog|uuid
19             display_span_id: true

Example json output:

JSON

1{
2  "timestamp": "2023-10-30T15:47:52.570482Z",
3  "level": "INFO",
4  "trace_id": "54ac7e5f0e8ab90ae67b822e95ffcbb8",
5  "span_id": "d52e3478c718b8a9",
6  "fields": {
7    "event_attr_1": "event_attr_1",
8    "event_attr_2": "event_attr_2"
9  },
10  "target": "event_target",
11  "filename": "event_file.rs",
12  "line_number": 32,
13  "span": {
14    "span_attr_1": "span_attr_1",
15    "span_attr_2": "span_attr_2",
16    "name": "span_name"
17  },
18  "spans": [
19    {
20      "span_attr_1": "span_attr_1",
21      "span_attr_2": "span_attr_2",
22      "name": "span_name"
23    }
24  ],
25  "threadName": "main",
26  "threadId": "ThreadId(1)"
27}

`display_current_span`

Events may also output information about the span that they are raised in, which is useful to log attributes attached to the span for a particular request.

To log span information, set the telemetry.exporters.logging.stdout.format.json.display_current_span option to true:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: 
7           json:
8             display_current_span: true

Example output with span information:

JSON

1{
2  "timestamp": "2023-10-30T14:09:34.771388Z",
3  "level": "INFO",
4  "fields": {
5    "event_attr_1": "event_attr_1",
6    "event_attr_2": "event_attr_2"
7  },
8  "target": "event_target",
9  "span": {
10    "span_attr_1": "span_attr_1",
11    "span_attr_2": "span_attr_2",
12    "name": "span_name"
13  }
14}

`display_span_list`

The telemetry.exporters.logging.stdout.format.json.display_span_list option is like display_current_span but instead of outputting information for the current span, display_span_list outputs information for all spans that an event was raised in.

For example, if you have a custom trace_id from a request header, as long as the attribute is configured on the router span it will appear on all log events associated with the request.

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: 
7           json:
8             display_span_list: true

Example output with a list of spans:

JSON

1{
2  "timestamp": "2023-10-30T14:09:34.771388Z",
3  "level": "INFO",
4  "fields": {
5    "event_attr_1": "event_attr_1",
6    "event_attr_2": "event_attr_2"
7  },
8  "target": "event_target",
9  "spans": [
10    {
11      "span_attr_1": "span_attr_1",
12      "span_attr_2": "span_attr_2",
13      "name": "span_name"
14    }
15  ]
16}

`span_attributes`

The telemetry.exporters.logging.stdout.format.json.span_attributes option allows you to display a subset of all span attributes. It takes as input an array of span attribute names to log.

When span_attributes is specified, the router searches for the first attribute in the list of span attributes from the root span to the current span and attaches it to the outermost JSON object for the log event.

If you set the same attribute name for different spans at different levels, the router chooses the attributes of child spans before the attributes of parent spans.

For example, you can display just the span_attr_1 span attribute:

YAML

router.yaml

1telemetry:
2  exporters:
3     logging:
4       stdout:
5         enabled: true
6         format: 
7           json:
8             display_span_list: false
9             span_attributes:
10             - span_attr_1

Example output with a list of spans:

JSON

1{
2  "timestamp": "2023-10-30T14:09:34.771388Z",
3  "level": "INFO",
4  "fields": {
5    "event_attr_1": "event_attr_1",
6    "event_attr_2": "event_attr_2"
7  },
8  "target": "event_target",
9  "span_attr_1": "span_attr_1"
10}

`display_resource`

The telemetry.logging.stdout.format.json.display_resource option configures whether resources configured in router.yaml are displayed in log messages. By default, display_resource is true.

For example, given a router.yaml with display_resource: true and a configured resource, log messages will display the resource:

YAML

router.yaml

1telemetry:
2  exporters:
3    logging:
4      stdout:
5        format: 
6          json:
7            display_resource: true
8      common:
9        service_name: bryn-router
10        resource:
11          test.resource: test

Text

"resource":{"test.resource":"test","service.name":"bryn-router"}

`json` configuration reference

Option	Values	Default	Event Field	Description
`display_current_span`	`true`\|`false`	`false`	`span`	The span in which the event was raised and all of its' attributes.
`display_filename`	`true`\|`false`	`false`	`filename`	The filename where the log event was raised.
`display_level`	`true`\|`false`	`true`	`level`	The level of the log event, e.g. INFO, WARN, ERROR, TRACE.
`display_line_number`	`true`\|`false`	`false`	`line_number`	The line number where the event was raised.
`display_target`	`true`\|`false`	`true`	`target`	The module name where the event was raised.
`display_thread_id`	`true`\|`false`	`false`	`thread_id`	The id of the thread where the event was raised.
`display_thread_name`	`true`\|`false`	`false`	`thread_name`	The name of the thread where the event was raised.
`display_timestamp`	`true`\|`false`	`true`	`timestamp`	The timestamp of when the event was raised.
`display_span_list`	`true`\|`false`	`true`	`spans`	A list of all spans to root in which the event was raised and all of their attributes.
`display_resource`	`true`\|`false`	`true`	`resource`	The resource as configured in tracing common.
`display_trace_id`	`true`\|`false`\|`open_telemetry`\|`hexadecimal`\|`decimal`\|`datadog`\|`uuid`	The trace id of the span in which the event was raised.
`display_span_id`	`true`\|`false`	`true`	`span_id`	The span id of the span in which the event was raised.
`span_attributes`	`[string]`	`[]`	`*`	List of span attributes to attach to the JSON log object.

Log Exporters

Metrics Exporters

Trace Exporters

Instrumentation

AWS Lattice