External coprocessing in the Apollo Router

With external coprocessing, you can hook into the Apollo Router's request-handling lifecycle by writing standalone code in any language and framework. This code (i.e., your coprocessor) can run anywhere on your network that's accessible to the router over HTTP.

You can configure your router to "call out" to your coprocessor at different stages throughout the request-handling lifecycle, enabling you to perform custom logic based on a client request's headers, query string, and other details. This logic can access disk and perform network requests, all while safely isolated from the critical router process.

When your coprocessor responds to these requests, its response body can modify various details of the client's request or response. You can even terminate a client request.

Recommended locations for hosting your coprocessor include:

• On the same host as your router (minimal request latency)
• In the same Pod as your router, as a "sidecar" container (minimal request latency)
• In the same availability zone as your router (low request latency with increased deployment isolation)

How it works

Whenever your router receives a client request, at various stages in the request-handling lifecycle it can send HTTP POST requests to your coprocessor:

As shown in the diagram above, only the RouterService and SubgraphService of the request-handling lifecycle can send these POST requests (also called coprocessor requests).

Each supported service can send its coprocessor requests at two different stages:

• As excecution proceeds "down" from the client to individual subgraphs
  • Here, the coprocessor can inspect and modify details of requests before GraphQL operations are processed.
  • The coprocessor can also instruct the router to terminate a client request immediately.
• As execution proceeds back "up" from subgraphs to the client
  • Here, the coprocessor can inspect and modify details of the router's response to the client.

At every stage, the router waits for your coprocessor's response before it continues processing the corresponding request. Because of this, you should maximize responsiveness by configuring only whichever coprocessor requests your customization requires.

Multiple requests with SubgraphService

If your coprocessor hooks into your router's SubgraphService, the router sends a separate coprocessor request for each subgraph request in its query plan. In other words, if your router needs to query three separate subgraphs to fully resolve a client operation, it sends three separate coprocessor requests. Each coprocessor request includes the name and URL of the subgraph being queried.

Setup

You configure external coprocessing in your router's YAML config file, under the coprocessor key.

Typical configuration

This example configuration sends commonly used request and response details to your coprocessor (see the comments below for explanations of each field):

1
coprocessor:
2
  url: http://127.0.0.1:8081 # Required. Replace with the URL of your coprocessor's HTTP endpoint.
3
  timeout: 2s # The timeout for all coprocessor requests. Defaults to 1 second (1s)
4
  router: # This coprocessor hooks into the `RouterService`
5
    request: # By including this key, the `RouterService` sends a coprocessor request whenever it first receives a client request.
6
      headers: true # These boolean properties indicate which request data to include in the coprocessor request. All are optional and false by default.
7
      body: false
8
      context: false
9
      sdl: false
10
      path: false
11
      method: false
12
    response: # By including this key, the `RouterService` sends a coprocessor request whenever it's about to send response data to a client (including incremental data via @defer).
13
      headers: true
14
      body: false
15
      context: false
16
      sdl: false
17
      status_code: false
18
  subgraph:
19
    all:
20
      request: # By including this key, the `SubgraphService` sends a coprocessor request whenever it is about to make a request to a subgraph.
21
        headers: true # These boolean properties indicate which request data to include in the coprocessor request. All are optional and false by default.
22
        body: false
23
        context: false
24
        uri: false
25
        method: false
26
        service_name: false
27
      response: # By including this key, the `SubgraphService` sends a coprocessor request whenever receives a subgraph response.
28
        headers: true
29
        body: false
30
        context: false
31
        service_name: false
32
        status_code: false

Minimal configuration

You can confirm that your router can reach your coprocessor by setting this minimal configuration before expanding it as needed:

1
coprocessor:
2
  url: http://127.0.0.1:8081 # Replace with the URL of your coprocessor's HTTP endpoint.
3
  router:
4
    request:
5
      headers: false

In this case, the RouterService only sends a coprocessor request whenever it receives a client request. The coprocessor request body includes no data related to the client request (only "control" data, which is covered below).

Coprocessor request format

The router communicates with your coprocessor via HTTP POST requests (called coprocessor requests). The body of each coprocessor request is a JSON object with properties that describe either the current client request or the current router response.

Properties of the JSON body are divided into two high-level categories:

• "Control" properties
  • These provide information about the context of the specific router request or response. They provide a mechanism to influence the router's execution flow.
  • The router always includes these properties in coprocessor requests.
• Data properties
  • These provide information about the substance of a request or response, such as the GraphQL query string and any HTTP headers. Aside from sdl, your coprocessor can modify all of these properties.
  • You configure which of these fields the router includes in its coprocessor requests. By default, the router includes none of them.

Example requests by stage

RouterRequest

1
{
2
  // Control properties
3
  "version": 1,
4
  "stage": "RouterRequest",
5
  "control": "continue",
6
  "id": "1b19c05fdafc521016df33148ad63c1b",
7

8
  // Data properties
9
  "headers": {
10
    "cookie": [
11
      "tasty_cookie=strawberry"
12
    ],
13
    "content-type": [
14
      "application/json"
15
    ],
16
    "host": [
17
      "127.0.0.1:4000"
18
    ],
19
    "apollo-federation-include-trace": [
20
      "ftv1"
21
    ],
22
    "apollographql-client-name": [
23
      "manual"
24
    ],
25
    "accept": [
26
      "*/*"
27
    ],
28
    "user-agent": [
29
      "curl/7.79.1"
30
    ],
31
    "content-length": [
32
      "46"
33
    ]
34
  },
35
  "body": "{
36
    \"query\": \"query GetActiveUser {\n  me {\n  name\n}\n}\"
37
  }",
38
  "context": {
39
    "entries": {
40
      "accepts-json": false,
41
      "accepts-wildcard": true,
42
      "accepts-multipart": false
43
    }
44
  },
45
  "sdl": "..." // String omitted due to length
46
  "path": "/",
47
  "method": "POST"
48
}

RouterResponse

1
{
2
  // Control properties
3
  "version": 1,
4
  "stage": "RouterResponse",
5
  "control": "continue",
6
  "id": "1b19c05fdafc521016df33148ad63c1b",
7

8
  // Data properties
9
  "headers": {
10
    "vary": [
11
      "origin"
12
    ],
13
    "content-type": [
14
      "application/json"
15
    ]
16
  },
17
  "body": "{
18
    \"data\": {
19
      \"me\": {
20
        \"name\": \"Ada Lovelace\"
21
      }
22
    }
23
  }",
24
  "context": {
25
    "entries": {
26
      "apollo_telemetry::subgraph_metrics_attributes": {},
27
      "accepts-json": false,
28
      "accepts-multipart": false,
29
      "apollo_telemetry::client_name": "manual",
30
      "apollo_telemetry::usage_reporting": {
31
        "statsReportKey": "# Long\nquery Long{me{name}}",
32
        "referencedFieldsByType": {
33
          "User": {
34
            "fieldNames": [
35
              "name"
36
            ],
37
            "isInterface": false
38
          },
39
          "Query": {
40
            "fieldNames": [
41
              "me"
42
            ],
43
            "isInterface": false
44
          }
45
        }
46
      },
47
      "apollo_telemetry::client_version": "",
48
      "accepts-wildcard": true
49
    }
50
  },
51
  "statusCode": 200,
52
  "sdl": "..." // Omitted due to length
53
}

SubgraphRequest

1
{
2
  // Control properties
3
  "version": 1,
4
  "stage": "SubgraphRequest",
5
  "control": "continue",
6
  "id": "666d677225c1bc6d7c54a52b409dbd4e",
7

8
  // Data properties
9
  "headers": {},
10
  "body": {
11
    "query": "query TopProducts__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on Product{reviews{body id}}}}",
12
    "operationName": "TopProducts__reviews__1",
13
    "variables": {
14
      "representations": [
15
        {
16
          "__typename": "Product",
17
          "upc": "1"
18
        },
19
        {
20
          "__typename": "Product",
21
          "upc": "2"
22
        },
23
        {
24
          "__typename": "Product",
25
          "upc": "3"
26
        }
27
      ]
28
    }
29
  },
30
  "context": {
31
    "entries": {
32
      "content-negociation:accepts-wildcard": true,
33
      "apollo_telemetry::usage_reporting": {
34
        "statsReportKey": "# TopProducts\nquery TopProducts{topProducts{name price reviews{body id}}}",
35
        "referencedFieldsByType": {
36
          "Query": {
37
            "fieldNames": [
38
              "topProducts"
39
            ],
40
            "isInterface": false
41
          },
42
          "Review": {
43
            "fieldNames": [
44
              "body",
45
              "id"
46
            ],
47
            "isInterface": false
48
          },
49
          "Product": {
50
            "fieldNames": [
51
              "price",
52
              "name",
53
              "reviews"
54
            ],
55
            "isInterface": false
56
          }
57
        }
58
      },
59
      "apollo_telemetry::client_version": "",
60
      "apollo_telemetry::subgraph_metrics_attributes": {},
61
      "content-negociation:accepts-json": false,
62
      "apollo_telemetry::client_name": "",
63
      "content-negociation:accepts-multipart": false
64
    }
65
  },
66
  "uri": "https://reviews.demo.starstuff.dev/",
67
  "method": "POST",
68
  "serviceName": "reviews"
69
}

SubgraphResponse

1
{
2
  // Control properties
3
  "version": 1,
4
  "stage": "SubgraphResponse",
5
  "id": "b7810c6f7f95640fd6c6c8781e3953c0",
6
  "control": "continue",
7

8
  // Data properties
9
  "headers": {
10
    "etag": [
11
      "W/\"d3-7aayASjs0+e2c/TpiAYgEu/yyo0\""
12
    ],
13
    "via": [
14
      "2 fly.io"
15
    ],
16
    "server": [
17
      "Fly/90d459b3 (2023-03-07)"
18
    ],
19
    "date": [
20
      "Thu, 09 Mar 2023 14:28:46 GMT"
21
    ],
22
    "x-powered-by": [
23
      "Express"
24
    ],
25
    "x-ratelimit-limit": [
26
      "10000000"
27
    ],
28
    "access-control-allow-origin": [
29
      "*"
30
    ],
31
    "x-ratelimit-remaining": [
32
      "9999478"
33
    ],
34
    "content-type": [
35
      "application/json; charset=utf-8"
36
    ],
37
    "fly-request-id": [
38
      "01GV3CCG5EM3ZNVZD2GH0B00E2-lhr"
39
    ],
40
    "x-ratelimit-reset": [
41
      "1678374007"
42
    ]
43
  },
44
  "body": {
45
    "data": {
46
      "_entities": [
47
        {
48
          "reviews": [
49
            {
50
              "body": "Love it!",
51
              "id": "1"
52
            },
53
            {
54
              "body": "Prefer something else.",
55
              "id": "4"
56
            }
57
          ]
58
        },
59
        {
60
          "reviews": [
61
            {
62
              "body": "Too expensive.",
63
              "id": "2"
64
            }
65
          ]
66
        },
67
        {
68
          "reviews": [
69
            {
70
              "body": "Could be better.",
71
              "id": "3"
72
            }
73
          ]
74
        }
75
      ]
76
    }
77
  },
78
  "context": {
79
    "entries": {
80
      "content-negociation:accepts-wildcard": true,
81
      "content-negociation:accepts-json": false,
82
      "content-negociation:accepts-multipart": false,
83
      "apollo_telemetry::usage_reporting": {
84
        "statsReportKey": "# TopProducts\nquery TopProducts{topProducts{name price reviews{body id}}}",
85
        "referencedFieldsByType": {
86
          "Product": {
87
            "fieldNames": [
88
              "price",
89
              "name",
90
              "reviews"
91
            ],
92
            "isInterface": false
93
          },
94
          "Query": {
95
            "fieldNames": [
96
              "topProducts"
97
            ],
98
            "isInterface": false
99
          },
100
          "Review": {
101
            "fieldNames": [
102
              "body",
103
              "id"
104
            ],
105
            "isInterface": false
106
          }
107
        }
108
      },
109
      "apollo_telemetry::client_version": "",
110
      "apollo_telemetry::subgraph_metrics_attributes": {},
111
      "apollo_telemetry::client_name": ""
112
    }
113
  },
114
  "serviceName": "reviews",
115
  "statusCode": 200
116
}

Property reference

1
{ "break": 400 }

Responding to coprocessor requests

The router expects your coprocessor to respond with a 200 status code and a JSON body that matches the structure of the request body.

In the response body, your coprocessor can return modified values for certain properties. By doing so, you can modify the remainder of the router's execution for the client request.

The router supports modifying the following properties from your coprocessor:

• control
  • Modify this property to immediately terminate a client request.
• body
• headers
• context

Terminating a client request

Every coprocessor request body includes a control property with the string value continue. If your coprocessor's response body also sets control to continue, the router continues processing the client request as usual.

Alternatively, your coprocessor's response body can set control to an object with a break property, like so:

1
{
2
  "control": { "break": 401 }, 
3
  "body": {
4
    "errors": [
5
      {
6
        "message": "Not authenticated.",
7
        "extensions": {
8
          "code": "ERR_UNAUTHENTICATED"
9
        }
10
      }
11
    ]
12
  }
13
}

If the router receives an object with this format for control, it immediately terminates the request-handling lifecycle for the client request. It sends an HTTP response to the client with the following details:

• The HTTP status code is set to the value of the break property (401 in the example above).
• The response body is the coprocessor's returned value for body.
  • The value of body should adhere to the standard GraphQL JSON response format (see the example above).
  • Alternatively, you can specify a string value for body. If you do, the router returns an error response with that string as the error's message.

The example response above sets the HTTP status code to 400, which indicates a failed request.

You can also use this mechanism to immediately return a successful response:

1
{
2
  "control": { "break": 200 },
3
  "body": {
4
    "data": {
5
      "currentUser": {
6
        "name": "Ada Lovelace"
7
      }
8
    }
9
  }
10
}

Failed responses

Your router considers all of the following scenarios to be a failed response from your coprocessor:

• Your coprocessor doesn't respond within the amount of time specified by the timeout key in your configuration (default one second).
• Your coprocessor responds with a non-2xx HTTP code.
• Your coprocessor's response body doesn't match the JSON structure of the corresponding request body.
• Your coprocessor's response body sets different values for control properties that must not change, such as stage and version.

Handling deferred query responses

The Apollo Router supports the incremental delivery of query response data via the @defer directive:

For a single query with deferred fields, your router sends multiple "chunks" of response data to the client. If you enable coprocessor requests for the RouterResponse stage, your router sends a separate coprocessor request for each chunk it returns as part of a deferred query.

Note the following about handling deferred response chunks:

• The status_code and headers fields are included only in the coprocessor request for any response's first chunk. These values can't change after the first chunk is returned to the client, so they're subsequently omitted.

• If your coprocessor modifes the response body for a response chunk, it must provide the new value as a string, not as an object. This is because response chunk bodies include multipart boundary information in addition to the actual serialized JSON response data. See examples.

  • Many responses will not contain deferred streams and for these the body string can usually be fairly reliably transformed into a JSON object for easy manipulation within the coprocessor. Coprocessors should be carefully coded to allow for the presence of a body that is not a valid JSON object.

• Because the data is a JSON string at both RouterRequest and RouterResponse, it's entirely possible for a coprocessor to rewrite the body from invalid JSON content into valid JSON content. This is one of the primary use cases for RouterRequest body processing.

Examples of deferred response chunks

The examples below illustrate the differences between the first chunk of a deferred response and all subsequent chunks:

First response chunk

The first response chunk includes headers and statusCode fields:

1
{
2
  "version": 1,
3
  "stage": "RouterResponse",
4
  "id": "8dee7fe947273640a5c2c7e1da90208c",
5
  "sdl": "...", // String omitted due to length
6
  "headers": {
7
    "content-type": [
8
      "multipart/mixed;boundary=\"graphql\";deferSpec=20220824"
9
    ],
10
    "vary": [
11
      "origin"
12
    ]
13
  },
14
  "body": "\r\n--graphql\r\ncontent-type: application/json\r\n\r\n{\"data\":{\"me\":{\"id\":\"1\"}},\"hasNext\":true}\r\n--graphql\r\n",
15
  "context": {
16
    "entries": {
17
      "apollo_telemetry::operation_kind": "query",
18
      "apollo_telemetry::client_version": "",
19
      "apollo_telemetry::client_name": "manual"
20
    }
21
  },
22
  "statusCode": 200 
23
}

Subsequent response chunk

Subsequent response chunks omit the headers and statusCode fields:

1
{
2
  "version": 1,
3
  "stage": "RouterResponse",
4
  "id": "8dee7fe947273640a5c2c7e1da90208c",
5
  "sdl": "...", // String omitted due to length
6
  "body": "content-type: application/json\r\n\r\n{\"hasNext\":false,\"incremental\":[{\"data\":{\"name\":\"Ada Lovelace\"},\"path\":[\"me\"]}]}\r\n--graphql--\r\n",
7
  "context": {
8
    "entries": {
9
      "apollo_telemetry::operation_kind": "query",
10
      "apollo_telemetry::client_version": "",
11
      "apollo_telemetry::client_name": "manual"
12
    }
13
  }
14
}