Query Batching
Receive query batches with the GraphOS Router
Learn about query batching and how to configure the GraphOS Router to receive query batches.
About query batching
Modern applications often require several requests to render a single page. This is usually the result of a component-based architecture where individual micro-frontends (MFE) make requests separately to fetch data relevant to them. Not only does this cause a performance overhead—different components may be requesting the same data—it can also cause a consistency issue. To combat this, MFE-based UIs batch multiple client operations, issued close together, into a single HTTP request. This is supported in Apollo Client and Apollo Server.
The router's batching support is provided by two sets of functionality:
client batching
subgraph batching
With client batching, the router accepts batched requests from a client and processes each request of a batch separately. Consequently, the router doesn't present requests to subgraphs in batch form, so subgraphs must process the requests of a batch individually.
With subgraph batching, the router analyzes input client batch requests and issues batch requests to subgraphs. Subgraph batching is an extension to client batching and requires participating subgraphs to support batching requests. See the examples below to see illustrations of how this works in practice.
The GraphOS Router supports client and subgraph query batching.
If you’re using Apollo Client, you can leverage the built-in support for batching to reduce the number of individual operations sent to the router.
Once configured, Apollo Client automatically combines multiple operations into a single HTTP request. The number of operations within a batch is client-configurable, including the maximum number in a batch and the maximum duration to wait for operations to accumulate before sending the batch.
The GraphOS Router must be configured to receive query batches, otherwise it rejects them. When processing a batch, the router deserializes and processes each operation of a batch independently. It responds to the client only after all operations of the batch have been completed. Each operation executes concurrently with respect to other operations in the batch.
Configure client query batching
Both the GraphOS Router and client need to be configured to support query batching.
Configure router
Client query batching
By default, receiving client query batches is not enabled in the GraphOS Router.
To enable query batching, set the following fields in your router.yaml
configuration file:
1batching:
2 enabled: true
3 mode: batch_http_link
Attribute | Description | Valid Values | Default Value |
---|---|---|---|
enabled | Flag to enable reception of client query batches | boolean | false |
mode | Supported client batching mode | batch_http_link : the client uses Apollo Link and its BatchHttpLink link. | No Default |
Subgraph query batching
If client query batching is enabled, and the router's subgraphs support query batching , then subgraph query batching can be enabled by setting the following fields in your router.yaml
configuration file:
1batching:
2 enabled: true
3 mode: batch_http_link
4 subgraph:
5 # Enable batching on all subgraphs
6 all:
7 enabled: true
1batching:
2 enabled: true
3 mode: batch_http_link
4 subgraph:
5 # Disable batching on all subgraphs
6 all:
7 enabled: false
8 # Configure(over-ride) batching support per subgraph
9 subgraphs:
10 subgraph_1:
11 enabled: true
12 subgraph_2:
13 enabled: true
- The router can be configured to support batching for either all subgraphs or individually enabled per subgraph.
- There are limitations on the ability of the router to preserve batches from the client request into the subgraph requests. In particular, certain forms of queries will require data to be present before they are processed. Consequently, the router will only be able to generate batches from queries which are processed which don't contain such constraints. This may result in the router issuing multiple batches or requests.
- If query deduplication or entity caching are enabled, they will not apply to batched queries. Batching will take precedence over query deduplication and entity caching. Query deduplication and Entity caching will still be performed for non-batched queries.
Example: simple subgraph batching
This example shows how the router can batch subgraph requests in the most efficient scenario, where the queries of a batch don't have required fetch constraints.
Assume the federated graph contains three subgraphs: accounts
, products
, and reviews
.
The input client query to the federated graph:
1[
2 {"query":"query MeQuery1 {\n me {\n id\n }\n}"}
3 {"query":"query MeQuery2 {\n me {\n name\n }\n}"},
4 {"query":"query MeQuery3 {\n me {\n id\n }\n}"}
5 {"query":"query MeQuery4 {\n me {\n name\n }\n}"},
6 {"query":"query MeQuery5 {\n me {\n id\n }\n}"}
7 {"query":"query MeQuery6 {\n me {\n name\n }\n}"},
8 {"query":"query MeQuery7 {\n me {\n id\n }\n}"}
9 {"query":"query MeQuery8 {\n me {\n name\n }\n}"},
10 {"query":"query MeQuery9 {\n me {\n id\n }\n}"}
11 {"query":"query MeQuery10 {\n me {\n name\n }\n}"},
12 {"query":"query MeQuery11 {\n me {\n id\n }\n}"}
13 {"query":"query MeQuery12 {\n me {\n name\n }\n}"},
14 {"query":"query MeQuery13 {\n me {\n id\n }\n}"}
15 {"query":"query MeQuery14 {\n me {\n name\n }\n}"},
16 {"query":"query MeQuery15 {\n me {\n id\n }\n}"}
17]
From the input query, the router generates a set of subgraph queries:
1"query MeQuery1__accounts__0{me{id}}",
2"query MeQuery2__accounts__0{me{name}}",
3"query MeQuery3__accounts__0{me{id}}",
4"query MeQuery4__accounts__0{me{name}}",
5"query MeQuery5__accounts__0{me{id}}",
6"query MeQuery6__accounts__0{me{name}}",
7"query MeQuery7__accounts__0{me{id}}",
8"query MeQuery8__accounts__0{me{name}}",
9"query MeQuery9__accounts__0{me{id}}",
10"query MeQuery10__accounts__0{me{name}}",
11"query MeQuery11__accounts__0{me{id}}",
12"query MeQuery12__accounts__0{me{name}}",
13"query MeQuery13__accounts__0{me{id}}",
14"query MeQuery14__accounts__0{me{name}}",
15"query MeQuery15__accounts__0{me{id}}",
All of the queries can be combined into a single batch. So instead of 15 (non-batch) subgraph fetches, the router only has to make one fetch.
Subgraph | Fetch Count (without) | Fetch Count (with) |
---|---|---|
accounts | 15 | 1 |
Example: complex subgraph batching
This example shows how the router might batch subgraph requests for a graph, where the client batch contains a query for an entity.
Assume the federated graph contains three subgraphs: accounts
, products
, and reviews
.
The input client query to the federated graph:
1[
2 {"query":"query MeQuery1 {\n me {\n id\n }\n}"},
3 {"query":"query MeQuery2 {\n me {\n reviews {\n body\n }\n }\n}"},
4 {"query":"query MeQuery3 {\n topProducts {\n upc\n reviews {\n author {\n name\n }\n }\n }\n me {\n name\n }\n}"},
5 {"query":"query MeQuery4 {\n me {\n name\n }\n}"},
6 {"query":"query MeQuery5 {\n me {\n id\n }\n}"}
7]
From the input query, the router generates a set of subgraph queries:
1"query MeQuery1__accounts__0{me{id}}",
2"query MeQuery2__accounts__0{me{__typename id}}",
3"query MeQuery3__products__0{topProducts{__typename upc}}",
4"query MeQuery3__accounts__3{me{name}}",
5"query MeQuery4__accounts__0{me{name}}",
6"query MeQuery5__accounts__0{me{id}}",
7"query MeQuery2__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on User{reviews{body}}}}",
8"query MeQuery3__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on Product{reviews{author{__typename id}}}}}",
9"query MeQuery3__accounts__2($representations:[_Any!]!){_entities(representations:$representations){...on User{name}}}",
The first six queries can be combined into two batches—one for accounts
and one for products
. They must be fetched before the final three queries can be executed individually.
Overall, without subgraph batching, the router would make nine fetches in total across the three subgraphs, but with subgraph batching, that total is reduced to five fetches.
Subgraph | Fetch Count (without) | Fetch Count (with) |
---|---|---|
accounts | 6 | 2 |
products | 1 | 1 |
reviews | 2 | 2 |
Configure client
To enable batching in an Apollo client, configure BatchHttpLink
. For details on implementing BatchHttpLink
, see batching operations .
Configuration compatibility
If the router receives a query batch from a client, and batching is not enabled, the router sends a BATCHING_NOT_ENABLED
error to the client.
Metrics for query batching
Metrics in the GraphOS Router for query batching:
Name | Attributes | Description |
---|---|---|
| mode [subgraph] | Counter for the number of received (from client) or dispatched (to subgraph) batches. |
| mode [subgraph] | Histogram for the size of received batches. |
The subgraph
attribute is optional. If the attribute isn't present, the metric identifies batches received from clients. If the attribute is present, the metric identifies batches sent to a particular subgraph.
Query batch formats
Request format
A query batch is an array of operations.
1[
2query MyFirstQuery {
3 me {
4 id
5 }
6},
7query MySecondQuery {
8 me {
9 name
10 }
11}
12]
Response format
Responses are provided in JSON array, with the order of responses matching the order of operations in the query batch.
1[
2 {"data":{"me":{"id":"1"}}},
3 {"data":{"me":{"name":"Ada Lovelace"}}}
4]
Error handling for query batching
Batch error
If a batch of queries cannot be processed, the entire batch fails.
For example, this batch request is invalid because it has two commas to separate the constituent queries:
1[
2query MyFirstQuery {
3 me {
4 id
5 }
6},,
7query MySecondQuery {
8 me {
9 name
10 }
11}
12]
As a result, the router returns an invalid batch error:
1{"errors":
2 [
3 {"message":"Invalid GraphQL request","extensions":{"details":"failed to deserialize the request body into JSON: expected value at line 1 column 54","code":"INVALID_GRAPHQL_REQUEST"}}
4 ]
5}
Individual query error
If a single query in a batch cannot be processed, this results in an individual error.
For example, the query MyFirstQuery
is accessing a field that doesn't exist, while the rest of the batch query is valid.
1[
2query MyFirstQuery {
3 me {
4 thisfielddoesnotexist
5 }
6},
7query MySecondQuery {
8 me {
9 name
10 }
11}
12]
As a result, an error is returned for the individual invalid query and the other (valid) query returns a response.
1[
2 {"errors":
3 [
4 {"message":"cannot query field 'thisfielddoesnotexist' on type 'User'",
5 "extensions":{"type":"User","field":"thisfielddoesnotexist","code":"INVALID_FIELD"}
6 }
7 ]
8 },
9 {"data":{"me":{"name":"Ada Lovelace"}}}
10]
Known limitations
Unsupported query modes
When batching is enabled, any batch operation that results in a stream of responses is unsupported, including: