Docs
Launch GraphOS Studio

Traffic shaping in the Apollo Router


The provides various features to improve the performance and reliability of the traffic between the client and router and between the router and .

Configuration

To enable traffic shaping, add the traffic_shaping plugin to your

, like so:

router.yaml
traffic_shaping:
router: # Rules applied to requests from clients to the router
global_rate_limit: # Accept a maximum of 10 requests per 5 secs. Excess requests must be rejected.
capacity: 10
interval: 5s # Must not be greater than 18_446_744_073_709_551_615 milliseconds and not less than 0 milliseconds
timeout: 50s # If a request to the router takes more than 50secs then cancel the request (30 sec by default)
all:
deduplicate_query: true # Enable query deduplication for all subgraphs.
compression: br # Enable brotli compression for all subgraphs.
subgraphs: # Rules applied to requests from the router to individual subgraphs
products:
deduplicate_query: false # Disable query deduplication for the products subgraph.
compression: gzip # Enable gzip compression only for the products subgraph.
global_rate_limit: # Accept a maximum of 10 requests per 5 secs from the router. Excess requests must be rejected.
capacity: 10
interval: 5s # Must not be greater than 18_446_744_073_709_551_615 milliseconds and not less than 0 milliseconds
timeout: 50s # If a request to the subgraph 'products' takes more than 50secs then cancel the request (30 sec by default)
experimental_retry:
min_per_sec: 10 # minimal number of retries per second (`min_per_sec`, default is 10 retries per second)
ttl: 10s # for each successful request, we register a token, that expires according to this option (default: 10s)
retry_percent: 0.2 # defines the proportion of available retries to the current number of tokens
retry_mutations: false # allows retries on mutations. This should only be enabled if mutations are idempotent
experimental_http2: enable # Configures HTTP/2 usage. Can be 'enable' (default), 'disable' or 'http2only'

Client side traffic shaping

Rate limiting

The can apply rate limiting on client requests, as follows:

router.yaml
traffic_shaping:
router: # Rules applied to requests from clients to the router
global_rate_limit: # Accept a maximum of 10 requests per 5 secs. Excess requests must be rejected.
capacity: 10
interval: 5s # Must not be greater than 18_446_744_073_709_551_615 milliseconds and not less than 0 milliseconds

This rate limiting applies to all requests, there is no filtering per IP or other criteria.

Timeouts

The applies a default timeout of 30 seconds for all requests, including the following:

You can change the default timeout for client requests to the like so:

router.yaml
traffic_shaping:
router:
timeout: 50s # If client requests to the router take more than 50 seconds, cancel the request (30 seconds by default)

You can change the default timeout for all requests between the and like so:

router.yaml
traffic_shaping:
all:
timeout: 50s # If subgraph requests take more than 50 seconds, cancel the request (30 seconds by default)

NOTE

Since

are separate requests, each fragment's request is individually subject to timeouts.

Compression

Compression is automatically supported on the client side, depending on the Accept-Encoding header provided by the client.

Query batching

The has experimental support for receiving client batches:

router.yaml
experimental_batching:
enabled: true
mode: batch_http_link

For details, see

.

Subgraph traffic shaping

The supports various options affecting traffic destined for , that can either be defined for all subgraphs, or overriden per subgraph:

router.yaml
traffic_shaping:
all:
deduplicate_query: true # Enable query deduplication for all subgraphs.
subgraphs: # Rules applied to requests from the router to individual subgraphs
products:
deduplicate_query: false # Disable query deduplication for the products subgraph.

Compression

The can compress request bodies to (along with response bodies to clients). It currently supports these algorithms: gzip, br, and deflate.

router.yaml
traffic_shaping:
all:
compression: br # Enable brotli compression for all subgraphs.

response decompression is always supported for these algorithms: gzip, br, and deflate.

Rate limiting

request rate limiting uses the same configuration as client rate limiting, and is calculated per subgraph, not per backend host.

router.yaml
traffic_shaping:
all:
global_rate_limit: # Accept a maximum of 10 requests per 5 secs. Excess requests must be rejected.
capacity: 10
interval: 5s # Must not be greater than 18_446_744_073_709_551_615 milliseconds and not less than 0 milliseconds

Experimental request retry

On failure, requests can be retried automatically. This is deactivated by default for . This uses

, in which every successful request adds an expirable token to a bucket, and every retry consumes a number of those tokens. On top of that, a minimal number of retries per second is available, to test regularly when the retry budget was entirely consumed or on startup when very few requests have been sent. The tokens expire so the budget has a large number of available retries if a lot of recent requests were successful but reduces quickly on frequent failures to avoid sending too much traffic to the .

It is configurable as follows:

router.yaml
traffic_shaping:
all:
experimental_retry:
min_per_sec: 10 # minimal number of retries per second (`min_per_sec`, default is 10 retries per second)
ttl: 10s # for each successful request, we register a token, that expires according to this option (default: 10s)
retry_percent: 0.2 # defines the proportion of available retries to the current number of tokens
retry_mutations: false # allows retries on mutations. This should only be enabled if mutations are idempotent

Variable deduplication

When are sent requests by the using the _entities , it is often the case that the same (identified by a unique @key constraint) is requested multiple times within the execution of a single federated . For example, an author's name might need to be fetched multiple times when accessing a list of a reviews for a product for which the author has written multiple reviews.

To reduce the size of requests and the amount of work they might perform, the list of entities sent can be deduplicated. This is always active.

Query deduplication

If the is simultaneously processing similar queries, it may result in producing multiple identical requests to a . With the deduplicate_query functionality enabled (by default, it is disabled), the can avoid sending the same multiple times and instead buffer one or more of the dependent queries pending the result of the first, and reuse that result to fulfill all of the initial queries. This will reduce the overall traffic to the and the overall client request latency. To meet the criteria for deduplication, the feature must be enabled and the subgraph queries must have have the same HTTP path, headers and body:

router.yaml
traffic_shaping:
all:
deduplicate_query: true # Enable query deduplication for all subgraphs.

HTTP/2

The supports connections over:

  • HTTP/1.1
  • HTTP/1.1 with TLS
  • HTTP/2 with TLS
  • HTTP/2 Cleartext protocol (h2c). This uses HTTP/2 over plaintext connections.

Use the table below to look up the resulting protocol of a connection, based on the subgraph URL and the experimental_http2 option:

URL with http://URL with https://
experimental_http2: disableHTTP/1.1HTTP/1.1 with TLS
experimental_http2: enableHTTP/1.1Either HTTP/1.1 or HTTP/2 with TLS, as determined by the TLS handshake with a subgraph
experimental_http2: http2onlyh2cHTTP/2 with TLS
experimental_http2 not setHTTP/1.1Either HTTP/1.1 or HTTP/2 with TLS, as determined by the TLS handshake with a subgraph

NOTE

Configuring experimental_http2: http2only for a that doesn't support HTTP/2 results in a failed subgraph connection.

Ordering

Traffic shaping always executes these steps in the same order, to ensure a consistent behaviour. Declaration order in the configuration will not affect the runtime order:

  • preparing the request
  • deduplication
  • rate limiting
  • request retry
  • timeout
  • deduplication
  • compression
  • sending the request to the
Previous
Header propagation
Next
CORS
Edit on GitHubEditForumsDiscord

© 2024 Apollo Graph Inc.

Privacy Policy

Company