In-memory caching in the Apollo Router

The Apollo Router uses an in-memory LRU cache to store the following data:

• Generated query plans
• Automatic persisted queries (APQ)
• Introspection responses

You can configure certain caching behaviors for generated query plans and APQ (but not introspection responses).

Caching query plans

Whenever your router receives an incoming GraphQL operation, it generates a query plan to determine which subgraphs it needs to query to resolve that operation.

By caching previously generated query plans, your router can skip generating them again if a client later sends the exact same operation. This improves your router's responsiveness.

The Apollo Router enables query plan caching by default. In your router's YAML config file, you can configure the maximum number of query plan entries in the cache like so:

1
supergraph:
2
  query_planning:
3
    experimental_cache:
4
      in_memory:
5
        limit: 512 # This is the default value.

On schema reloads, the cache will be reset, and queries will need to go through query planning again. To avoid latencies right after the reload, you can configure the router to pregenerate query plans for the most used queries before switching to the new schema:

1
supergraph:
2
  query_planning:
3
    # Pre-plan the 100 most used operations when the supergraph changes.  (Default is "0", disabled.)
4
    warmed_up_queries: 100
5
    experimental_cache:
6
      in_memory:
7
        limit: 512

Cache warm-up

When loading a new schema, a query plan might change for some queries, so cached query plans cannot be reused.

To prevent increased latency upon query plan cache invalidation, the Router precomputes query plans for:

• The most used queries from the cache.
• The entire list of persisted queries.

Precomputed plans will be cached before the Router switches traffic over to the new schema.

By default, the Router warms up the cache with 30% of the queries already in cache, but it can be configured as follows:

1
supergraph:
2
  query_planning:
3
    # Pre-plan the 100 most used operations when the supergraph changes
4
    warmed_up_queries: 100

To get more information on the planning and warm-up process use the following metrics (where <storage> can be redis for distributed cache or memory):

• counters:

  • apollo_router_cache_size{kind="query planner", storage="<storage>}: current size of the cache (only for in-memory cache)
  • apollo_router_cache_hit_count{kind="query planner", storage="<storage>}
  • apollo_router_cache_miss_count{kind="query planner", storage="<storage>}

• histograms:

  • apollo_router_query_planning_time: time spent planning queries
  • apollo_router_schema_loading_time: time spent loading a schema
  • apollo_router_cache_hit_time{kind="query planner", storage="<storage>}: time to get a value from the cache
  • apollo_router_cache_miss_time{kind="query planner", storage="<storage>}

Typically, we would look at apollo_router_cache_size and the cache hit rate to define the right size of the in memory cache, then look at apollo_router_schema_loading_time and apollo_router_query_planning_time to decide how much time we want to spend warming up queries.

Cache warm-up with distributed caching

If the Router is using distributed caching for query plans, the warm-up phase will also store the new query plans in Redis. Since all Router instances might have the same distributions of queries in their in-memory cache, the list of queries is shuffled before warm-up, so each Router instance can plan queries in a different order and share their results through the cache.

Caching automatic persisted queries (APQ)

Automatic Persisted Queries (APQ) enable GraphQL clients to send a server the hash of their query string, instead of sending the query string itself. When query strings are very large, this can significantly reduce network usage.

The Apollo Router supports using APQ in its communications with both clients and subgraphs:

• In its communications with clients, the router acts as a GraphQL server, because it receives queries from clients.
• In its communications with subgraphs, the router acts as a GraphQL client, because it sends queries to subgraphs.

Because the router's role differs between these two interactions, you configure these APQ settings separately.

APQ with clients

The Apollo Router enables APQ caching for client operations by default. In your router's YAML config file, you can configure the maximum number of APQ entries in the cache like so:

1
apq:
2
  router:
3
    cache:
4
      in_memory:
5
        limit: 512 # This is the default value.

You can also disable client APQ support entirely like so:

1
apq:
2
  enabled: false

APQ with subgraphs

By default, the Apollo Router does not use APQ when sending queries to its subgraphs.

In your router's YAML config file, you can configure this APQ support with a combination of global and per-subgraph settings:

1
apq:
2
  subgraph:
3
    # Disables subgraph APQ globally except where overridden per-subgraph
4
    all:
5
      enabled: false
6
    # Override global APQ setting for individual subgraphs
7
    subgraphs:
8
      products:
9
        enabled: true

In the example above, subgraph APQ is disabled except for the products subgraph.