Docs
Launch GraphOS Studio

In-memory caching in the Apollo Router


The uses an in-memory LRU cache to store the following data:

  • Generated query plans
  • Automatic persisted queries (APQ)
  • responses

You can configure certain caching behaviors for generated and (but not responses).

💡 TIP

If you have a , you can also configure a Redis-backed distributed cache that enables multiple instances to share cached values. For details, see Distributed caching in the Apollo Router.

Caching query plans

Whenever your receives an incoming , it generates a query plan to determine which it needs to to resolve that .

By caching previously generated , your can skip generating them again if a client later sends the exact same . This improves your 's responsiveness.

The enables caching by default. In your router's YAML config file, you can configure the maximum number of entries in the cache like so:

router.yaml
supergraph:
query_planning:
cache:
in_memory:
limit: 512 # This is the default value.

On schema reloads, the cache will be reset, and queries will need to go through ning again. To avoid latencies right after the reload, you can configure the to pregenerate query plans for the most used queries before switching to the new schema:

router.yaml
supergraph:
query_planning:
# Pre-plan the 100 most used operations when the supergraph changes. (Default is "0", disabled.)
warmed_up_queries: 100
cache:
in_memory:
limit: 512

Cache warm-up

When loading a new schema, a might change for some queries, so cached query plans cannot be reused.

To prevent increased latency upon cache invalidation, the precomputes query plans for:

  • The most used queries from the cache.
  • The entire list of .

Precomputed plans will be cached before the switches traffic over to the new schema.

By default, the warms up the cache with 30% of the queries already in cache, but it can be configured as follows:

router.yaml
supergraph:
query_planning:
# Pre-plan the 100 most used operations when the supergraph changes
warmed_up_queries: 100

To get more information on the planning and warm-up process use the following metrics (where <storage> can be redis for distributed cache or memory):

  • counters:

    • apollo_router_cache_size{kind="query planner", storage="<storage>}: current size of the cache (only for in-memory cache)
    • apollo_router_cache_hit_count{kind="query planner", storage="<storage>}
    • apollo_router_cache_miss_count{kind="query planner", storage="<storage>}
  • histograms:

    • apollo_router_query_planning_time: time spent planning queries
    • apollo_router_schema_loading_time: time spent loading a schema
    • apollo_router_cache_hit_time{kind="query planner", storage="<storage>}: time to get a value from the cache
    • apollo_router_cache_miss_time{kind="query planner", storage="<storage>}

Typically, we would look at apollo_router_cache_size and the cache hit rate to define the right size of the in memory cache, then look at apollo_router_schema_loading_time and apollo_router_query_planning_time to decide how much time we want to spend warming up queries.

Cache warm-up with distributed caching

If the is using for , the warm-up phase will also store the new query plans in Redis. Since all Router instances might have the same distributions of queries in their in-memory cache, the list of queries is shuffled before warm-up, so each Router instance can plan queries in a different order and share their results through the cache.

Caching automatic persisted queries (APQ)

Automatic Persisted Queries (APQ) enable to send a server the hash of their string, instead of sending the string itself. When query strings are very large, this can significantly reduce network usage.

The supports using in its communications with both clients and :

  • In its communications with clients, the acts as a server, because it receives queries from clients.
  • In its communications with subgraphs, the acts as a client, because it sends queries to .

Because the 's role differs between these two interactions, you configure these settings separately.

APQ with clients

The enables caching for client by default. In your router's YAML config file, you can configure the maximum number of entries in the cache like so:

router.yaml
apq:
router:
cache:
in_memory:
limit: 512 # This is the default value.

You can also disable client support entirely like so:

router.yaml
apq:
enabled: false

APQ with subgraphs

By default, the does not use when sending queries to its .

In your 's YAML config file, you can configure this support with a combination of global and per- settings:

router.yaml
apq:
subgraph:
# Disables subgraph APQ globally except where overridden per-subgraph
all:
enabled: false
# Override global APQ setting for individual subgraphs
subgraphs:
products:
enabled: true

In the example above, is disabled except for the products .

Previous
Overview
Next
Distributed caching
Edit on GitHubEditForumsDiscord

© 2024 Apollo Graph Inc.

Privacy Policy

Company