Caching with Caffeine

Overview

When implementing operation caching with the PreparsedDocumentProvider, our DGS server (built on top of Spring Boot) is prepared to take care of a lot of the details. But there's a couple of things that we will need to provide. The first is a cache implementation.

In this lesson, we will:

Introduce Caffeine, a caching library
Create a new cache instance and configure its size and expiration
Implement the required methods for our implementation of PreparsedDocumentProvider
Test our cache and observe cache hits and cache misses

Caffeine

To create a new cache, we'll use Caffeine, a high-performance caching library commonly used in Java applications. Caffeine is an in-memory cache. This means that each server instance has its own cache that is populated separately; this is in contrast to a distributed cache, such as Redis, where each server shares the same instance.

Our PreparsedDocumentProvider implementation will use this cache to store the parsed and validated GraphQL operations our server receives.

Open up your project's build.gradle file. Down under dependencies, let's bring in the Caffeine library.

build.gradle

dependencies {
    implementation("com.github.ben-manes.caffeine:caffeine:3.1.8")
  // ... other dependencies
}

Be sure to reload your Gradle dependencies so that the new package is incorporated into our project.

Implementing a cache

There are several ways to implement a cache with Caffeine. To adhere to the reactive style of our codebase, we'll use an interface called AsyncCache. By using an asynchronous cache, we can store and load data when it's ready, rather than creating bottlenecks in our code with multiple blocking requests.

Note: Check out the official Caffeine wiki for details on other ways of creating a new cache.

Let's return to our CachingPreparsedDocumentProvider class file and import some dependencies.

CachingPreparsedDocumentProvider

import com.github.benmanes.caffeine.cache.AsyncCache;
import com.github.benmanes.caffeine.cache.Caffeine;
import java.util.concurrent.CompletableFuture;
import graphql.ExecutionInput;
import java.time.Duration;
import java.util.function.Function;

Inside of our class, we'll create a private final property called cache. Our cache will be an AsyncCache type that accepts two type variables; we'll come back to this step, so leave them empty for now.

private final AsyncCache<> cache

Next, to create the cache, we'll call Caffeine.newBuilder. From here, we can chain on configuration for our cache, including how many entries it's allowed to hold, and when a record should be evicted from the cache.

CachingPreparsedDocumentProvider

private final AsyncCache<> cache = Caffeine.newBuilder()
  .maximumSize() // How many entries can the cache contain?
  .expireAfterAccess() // When should the record be removed?
  .buildAsync();

Let's give our cache a generous capacity of 250 records, and an expiration time of 2 minutes to make it easy to test.

Caffeine.newBuilder()
  .maximumSize(250)
  .expireAfterAccess(Duration.ofMinutes(2))
  .buildAsync();

Now we'll jump back to our cache's type definition. AsyncCache accepts two type variables: one for its key type, another for the type of object it stores.

The cache type signature

AsyncCache<KeyType, ValueType>

In our case, our cache will contain objects of type PreparsedDocumentEntry. PreparsedDocumentEntry, imported earlier from the same graphql package as our PreparsedDocumentProvider interface, represents the result of a query that has already been parsed and validated. For our key data type, we'll provide String.

CachingPreparsedDocumentProvider

private final AsyncCache<String, PreparsedDocumentEntry> cache = Caffeine
        .newBuilder()
        .maximumSize(250)
        .expireAfterAccess(Duration.ofMinutes(2))
        .buildAsync();

Now let's put that cache to work!

Building out our class

Recall that the PreparsedDocumentProvider interface requires a specific method to be defined on our class: getDocumentAsync. This method is what gets called with each of the incoming operations our GraphQL server receives. This method's responsibility is to either return the cached operation (if it finds it in the cache), or parse, validate, and then cache it (if it's not present in the cache).

The `getDocumentAsync` method

Make some space in your CachingPreparsedDocumentProvider and let's add the structure for our getDocumentAsync method.

CachingPreparsedDocumentProvider

@Override
public void getDocumentAsync() {}

Note: We need to apply the @Override here to override the behavior of the getDocumentAsync type on the class' supertype, the PreparsedDocumentProvider interface.

Remember the parameters the getDocumentAsync receives? The first is our execution input, the object that contains all the detail about the current operation our GraphQL server is handling. We'll specify a parameter called executionInput, of type ExecutionInput.

public void getDocumentAsync(ExecutionInput executionInput) {}

The second parameter is the function we'll call if the requested operation is not found in the cache. It takes care of the parsing, validating, and caching steps. We'll call this the parseAndValidateFunction, and it's a Function type with two type variables: ExecutionInput and PreparsedDocumentEntry. Let's update our function signature for now—we'll see how these parameters come into play in just a moment.

public void getDocumentAsync(
  ExecutionInput executionInput,
  Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {}

Before filling out our method's logic, let's update our return type. This is an asynchronous function, so we'll return a type of CompletableFuture. Our CompletableFuture will accept a type variable of PreparsedDocumentEntry—this is the actual parsed and validated document that our cache is responsible for storing and retrieving.

public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
  ExecutionInput executionInput,
  Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {}

Great! With our method's signature complete, let's review its mission.

Check the cache to see if it contains the pre-parsed, pre-validated query. If so, return it!
If not, call the parseAndValidateFunction instead

Accordingly, getDocumentAsync needs to call our cache's get method. It will pass in the query that the server is trying to resolve, giving the cache the reference for which pre-parsed, pre-validated document to return. But because the cache might not contain the operation in question, we need to pass it a second argument: a function that should be run in the event of a "cache miss".

cache.get(
  operationString, // The operation string we can use to look up the right parsed and validated doc
  fallbackFunction // The function that will run if the operation is not found (a "cache miss")
)

Accessing the operation

We can get access to the actual operation string (not yet parsed or validated) on our method's executionInput. A handy method, getQuery, gets us exactly what we're looking for.

CachingPreparsedDocumentProvider

public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
  ExecutionInput executionInput,
  Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {

  cache.get(executionInput.getQuery());

}

The fallback function

Now we'll specify the function that should run in the event an operation is not found in the cache.

Here, we want to take our executionInput and pass it on to be parsed and validated. To do that, we'll use the parseAndValidateFunction that the GraphQL process will pass to our getDocumentAsync method.

But we won't call parseAndValidateFunction directly; instead, we'll pass a function that uses apply to call the parseAndValidate function with our executionInput. And to comply with the signature our cache.get call expects for its second argument, we need to give our function some arbitrary input—we'll just call it s for simplicity.

Finally, let's be sure we return the results.

return cache.get(
  executionInput.getQuery(),
  s -> parseAndValidateFunction.apply(executionInput)
);

Note: The parseAndValidateFunction adheres to the Function interface, which gives us four methods to choose from. Here we're using apply to specifically "apply" the function to our execution input. Read more about the Function interface in the Java docs.

Zooming out, here's how our entire method should look.

CachingPreparsedDocumentProvider

@Override
public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
  ExecutionInput executionInput,
  Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {
    return cache.get(
      executionInput.getQuery(),
      s -> parseAndValidateFunction.apply(executionInput)
    );
}

When it comes time to execute a particular operation, the GraphQL process will call our CachingPreparsedDocumentProvider's getDocumentAsync method.

It will pass in the current execution input, as well as a special GraphQL Java-provided function as the value of our parseAndValidateFunction.

Normally, it would automatically use this function to parse, validate, and execute based on the contents of the execution input—but we've just added the wiring to make sure it checks our cache for the parsed and validated document first!

Checking for cache misses

We can also modify getDocumentAsync method to print out when and if there's a cache miss.

Let's add the method below, callIfCacheMiss, to our class. This method will overtake the responsibility of calling parseAndValidateFunction.apply(executionInput), and it will log out the operation it failed to find in the cache.

CachingPreparsedDocumentProvider

public PreparsedDocumentEntry callIfCacheMiss(
  ExecutionInput executionInput,
  Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction)
  {
    System.out.println("Pre-parsed operation wasn't found in cache: " + executionInput.getQuery());
    return parseAndValidateFunction.apply(executionInput);
  }

Next, we'll update getDocumentAsync to delegate this responsibility to our new callIfCacheMiss.

@Override
public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
  ExecutionInput executionInput,
  Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {
  return cache.get(
    executionInput.getQuery(),
    s -> callIfCacheMiss(executionInput, parseAndValidateFunction)
  );
}

Let's restart the server, and jump back into Sandbox.

Task!

I've restarted my server.

We'll set up a new query to ask for a particular listing's details, along with some information about its amenities.

query GetListingAndAmenities($listingId: ID!) {
  listing(id: $listingId) {
    title
    description
    numOfBeds
    amenities {
      name
      category
    }
  }
}

In the Variables panel, provide the following:

{
  "listingId": "listing-1"
}

When we run the query, we should see data in the Response panel, as well as how long the request took in milliseconds. (Take note of this number!) Back in our server terminal we should also see some additional output.

Pre-parsed operation wasn't found in cache: query GetListingAndAmenities($listingId: ID!) {
  listing(id: $listingId) {
    title
    description
    numOfBeds
    amenities {
      name
      category
    }
  }
}

Makes sense. This is our first time running the query, so naturally its pre-parsed, pre-validated self wasn't found in the cache. Let's try running it again, swapping in a different listing ID:

{
  "listingId": "listing-2"
}

This time, there's no output from our server! Our operation was successfully cached, and we could reuse it even with a different listing ID. It's a "cache hit", because we found what we were looking for.

Additionally, we should see that the amount of time that our operation took dropped significantly. We've shaved off more than a few milliseconds—wasted time we otherwise would spend re-parsing and re-validating a familiar operation.

https://studio.apollographql.com/sandbox/explorer

The results of running the operation in Explorer, with the milliseconds highlighted

Let's tweak our operation a little bit. Remove the fields for amenities, and run the operation again. We'll see some new output in our server's terminal; we ran a new operation, so it created a new entry in the cache!

Pre-parsed operation wasn't found in cache: query GetListingAndAmenities($listingId: ID!) {
  listing(id: $listingId) {
    title
    description
    numOfBeds
  }
}

Each time we run a new operation, we should see our terminal output signaling our "cache miss".

And if we wait two minutes and run an operation that was already added to our cache, we should see the same "cache miss" message—after two minutes, the entry has been evicted!

Our operations are being cached, and we've seen our query variables best practice in action: we can cache a single operation, but have it applied to multiple different values without needing to re-parse and re-validate.

Feel free to delete the callIfCacheMiss method, and revert the changes to the getDocumentAsync method. Check out the collapsible section below for guidance.

Practice

The Caffeine caching library

Caffeine is a

we can use to create

. When building a new cache, we provide our type annotation with two

: one that indicates the cache's key data type, and another for the type of object it stores. When configured as part of our PreparsedDocumentProvider implementation, we can use a cache to store the

we've already

once before.

Drag items from this box to the blanks above

a distributed cache
evicted
an in-memory cache
methods
Java class
GraphQL operations
parsed and validated
type variables
caching library
GraphQL variables

Key takeaways

Caffeine is a high-performance caching library we can use to instantiate new caches in Spring Boot
AsyncCache is one cache implementation that avoid blocking requests and keeps our application reactive
We can configure our cache with specific limitations on capacity as well as expiration time
When implementing the PreparsedDocumentProvider interface, we need to provide a getDocumentAsync function that follows a specific signature
When a particular operation is not found in our cache, we need to call getDocumentAsync's second argument, a parse and validate function, to complete these two steps and continue execution

Up next

We've just wrapped up the first half of our caching journey. Now that we've removed the need to re-parse and re-validate operations that our server's already processed, we can move onto our next topic: caching our data source responses.

Overview

Caffeine

Implementing a cache

Building out our class

The getDocumentAsync method

Accessing the operation

The fallback function

Checking for cache misses

Practice

Key takeaways

Up next

Share your questions and comments about this lesson

The `getDocumentAsync` method