3. Caching with Caffeine
10m

Overview

When implementing caching with the PreparsedDocumentProvider, our DGS server (built on top of Spring Boot) is prepared to take care of a lot of the details. But there's a couple of things that we will need to provide. The first is a cache implementation.

In this lesson, we will:

  • Introduce Caffeine, a caching library
  • Create a new cache instance and configure its size and expiration
  • Implement the required methods for our implementation of PreparsedDocumentProvider
  • Test our cache and observe cache hits and cache misses

Caffeine

To create a new cache, we'll use Caffeine, a high-performance caching library commonly used in Java applications. Caffeine is an in-memory cache. This means that each server instance has its own cache that is populated separately; this is in contrast to a distributed cache, such as Redis, where each server shares the same instance.

Our PreparsedDocumentProvider implementation will use this cache to store the parsed and validated our server receives.

Open up your project's build.gradle file. Down under dependencies, let's bring in the Caffeine library.

build.gradle
dependencies {
implementation("com.github.ben-manes.caffeine:caffeine:3.1.8")
// ... other dependencies
}

Be sure to reload your Gradle dependencies so that the new package is incorporated into our project.

Implementing a cache

There are several ways to implement a cache with Caffeine. To adhere to the reactive style of our codebase, we'll use an interface called AsyncCache. By using an asynchronous cache, we can store and load data when it's ready, rather than creating bottlenecks in our code with multiple blocking requests.

Note: Check out the official Caffeine wiki for details on other ways of creating a new cache.

Let's return to our CachingPreparsedDocumentProvider class file and import some dependencies.

CachingPreparsedDocumentProvider
import com.github.benmanes.caffeine.cache.AsyncCache;
import com.github.benmanes.caffeine.cache.Caffeine;
import java.util.concurrent.CompletableFuture;
import graphql.ExecutionInput;
import java.time.Duration;
import java.util.function.Function;

Inside of our class, we'll create a private final property called cache. Our cache will be an AsyncCache type that accepts two type ; we'll come back to this step, so leave them empty for now.

private final AsyncCache<> cache

Next, to create the cache, we'll call Caffeine.newBuilder. From here, we can chain on configuration for our cache, including how many entries it's allowed to hold, and when a record should be evicted from the cache.

CachingPreparsedDocumentProvider
private final AsyncCache<> cache = Caffeine.newBuilder()
.maximumSize() // How many entries can the cache contain?
.expireAfterAccess() // When should the record be removed?
.buildAsync();

Let's give our cache a generous capacity of 250 records, and an expiration time of 2 minutes to make it easy to test.

Caffeine.newBuilder()
.maximumSize(250)
.expireAfterAccess(Duration.ofMinutes(2))
.buildAsync();

Now we'll jump back to our cache's type definition. AsyncCache accepts two type : one for its key type, another for the type of object it stores.

The cache type signature
AsyncCache<KeyType, ValueType>

In our case, our cache will contain objects of type PreparsedDocumentEntry. PreparsedDocumentEntry, imported earlier from the same graphql package as our PreparsedDocumentProvider interface, represents the result of a that has already been parsed and validated. For our key data type, we'll provide String.

CachingPreparsedDocumentProvider
private final AsyncCache<String, PreparsedDocumentEntry> cache = Caffeine
.newBuilder()
.maximumSize(250)
.expireAfterAccess(Duration.ofMinutes(2))
.buildAsync();

Now let's put that cache to work!

Building out our class

Recall that the PreparsedDocumentProvider interface requires a specific method to be defined on our class: getDocumentAsync. This method is what gets called with each of the incoming our receives. This method's responsibility is to either return the cached operation (if it finds it in the cache), or parse, validate, and then cache it (if it's not present in the cache).

The getDocumentAsync method

Make some space in your CachingPreparsedDocumentProvider and let's add the structure for our getDocumentAsync method.

CachingPreparsedDocumentProvider
@Override
public void getDocumentAsync() {}

Note: We need to apply the @Override here to override the behavior of the getDocumentAsync type on the class' supertype, the PreparsedDocumentProvider interface.

Remember the parameters the getDocumentAsync receives? The first is our execution input, the object that contains all the detail about the current our is handling. We'll specify a parameter called executionInput, of type ExecutionInput.

public void getDocumentAsync(ExecutionInput executionInput) {}

The second parameter is the function we'll call if the requested is not found in the cache. It takes care of the parsing, validating, and caching steps. We'll call this the parseAndValidateFunction, and it's a Function type with two type : ExecutionInput and PreparsedDocumentEntry. Let's update our function signature for now—we'll see how these parameters come into play in just a moment.

public void getDocumentAsync(
ExecutionInput executionInput,
Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {}

Before filling out our method's logic, let's update our return type. This is an asynchronous function, so we'll return a type of CompletableFuture. Our CompletableFuture will accept a type of PreparsedDocumentEntry—this is the actual parsed and validated that our cache is responsible for storing and retrieving.

public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
ExecutionInput executionInput,
Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {}

Great! With our method's signature complete, let's review its mission.

  1. Check the cache to see if it contains the pre-parsed, pre-validated . If so, return it!
  2. If not, call the parseAndValidateFunction instead

Accordingly, getDocumentAsync needs to call our cache's get method. It will pass in the that the server is trying to resolve, giving the cache the reference for which pre-parsed, pre-validated to return. But because the cache might not contain the in question, we need to pass it a second : a function that should be run in the event of a "cache miss".

cache.get(
operationString, // The operation string we can use to look up the right parsed and validated doc
fallbackFunction // The function that will run if the operation is not found (a "cache miss")
)

Accessing the operation

We can get access to the actual string (not yet parsed or validated) on our method's executionInput. A handy method, getQuery, gets us exactly what we're looking for.

CachingPreparsedDocumentProvider
public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
ExecutionInput executionInput,
Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {
cache.get(executionInput.getQuery());
}

The fallback function

Now we'll specify the function that should run in the event an is not found in the cache.

Here, we want to take our executionInput and pass it on to be parsed and validated. To do that, we'll use the parseAndValidateFunction that the process will pass to our getDocumentAsync method.

But we won't call parseAndValidateFunction directly; instead, we'll pass a function that uses apply to call the parseAndValidate function with our executionInput. And to comply with the signature our cache.get call expects for its second , we need to give our function some arbitrary input—we'll just call it s for simplicity.

Finally, let's be sure we return the results.

return cache.get(
executionInput.getQuery(),
s -> parseAndValidateFunction.apply(executionInput)
);

Note: The parseAndValidateFunction adheres to the Function interface, which gives us four methods to choose from. Here we're using apply to specifically "apply" the function to our execution input. Read more about the Function interface in the Java docs.

Zooming out, here's how our entire method should look.

CachingPreparsedDocumentProvider
@Override
public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
ExecutionInput executionInput,
Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {
return cache.get(
executionInput.getQuery(),
s -> parseAndValidateFunction.apply(executionInput)
);
}

When it comes time to execute a particular , the process will call our CachingPreparsedDocumentProvider's getDocumentAsync method.

It will pass in the current execution input, as well as a special Java-provided function as the value of our parseAndValidateFunction.

Normally, it would automatically use this function to parse, validate, and execute based on the contents of the execution input—but we've just added the wiring to make sure it checks our cache for the parsed and validated first!

Checking for cache misses

We can also modify getDocumentAsync method to print out when and if there's a cache miss.

Let's add the method below, callIfCacheMiss, to our class. This method will overtake the responsibility of calling parseAndValidateFunction.apply(executionInput), and it will log out the it failed to find in the cache.

CachingPreparsedDocumentProvider
public PreparsedDocumentEntry callIfCacheMiss(
ExecutionInput executionInput,
Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction)
{
System.out.println("Pre-parsed operation wasn't found in cache: " + executionInput.getQuery());
return parseAndValidateFunction.apply(executionInput);
}

Next, we'll update getDocumentAsync to delegate this responsibility to our new callIfCacheMiss.

@Override
public CompletableFuture<PreparsedDocumentEntry> getDocumentAsync(
ExecutionInput executionInput,
Function<ExecutionInput, PreparsedDocumentEntry> parseAndValidateFunction
) {
return cache.get(
executionInput.getQuery(),
s -> callIfCacheMiss(executionInput, parseAndValidateFunction)
);
}

Let's restart the server, and jump back into Sandbox.

Task!

We'll set up a new to ask for a particular listing's details, along with some information about its amenities.

query GetListingAndAmenities($listingId: ID!) {
listing(id: $listingId) {
title
description
numOfBeds
amenities {
name
category
}
}
}

In the Variables panel, provide the following:

{
"listingId": "listing-1"
}

When we run the , we should see data in the Response panel, as well as how long the request took in milliseconds. (Take note of this number!) Back in our server terminal we should also see some additional output.

Pre-parsed operation wasn't found in cache: query GetListingAndAmenities($listingId: ID!) {
listing(id: $listingId) {
title
description
numOfBeds
amenities {
name
category
}
}
}

Makes sense. This is our first time running the , so naturally its pre-parsed, pre-validated self wasn't found in the cache. Let's try running it again, swapping in a different listing ID:

{
"listingId": "listing-2"
}

This time, there's no output from our server! Our was successfully cached, and we could reuse it even with a different listing ID. It's a "cache hit", because we found what we were looking for.

Additionally, we should see that the amount of time that our took dropped significantly. We've shaved off more than a few milliseconds—wasted time we otherwise would spend re-parsing and re-validating a familiar .

https://studio.apollographql.com/sandbox/explorer

The results of running the operation in Explorer, with the milliseconds highlighted

Let's tweak our a little bit. Remove the for amenities, and run the again. We'll see some new output in our server's terminal; we ran a new , so it created a new entry in the cache!

Pre-parsed operation wasn't found in cache: query GetListingAndAmenities($listingId: ID!) {
listing(id: $listingId) {
title
description
numOfBeds
}
}

Each time we run a new , we should see our terminal output signaling our "cache miss".

And if we wait two minutes and run an that was already added to our cache, we should see the same "cache miss" message—after two minutes, the entry has been evicted!

Our are being cached, and we've seen our best practice in action: we can cache a single operation, but have it applied to multiple different values without needing to re-parse and re-validate.

Feel free to delete the callIfCacheMiss method, and revert the changes to the getDocumentAsync method. Check out the collapsible section below for guidance.

Practice

The Caffeine caching library
Caffeine is a 
 
 we can use to create 
 
. When building a new cache, we provide our type annotation with two 
 
: one that indicates the cache's key data type, and another for the type of object it stores. When configured as part of our PreparsedDocumentProvider implementation, we can use a cache to store the 
 
 we've already 
 
 once before.

Drag items from this box to the blanks above

  • type variables

  • GraphQL operations

  • caching library

  • an in-memory cache

  • evicted

  • parsed and validated

  • Java class

  • methods

  • GraphQL variables

  • a distributed cache

Key takeaways

  • Caffeine is a high-performance caching library we can use to instantiate new caches in Spring Boot
  • AsyncCache is one cache implementation that avoid blocking requests and keeps our application reactive
  • We can configure our cache with specific limitations on capacity as well as expiration time
  • When implementing the PreparsedDocumentProvider interface, we need to provide a getDocumentAsync function that follows a specific signature
  • When a particular is not found in our cache, we need to call getDocumentAsync's second , a parse and validate function, to complete these two steps and continue execution

Up next

We've just wrapped up the first half of our caching journey. Now that we've removed the need to re-parse and re-validate that our server's already processed, we can move onto our next topic: caching our responses.

Previous

Share your questions and comments about this lesson

This course is currently in

beta
. Your feedback helps us improve! If you're stuck or confused, let us know and we'll help you out. All comments are public and must follow the Apollo Code of Conduct. Note that comments that have been resolved or addressed may be removed.

You'll need a GitHub account to post below. Don't have one? Post in our Odyssey forum instead.