Odyssey

Data loaders with TypeScript & Apollo Server

Course overview and setupThe n+1 problemData loaders under the hoodUsing a data loader
4. Using a data loader
3m

Overview

It's time to introduce data loaders to our application.

In this lesson, we will:

  • Implement the DataLoader class with a batching function
  • Update our getAmenities method to pass each listing ID to the data loader
  • Test out our new data loading capability

Adding the data loader

Let's bring the dataloader package into our application. Open up a new terminal in the root of your project, and run the following command.

odyssey-dataloaders-typescript
npm install --save dataloader

Next, we'll jump to our ListingAPI class. Here, we'll introduce a new method that does the following:

  1. Creates a new DataLoader
  2. Provides a batching function that gathers listingIds and makes a single request to the REST API

At the top of the file, bring in DataLoader from the dataloader package.

listing-api.ts
import DataLoader from "dataloader";

Inside of our ListingAPI class, let's set up a new private field for our data loader instantiation. This will make it accessible only from within this same class.

private batchAmenities = new DataLoader()

The DataLoader constructor accepts a batch loading function, which we'll define in-line. This batch function accepts an array of keys, and returns a Promise that resolves to an array of values. For the array of keys, we'll set a parameter called listingIds, and we can add our return type annotation as well: Promise<Amenity[][]>.

private batchAmenities = new DataLoader(
(listingIds): Promise<Amenity[][]> => {
// TODO
}
)

By default, our data loader returns a Promise that resolves to an array of values.

If we had just one amenity per listing, then a simple annotation of Promise<Amenity[]> would suffice: our Promise would resolve to one list of Amenity objects. However, this is not the case. Each listing has its own list of amenities: for this reason, the Promise will resolve to a list of amenity lists: one for each listing. We'll log out the results of our REST API call shortly to see the shape of the response in action.

Next, we'll make this function async, and await the results of calling our REST endpoint, GET /amenities/listings.

private batchAmenities = new DataLoader(
async (listingIds): Promise<Amenity[][]> => {
await this.get<Amenity[][]>("amenities/listings");
}
);

This endpoint expects a query parameter called ids, a comma-separated string of listing IDs. We'll add this as a second argument to the this.get function, an object with a params property, passing in another object with the query parameter ids. Because our data loader receives its listingIds as an array, we'll use the join method, specifying a comma as the separator for each value.

private batchAmenities = new DataLoader(
async (listingIds): Promise<Amenity[][]> => {
await this.get<Amenity[][]>("amenities/listings", {
params: {
ids: listingIds.join(","),
},
});
}
);

Finally, let's capture the results of calling our endpoint in a new constant called amenitiesLists, then return the results.

private batchAmenities = new DataLoader(
async (listingIds): Promise<Amenity[][]> => {
const amenitiesLists = await this.get<Amenity[][]>("amenities/listings", {
params: {
ids: listingIds.join(","),
},
});
return amenitiesLists;
}
);

Before we move on, let's add a couple of log statements so that we can keep an eye on how our method is working. We'll log out the list of listingIds passed into our data loader, and the amenitiesLists results.

private batchAmenities = new DataLoader(
async (listingIds): Promise<Amenity[][]> => {
console.log("Making one batched call with ", listingIds);
const amenitiesLists = await this.get<Amenity[][]>("amenities/listings", {
params: {
ids: listingIds.join(","),
},
});
console.log(amenitiesLists);
return amenitiesLists;
}
);

That's our data loader method taken care of; now we need to call it from inside this class.

import { RESTDataSource } from "@apollo/datasource-rest";
import DataLoader from "dataloader";
import { Listing, Amenity, CreateListingInput } from "../types";
export class ListingAPI extends RESTDataSource {
baseURL = "https://rt-airlock-services-listing.herokuapp.com/";
private batchAmenities = new DataLoader(
async (listingIds): Promise<Amenity[][]> => {
console.log("Making one batched call with ", listingIds);
const amenitiesLists = await this.get<Amenity[][]>("amenities/listings", {
params: {
ids: listingIds.join(","),
},
});
console.log(amenitiesLists);
return amenitiesLists;
}
);
getAmenities(listingId: string): Promise<Amenity[]> {
console.log("Making a follow-up call for amenities with ", listingId);
return this.get<Amenity[]>(`listings/${listingId}/amenities`);
}
// ... other methods
}

Updating getAmenities

Scrolling down in our class, we'll find the getAmenities method that our Listing.amenities resolver has historically called.

Rather than calling the GET /listings/:id/amenities endpoint for each listing, we'll update our method to instead pass each listing ID it receives to the data loader. We can do this by calling load on our private field, batchAmenities.

getAmenities(listingId: string): Promise<Amenity[]> {
console.log("Making a follow-up call for amenities with ", listingId);
- return this.get<Amenity[]>(`listings/${listingId}/amenities`);
+ return this.batchAmenities.load(listingId);
}

Let's also update the log statement here: rather than reporting that we're making a follow-up request, let's change the wording so it's clear that we're passing a particular listing ID to the data loader.

getAmenities(listingId: string): Promise<Amenity[]> {
- console.log("Making a follow-up call for amenities with ", listingId);
+ console.log("Passing listing ID to the data loader: ", listingId);
return this.batchAmenities.load(listingId);
}

That's it for our data source class! And we have no changes to make to our resolver: it will continue to call the getAmenities method, which will delegate the responsibility for batching together and loading data for all the listing IDs requested in a single query. Let's test it out!

import { RESTDataSource } from "@apollo/datasource-rest";
import DataLoader from "dataloader";
import { Listing, Amenity, CreateListingInput } from "../types";
export class ListingAPI extends RESTDataSource {
baseURL = "https://rt-airlock-services-listing.herokuapp.com/";
private batchAmenities = new DataLoader(
async (listingIds): Promise<Amenity[][]> => {
console.log("Making one batched call with ", listingIds);
const amenitiesLists = await this.get<Amenity[][]>("amenities/listings", {
params: {
ids: listingIds.join(","),
},
});
console.log(amenitiesLists);
return amenitiesLists;
}
);
getAmenities(listingId: string): Promise<Amenity[]> {
console.log("Passing listing ID to the data loader: ", listingId);
return this.batchAmenities.load(listingId);
}
// ... other methods
}

Testing the data loader

Make sure that your server is still running, and let's jump back to Sandbox.

We'll run the same query again for featured listings and their amenities.

query GetFeaturedListingsAmenities {
featuredListings {
id
title
amenities {
id
name
category
}
}
}

When we execute the query, we get data just like before! But let's check on what happened under the hood, back in our terminal.

First, we'll see three lines: one for each listing ID involved in our featured listings request.

Passing listing ID to the data loader: listing-1
Passing listing ID to the data loader: listing-2
Passing listing ID to the data loader: listing-3

These lines are followed by output from our single batched call:

Making one batched call with [ 'listing-1', 'listing-2', 'listing-3' ]

After that we should see one big array printed out: it contains multiple smaller arrays, each of which contains multiple amenities! These are the amenity lists for each of our requested listing IDs. Our listing IDs have successfully been batched together in a single request! 👏👏👏

Practice

Which of the following statements describes the relationship between a data loader and a resolver function?
Code Challenge!

Use the DataLoader class to create a new data loader instance on a property called batchedMissions. The DataLoader constructor should 1) accept a list of planetIds, 2) return a Promise that resolves to an array of Mission types, and 3) return the results of calling this.get with the nextMission endpoint. Pass a string of comma-separated planetIds as a parameter called planets.

Code Challenge!

Update getPlanetMissions so that it passes its planetId argument to the batchedMissions data loader. (Remove the this.get call.)

Key takeaways

  • We create a new DataLoader per request, passing into its constructor a batching function that receives an array of keys and returns an array of values.
  • To actually use a data loader, we call its load method and pass in each key.
  • The GraphQL dataloader package comes with caching benefits, and will deduplicate the keys it makes requests for.

Congratulations!

And with that, you've done it! You've tackled the n+1 problem—with data loaders in your toolbelt, you have the resources to boost your app's efficiency. We've taken a basic data fetching strategy in our application, and made it into something that can scale with our queries and features. By employing data loaders, we've seen how we can make fewer, more efficient requests across the network, delivering up data much faster than we ever could have before.

Next up: learn how to grow your GraphQL API with an entirely new domain in Federation with Apollo Server & TypeScript. We'll cover the best practices of building a federated graph, along with the GraphOS tools you'll use to make the path toward robust enterprise APIs smooth, observable, and performant!

Thanks for joining us in this course, and we can't wait to see you in the next.

Previous

Share your questions and comments about this lesson

Your feedback helps us improve! If you're stuck or confused, let us know and we'll help you out. All comments are public and must follow the Apollo Code of Conduct. Note that comments that have been resolved or addressed may be removed.

You'll need a GitHub account to post below. Don't have one? Post in our Odyssey forum instead.

              field

              A unit of data that belongs to a type in a schema. Every GraphQL query requests one or more fields.

              type Author {
              # id, firstName, and lastName are all fields of the Author type
              id: Int!
              firstName: String
              lastName: String
              }
              query

              A request for specific data from a GraphQL server. Clients define the structure of the response, enabling precise and efficient data retrieval.

              argument

              A key-value pair associated with a particular schema field that lets operations pass data to that field's resolver.

              Argument values can be hardcoded as literal values (shown below for clarity) or provided via GraphQL variables (recommended).

              query GetHuman {
              human(id: "200") {
              name
              height(unit: "meters")
              }
              }
              query

              A request for specific data from a GraphQL server. Clients define the structure of the response, enabling precise and efficient data retrieval.

              resolver

              A function that populates data for a particular field in a GraphQL schema. For example:

              const resolvers = {
              Query: {
              author(root, args, context, info) {
              return find(authors, { id: args.id });
              },
              },
              };
              field

              A unit of data that belongs to a type in a schema. Every GraphQL query requests one or more fields.

              type Author {
              # id, firstName, and lastName are all fields of the Author type
              id: Int!
              firstName: String
              lastName: String
              }
              resolver

              A function that populates data for a particular field in a GraphQL schema. For example:

              const resolvers = {
              Query: {
              author(root, args, context, info) {
              return find(authors, { id: args.id });
              },
              },
              };
              query

              A request for specific data from a GraphQL server. Clients define the structure of the response, enabling precise and efficient data retrieval.

              query

              A request for specific data from a GraphQL server. Clients define the structure of the response, enabling precise and efficient data retrieval.

              query

              A request for specific data from a GraphQL server. Clients define the structure of the response, enabling precise and efficient data retrieval.

              GraphQL

              An open-source query language and specification for APIs that enables clients to request specific data, promoting efficiency and flexibility in data retrieval.

              GraphQL

              An open-source query language and specification for APIs that enables clients to request specific data, promoting efficiency and flexibility in data retrieval.

              graph

              A schema-based data model representing how different data elements interconnect and can be accessed.

              GraphOS

              A platform for building and managing a supergraph. It provides a management plane to test and ship changes and runtime capabilities to secure and monitor the graph.

              NEW COURSE ALERT

              Introducing Apollo Connectors

              Connectors are the new and easy way to get started with GraphQL, using existing REST APIs.

              Say goodbye to GraphQL servers and resolvers—now, everything happens in the schema!

              Take the course