Validating schema changes

How to validate your schema in your existing CI workflow

The Apollo GraphQL Platform allows developers to confidently iterate a GraphQL schema by validating the new schema against field-level usage data from the previous schema. By knowing exactly which clients will be broken by a new schema, developers can avoid inadvertently deploying a breaking change.

A GraphQL schema can change in a number of ways between releases and, depending on the type of change, can affect clients in a variety of ways. Since changes can range from “decidedly safe” to “certain breakage”, it’s helpful to use schema tools which are aware of actual API usage.

By comparing a new schema to the last published schema, the Apollo Platform can highlight points of concern by showing detailed schema changes alongside current usage information for those fields. With this pairing of data, the risks of changes can be greatly reduced.

Understanding schema changes

Versioning is a technique to prevent necessary changes from becoming “breaking changes” which affect the existing consumers of an API. These iterations might be as trivial as renaming a field, or as substantial as refactoring the whole data model.

Developers who have worked with REST APIs in the past have probably recognized various patterns for versioning the API, commonly by using a different URI (e.g. /api/v1, /api/v2, etc.) or a query parameter (e.g. ?version=1). With this technique, an application can easily end up with many different API endpoints over time, and the question of when an API can be deprecated can become problematic.

It might be tempting to version a GraphQL API the same way, but it’s unnecessary with the right techniques. By following the strategies and precautions outlined in this guide and using Apollo tooling that adds clarity to every change, many iterations of an API can be served from a single endpoint.

Field usage

Rather than returning extensive amounts of data which might not be necessary, GraphQL allows consumers to specify exactly what data they need. This field-based granularity is valuable and avoids “over-fetching” but also makes it more difficult to understand what data is currently being used.

To improve the understanding of field usage within an API, Apollo Server extends GraphQL with rich tracing data that demonstrates how a GraphQL field is used and when it’s safe to change or eliminate a field.

For details on how tracing data can be used to avoid shipping breaking changes to clients, check out the schema history tooling in Apollo Engine which utilizes actual usage data to provide warnings and notices about changes that might break existing clients.

Since GraphQL clients only receive exactly what they ask for, adding new fields, arguments, queries, or mutations won’t introduce any new breaking changes and these changes can be confidently made without consideration about existing clients or field usage metrics.

Field rollover is a term given to an API change that’s an evolution of a field, such as a rename or a change in arguments. Some of these changes can be really small, resulting in many variations and making an API harder to manage.

We’ll go over these two kinds of field rollovers separately and show how to make these changes safely.

Renaming or removing a field

When a field is unused, renaming or removing it is as straightforward as it sounds: it can be renamed or removed. Unfortunately, if a GraphQL deployment doesn’t have per-field usage metrics, additional considerations should be made.

Take the following user query as an example:

type Query {
 user(id: ID!): User
}

We may want to rename it to getUser to be more descriptive of what the query is for, like so:

type Query {
  getUser(id: ID!): User
}

Even if that was the only change, this would be a breaking change for some clients, since those expecting a user query would receive error.

To make this change safely, instead of renaming the existing field we can simply add a new getUser field and leave the existing user field untouched. To prevent code duplication, the resolver logic can be shared between the two fields:

const getUserResolver = (root, args, context) => {
  context.User.getById(args.id);
};

const resolvers = {
  Query: {
    getUser: getUserResolver,
    user: getUserResolver,
  },
};

Deprecating a field

The tactic we used works well to avoid breaking changes, but we still haven’t provided a way for consumers to know that they should switch to using the new field name. Luckily, the GraphQL specification provides a built-in @deprecated schema directive (sometimes called decorators in other languages):

type Query {
  user(id: ID!): User @deprecated(reason: "renamed to 'getUser'")
  getUser(id: ID!): User
}

GraphQL-aware client tooling, like GraphQL Playground and GraphiQL, use this information to assist developers in making the right choices. These tools will:

  • Provide developers with the helpful deprecation message referring them to the new name.
  • Avoid auto-completing the field.

Over time, usage will fall for the deprecated field and grow for the new field.

Using tools like Apollo Engine, it’s possible to make educated decisions about when to retire a field based on actual usage data through schema analytics.

Non-breaking changes

Sometimes we want to keep a field, but change how clients use it by adjusting its variables. For example, if we had a getUsers query that we used to fetch user data based off of a list of user ids, but wanted to change the arguments to support a groupId to look up users of a group or filter the users requested by the ids argument to only return users in the group:

type Query {
  # what we have
  getUsers(ids: [ID!]!): [User]!

  # what we want to end up with
  getUsers(ids: [ID!], groupId: ID!): [User]!
}

Since this is an additive change, and doesn’t actually change the default behavior of the getUsers query, this isn’t a breaking change!

Breaking changes

An example of a breaking change on an argument would be renaming (or deleting) an argument.

type Query {
  # What we have.
  getUsers(ids: [ID!], groupId: ID!): [User]!

  # What we want to end up with.
  getUsers(ids: [ID!], groupIds: [ID!]): [User]!
}

There’s no way to mark an argument as deprecated, but there are a couple options.

If we wanted to leave the old groupId argument active, we wouldn’t need to do anything; adding a new argument isn’t a breaking change as long as existing functionality doesn’t change.

Instead of supporting it, if we wanted to remove the old argument, the safest option would be to create a new field and deprecate the current getUsers field.

Using an API management tool, like the Apollo platform, it’s possible to determine when usage of an old field has dropped to an acceptable level and remove it. The previously discussed field rollover section gives more info on how to do that.

Of course, it’s also possible to leave the field in place indefinitely!

Checking schema changes with the Apollo CLI

To check and see the difference between the current published schema and a new version, run the following command, substituting the appropriate GraphQL endpoint URL and an API key:

An API key can be obtained from a service’s Settings menu within the Apollo Engine dashboard.

apollo service:check --key="<API_KEY>" --endpoint="http://localhost:4000/graphql"

For accuracy, it’s best to retrieve the schema from a running GraphQL server (with introspection enabled), though the CLI also reference a local file. See config options for more information.

After analyzing the changes against current usage metrics, Apollo will identify three categories of changes and report them to the developer on the command line or within a GitHub pull-request:

  1. Failure: Either the schema is invalid or the changes will break current clients.
  2. Warning: There are potential problems that may come from this change, but no clients are immediately impacted.
  3. Notice: This change is safe and will not break current clients.

The more performance metrics that Apollo has, the better the report of these changes will become.

GitHub Integration

GitHub Status View

Schema validation is best used when integrated in a team’s development workflow. To make this easy, Apollo integrates with GitHub to provide status checks on pull requests when schema changes are proposed. To enable schema validation in GitHub, follow these steps:

Install GitHub application

Go to https://github.com/apps/apollo-engine and click the Configure button to install the Apollo Engine integration on the appropriate GitHub profile or organization.

Run validation on each commit

By enabling schema validation in a continuous integration workflow (e.g. CircleCI, etc.), validation can be performed automatically and potential problems can be displayed directly on a pull-request’s status checks — providing feedback to developers where they can appreciate it the most.

To run the validation command, the GraphQL server must have introspection enabled and run the apollo service:check command. An example of what this could look like is shown below with a CircleCI config:

version: 2

jobs:
  build:
    docker:
      - image: circleci/node:8

    steps:
      - checkout

      - run: npm install
      # CircleCI needs global installs to be sudo
      - run: sudo npm install --global apollo

      # Start the GraphQL server.  If a different command is used to
      # start the server, use it in place of `npm start` here.
      - run:
          name: Starting server
          command: npm start
          background: true

      # make sure the server has enough time to start up before running
      # commands against it
      - run: sleep 5

      # This will authenticate using the `ENGINE_API_KEY` environment
      # variable. If the GraphQL server is available elsewhere than
      # http://localhost:4000/graphql, set it with `--endpoint=<URL>`.
      - run: apollo service:check

      # When running on the 'master' branch, publish the latest version
      # of the schema to Apollo Engine.
      - run: |
          if [ "${CIRCLE_BRANCH}" == "master" ]; then
            apollo service:push
          fi
Edit on GitHub
// search box