9. Operation check errors
10m

Overview

We've introduced and discussed the important role they play in the management of your . In addition to composition checks (which ensure that changes to a can be successfully composed with other subgraphs), we have another tool to add to our toolbelt: operation checks.

checks compare schema changes against client operations made in the past, to ensure that we're not introducing any breaking changes for clients consuming data from our .

In this lesson, we will:

  • Learn how to navigate checks in Studio
  • Learn how to override flagged changes in Studio
  • Learn how to re-run checks in Studio

Updating the subgraph

Not only has the Airlock team been working on new features, they've also been keeping an eye on the health of their and how clients have been using it. They recently discovered that both the mobile and web clients have been using the schema's featuredListings and implementing additional logic using the results they receive.

The featuredListings returns a list of Listing objects. More specifically, it returns exactly eight packages. However, the web client's layout works best when only displaying six of these packages. Meanwhile, the mobile client finds that displaying only three works best for their audience. So currently, both clients are fetching more packages than they need.

We don't want the clients to implement that extra logic by themselves, so let's help them fix that. Let's start back at the beginning of our CI/CD workflow.

👩🏽‍🏫 Lisa from the Listings team has made the listings much more flexible for client queries by adding a new to the featuredListings .

Adding a new argument

Let's look at the listings . To customize the number of featured listings that can be displayed in the UI, Lisa has added a new non-nullable , limit, to include to the featuredListings .

subgraph-listings/schema.graphql
"A curated array of listing packages to feature on the homepage"
featuredListings(limit: Int!): [Listing!]!

They also updated the and accordingly to implement this new schema change.

subgraph-listings/resolvers.js
featuredListings: async (_, { limit }, { dataSources }) => {
const featuredListings = await dataSources.listingsAPI.getFeaturedListings(
limit
);
return featuredListings;
};

With the code changes ready to go, let's execute the next step in our workflow: running a schema check locally.

Diagram of CI workflow, with the second step local schema checks up next. See detailed image description below.

Running the rover subgraph check command results in no errors, but now checks have failed for some of the changes! Let's dig into the error:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Compared 1 schema changes against 16 operations
┌────────┬────────────────────┬─────────────────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼────────────────────┼─────────────────────────────────────────────────────────────────┤
│ FAIL │ REQUIRED_ARG_ADDED │ field `Query.featuredListings`: required argument `limit` added │
└────────┴────────────────────┴─────────────────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/{URL}
error[E030]: This operation check has encountered 1 schema change that would break operations from existing client traffic.
The changes in the schema you proposed are incompatible with graph airlock-managed-fed@staging. See https://www.apollographql.com/docs/studio/schema-checks/ for more information on resolving operation check errors.

Historically, clients have queried the featuredListings without the limit . If the Listings team pushes this update to the without giving downstream clients a chance to update their queries, they could be introducing a breaking change.

checks assess whether the proposed changes to the will affect any operations that have been executed against the schema within a certain timeframe. In our team's case, the for featuredListings runs many times per day, whenever Airlock receives more visitors interested in booking a trip.

Note: By default, is configured to run checks against that have been executed within the last seven days. To learn how to customize this time range (along with other configuration settings), see the Apollo docs on schema check configurations.

Because Airlock is configured to use , this analytics data about usage is automatically collected and stored for our teams to review in Studio. Information about how many times an operation is called by clients provides useful insight into when parts of the schema can be safely changed or even removed.

We can see the data for the GetFeaturedListings below:

Screenshot of the Operations page on Studio

To fix the error introduced by adding the new limit , Lisa from the Listings team modifies the argument to be nullable, and sets 8 as the default value.

subgraph-listings/schema.graphql
"A curated array of listings to feature on the homepage"
featuredListings(limit: Int = 8): [Listing!]!

By setting the as nullable, no breaking changes are introduced and existing client queries work as usual, retrieving eight listings to display as the default.

After running another local schema check, we see the results below:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Check Result:
Compared 1 schema changes against 16 operations
┌────────┬────────────────────┬─────────────────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼────────────────────┼─────────────────────────────────────────────────────────────────┤
│ PASS │ OPTIONAL_ARG_ADDED │ field `Query.featuredListings`: optional argument `limit` added │
└────────┴────────────────────┴─────────────────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/3a566c0d-b46f-49f9-b5b5-3a6bde6560f2?variant=staging

All good to go! Lisa can run through the whole CI/CD process again to get the change live in production. When this change is live, clients can then work to modify their queries to include the new limit . The web client can set it to 6 and the mobile client can set it to 3. No more extra logic needed on the client-side!

We're not done with Airlock maintenance yet! Lisa has discovered another in the listings that needs updating. In this case, the should be removed, because no one seems to be using it.

Removing a field

An old from an abandoned project to re-invent the homepage is still present in the schema: the Listing.photoInHexagonShape .

An old homepage mock-up shows how this was intended to be used.

Mockup of homepage with photos in hexagon shape

To keep their schema lean and maintainable, Lisa has opted to remove the Listing.photoInHexagonShape entirely.

When we run locally, the changes result in an error. We can see it in the terminal:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Compared 1 schema changes against 17 operations
┌────────┬───────────────┬─────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼───────────────┼─────────────────────────────────────────────────────┤
│ FAIL │ FIELD_REMOVED │ type `Listing`: field `photoInHexagonShape` removed │
└────────┴───────────────┴─────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/35201113-cdbe-4a79-847b-856acd362bbf?variant=staging
error[E030]: This operation check has encountered 1 schema change that would break operations from existing client traffic.
The changes in the schema you proposed are incompatible with graph airlock-managed-fed@staging. See https://www.apollographql.com/docs/studio/schema-checks/ for more information on resolving operation check errors.

We can also check these results in Studio under the Checks page, or by following the link provided in the terminal output. Based on historical executed against our , we see the warning that removing this might cause a breaking change for downstream clients.

https://studio.apollographql.com
Screenshot of the detail for operation check in Studio

To address this check error, Lisa first needs to make sure that no one is relying on this before they remove it. To do so, they'll first mark the field in the schema file with the @deprecated and provide a reason (which any downstream clients consuming the can see). By communicating this deprecated status to clients prior to removing the field, it gives everyone a chance to update their queries and avoid breaking changes.

subgraph-listings/listings.graphql
photoInHexagonShape: String @deprecated(reason: "Use photo instead.")

After running a local schema check, we'll see no errors in the terminal:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Check Result:
Compared 1 schema changes against 17 operations
┌────────┬──────────────────┬────────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼──────────────────┼────────────────────────────────────────────────────────┤
│ PASS │ FIELD_DEPRECATED │ type `Listing`: field `photoInHexagonShape` deprecated │
└────────┴──────────────────┴────────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/9e2333a4-82bd-4ceb-a554-ca93e52ff35f?variant=staging

We can see that the has been deprecated in Studio:

https://studio.apollographql.com
Screenshot of the deprecated field in Studio

After some time has passed, we'll have confidence that any downstream clients that historically queried for the Listing.photoInHexagonShape have since removed it from their queries.

Lisa can now completely remove the Listing.photoInHexagonShape , along with its @deprecated . We'll go through the CI/CD process again, running a local schema check and investigating the check errors on Studio. Knowing that this change is safe, we can override the resulting operation error.

Marking changes as safe

Within the error view in Studio, we can select the Override dropdown and click Mark changes as safe.

https://studio.apollographql.com
Screenshot of the check, marking changes as safe

This tells the process that this exact change can be safely ignored in future checks, but it does not give the operation itself the green light for any changes in the future!

For instance, in the above example, we saw that GetFeaturedListings was the name of the flagged with our breaking change. This operation queries for all of the Listing objects in Airlock, and is therefore concerned with the change we've just made to the Listing type. Even though we've approved and overridden the error that resulted from removing the Listing.photoInHexagonShape , any new breaking changes introduced to the GetFeaturedListings will still be flagged in the future.

Similarly, if the Listings team makes another breaking change (like changing the return type for a on Listing, or removing another of its entirely), they will encounter a new check failure.

Note: Overrides are set on an -by-operation basis. In addition to marking a specific change as safe, we can also ignore breaking changes to an entire . For example, we could tell to always ignore errors that result from making changes that affect the GetFeaturedListings . This option is useful when the scope of an operation is limited to clients or client versions you don't actively support. Otherwise, it is best practice to fix or approve flagged changes to an operation one at a time.

With the override in place, we can rerun the check right here in Studio, by clicking Rerun check in the Checks page.

https://studio.apollographql.com
Screenshot of the Checks page, showing the Rerun check button

This time, the check will use the new configurations we've set on this operation, and our removal of the Listing.photoInHexagonShape will no longer raise the alarm.

https://studio.apollographql.com
Screenshot of the Checks page, checks passing

We can now run the same CI/CD process we detailed in the previous lesson to bring these schema changes into staging, then production.

Practice

In which of the following scenarios might you want to override an operation check failure?

Key takeaways

  • checks consider historical operations against the schema to ensure updates to a do not introduce breaking changes for clients.
  • Errors that occur during an check can be overridden in Studio.
  • check overrides can be set on an operation-by-operation basis, enabling you to mark specific changes as approved or ignore an operation entirely.

Up next

We've introduced and checks to feel more confident in our changes to Airlock's schema. With these steps baked into the workflow, we've ensured that Airlock's developers can ship their changes without accidentally breaking something for another team or client.

In the next lesson, we'll see how we can enhance our insight into the by introducing usage reporting.

Previous

Share your questions and comments about this lesson

Your feedback helps us improve! If you're stuck or confused, let us know and we'll help you out. All comments are public and must follow the Apollo Code of Conduct. Note that comments that have been resolved or addressed may be removed.

You'll need a GitHub account to post below. Don't have one? Post in our Odyssey forum instead.