9. Operation check errors
10m

Overview

We've introduced schema checks and discussed the important role they play in the management of your graph. In addition to composition checks (which ensure that changes to a subgraph can be successfully composed with other subgraphs), we have another tool to add to our toolbelt: operation checks.

Operation checks compare schema changes against client operations made in the past, to ensure that we're not introducing any breaking changes for clients consuming data from our graph.

In this lesson, we will:

  • Learn how to navigate operation checks in Studio
  • Learn how to override flagged changes in Studio
  • Learn how to re-run checks in Studio

Updating the subgraph

Not only has the Airlock team been working on new features, they've also been keeping an eye on the health of their graph and how clients have been using it. They recently discovered that both the mobile and web clients have been using the schema's featuredListings field and implementing additional logic using the results they receive.

The featuredListings field returns a list of Listing objects. More specifically, it returns exactly eight packages. However, the web client's layout works best when only displaying six of these packages. Meanwhile, the mobile client finds that displaying only three works best for their audience. So currently, both clients are fetching more packages than they need.

We don't want the clients to implement that extra logic by themselves, so let's help them fix that. Let's start back at the beginning of our CI/CD workflow.

👩🏽‍🏫 Lisa from the Listings team has made the listings subgraph schema much more flexible for client queries by adding a new argument to the featuredListings field.

Adding a new argument

Let's look at the listings subgraph. To customize the number of featured listings that can be displayed in the UI, Lisa has added a new non-nullable argument, limit, to include to the featuredListings field.

subgraph-listings/schema.graphql
"A curated array of listing packages to feature on the homepage"
featuredListings(limit: Int!): [Listing!]!

They also updated the resolvers and data sources accordingly to implement this new schema change.

subgraph-listings/resolvers.js
featuredListings: async (_, {limit}, {dataSources}) => {
const featuredListings = await dataSources.listingsAPI.getFeaturedListings(
limit
);
return featuredListings;
};

With the code changes ready to go, let's execute the next step in our workflow: running a schema check locally.

Diagram of CI workflow, with the second step local schema checks up next. See detailed image description below.

Running the rover subgraph check command results in no composition errors, but now operation checks have failed for some of the changes! Let's dig into the error:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Compared 1 schema changes against 16 operations
┌────────┬────────────────────┬─────────────────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼────────────────────┼─────────────────────────────────────────────────────────────────┤
│ FAIL │ REQUIRED_ARG_ADDED │ field `Query.featuredListings`: required argument `limit` added │
└────────┴────────────────────┴─────────────────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/{URL}
error[E030]: This operation check has encountered 1 schema change that would break operations from existing client traffic.
The changes in the schema you proposed are incompatible with graph airlock-managed-fed@staging. See https://www.apollographql.com/docs/studio/schema-checks/ for more information on resolving operation check errors.

Historically, clients have queried the featuredListings field without the limit argument. If the Listings team pushes this update to the subgraph without giving downstream clients a chance to update their queries, they could be introducing a breaking change.

Operation checks assess whether the proposed changes to the subgraph will affect any GraphQL operations that have been executed against the schema within a certain timeframe. In our team's case, the query for featuredListings runs many times per day, whenever Airlock receives more visitors interested in booking a trip.

Note: By default, Studio is configured to run checks against operations that have been executed within the last seven days. To learn how to customize this time range (along with other configuration settings), see the Apollo docs on schema check configurations.

Because Airlock is configured to use managed federation, this analytics data about operation usage is automatically collected and stored for our teams to review in Studio. Information about how many times an operation is called by clients provides useful insight into when parts of the schema can be safely changed or even removed.

We can see the data for the GetFeaturedListings operation below:

Screenshot of the Operations page on Studio

To fix the error introduced by adding the new limit argument, Lisa from the Listings team modifies the argument to be nullable, and sets 8 as the default value.

subgraph-listings/schema.graphql
"A curated array of listings to feature on the homepage"
featuredListings(limit: Int = 8): [Listing!]!

By setting the argument as nullable, no breaking changes are introduced and existing client queries work as usual, retrieving eight listings to display as the default.

After running another local schema check, we see the results below:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Check Result:
Compared 1 schema changes against 16 operations
┌────────┬────────────────────┬─────────────────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼────────────────────┼─────────────────────────────────────────────────────────────────┤
│ PASS │ OPTIONAL_ARG_ADDED │ field `Query.featuredListings`: optional argument `limit` added │
└────────┴────────────────────┴─────────────────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/3a566c0d-b46f-49f9-b5b5-3a6bde6560f2?variant=staging

All good to go! Lisa can run through the whole CI/CD process again to get the change live in production. When this change is live, clients can then work to modify their queries to include the new limit argument. The web client can set it to 6 and the mobile client can set it to 3. No more extra logic needed on the client-side!

We're not done with Airlock graph maintenance yet! Lisa has discovered another field in the listings subgraph that needs updating. In this case, the field should be removed, because no one seems to be using it.

Removing a field

An old field from an abandoned project to re-invent the homepage is still present in the schema: the Listing.photoInHexagonShape field.

An old homepage mock-up shows how this field was intended to be used.

Mockup of homepage with photos in hexagon shape

To keep their schema lean and maintainable, Lisa has opted to remove the Listing.photoInHexagonShape field entirely.

When we run schema checks locally, the changes result in an operation error. We can see it in the terminal:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Compared 1 schema changes against 17 operations
┌────────┬───────────────┬─────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼───────────────┼─────────────────────────────────────────────────────┤
│ FAIL │ FIELD_REMOVED │ type `Listing`: field `photoInHexagonShape` removed │
└────────┴───────────────┴─────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/35201113-cdbe-4a79-847b-856acd362bbf?variant=staging
error[E030]: This operation check has encountered 1 schema change that would break operations from existing client traffic.
The changes in the schema you proposed are incompatible with graph airlock-managed-fed@staging. See https://www.apollographql.com/docs/studio/schema-checks/ for more information on resolving operation check errors.

We can also check these results in Studio under the Checks page, or by following the link provided in the terminal output. Based on historical operations executed against our graph, we see the warning that removing this field might cause a breaking change for downstream clients.

https://studio.apollographql.com
Screenshot of the detail for operation check in Studio

To address this operation check error, Lisa first needs to make sure that no one is relying on this field before they remove it. To do so, they'll first mark the field in the schema file with the @deprecated directive and provide a reason (which any downstream clients consuming the field can see). By communicating this deprecated status to clients prior to removing the field, it gives everyone a chance to update their queries and avoid breaking changes.

subgraph-listings/listings.graphql
photoInHexagonShape: String @deprecated(reason: "Use photo instead.")

After running a local schema check, we'll see no errors in the terminal:

Checking the proposed schema for subgraph listings against airlock-managed-fed@staging
Check Result:
Compared 1 schema changes against 17 operations
┌────────┬──────────────────┬────────────────────────────────────────────────────────┐
│ Change │ Code │ Description │
├────────┼──────────────────┼────────────────────────────────────────────────────────┤
│ PASS │ FIELD_DEPRECATED │ type `Listing`: field `photoInHexagonShape` deprecated │
└────────┴──────────────────┴────────────────────────────────────────────────────────┘
View full details at https://studio.apollographql.com/graph/airlock-managed-fed/operationsCheck/9e2333a4-82bd-4ceb-a554-ca93e52ff35f?variant=staging

Note: We covered field deprecation in more detail in Lift-off V, so check it out if you need a refresher.

We can see that the field has been deprecated in Studio:

https://studio.apollographql.com
Screenshot of the deprecated field in Studio

After some time has passed, we'll have confidence that any downstream clients that historically queried for the Listing.photoInHexagonShape field have since removed it from their queries.

Lisa can now completely remove the Listing.photoInHexagonShape field, along with its @deprecated directive. We'll go through the CI/CD process again, running a local schema check and investigating the operation check errors on Studio. Knowing that this change is safe, we can override the resulting operation error.

Marking changes as safe

Within the error view in Studio, we can select the Override dropdown and click Mark changes as safe.

https://studio.apollographql.com
Screenshot of the check, marking changes as safe

This tells the schema checks process that this exact change can be safely ignored in future operation checks, but it does not give the operation itself the green light for any changes in the future!

For instance, in the above example, we saw that GetFeaturedListings was the name of the operation flagged with our breaking change. This operation queries for all of the Listing objects in Airlock, and is therefore concerned with the change we've just made to the Listing type. Even though we've approved and overridden the error that resulted from removing the Listing.photoInHexagonShape field, any new breaking changes introduced to the GetFeaturedListings operation will still be flagged in the future.

Similarly, if the Listings team makes another breaking change (like changing the return type for a field on Listing, or removing another of its fields entirely), they will encounter a new operation check failure.

Note: Overrides are set on an operation-by-operation basis. In addition to marking a specific change as safe, we can also ignore breaking changes to an entire operation. For example, we could tell Studio to always ignore errors that result from making changes that affect the GetFeaturedListings operation. This option is useful when the scope of an operation is limited to clients or client versions you don't actively support. Otherwise, it is best practice to fix or approve flagged changes to an operation one at a time.

With the override in place, we can rerun the check right here in Studio, by clicking Rerun check in the Checks page.

https://studio.apollographql.com
Screenshot of the Checks page, showing the Rerun check button

This time, the operation check will use the new configurations we've set on this operation, and our removal of the Listing.photoInHexagonShape field will no longer raise the alarm.

https://studio.apollographql.com
Screenshot of the Checks page, checks passing

We can now run the same CI/CD process we detailed in the previous lesson to bring these schema changes into staging, then production.

Practice

In which of the following scenarios might you want to override an operation check failure?

Key takeaways

  • Operation checks consider historical operations against the schema to ensure updates to a subgraph do not introduce breaking changes for clients.
  • Errors that occur during an operation check can be overridden in Apollo Studio.
  • Operation check overrides can be set on an operation-by-operation basis, enabling you to mark specific changes as approved or ignore an operation entirely.

Up next

We've introduced composition and operation checks to feel more confident in our changes to Airlock's schema. With these steps baked into the workflow, we've ensured that Airlock's developers can ship their changes without accidentally breaking something for another team or client.

In the next lesson, we'll see how we can enhance our insight into the graph by introducing usage reporting.

Previous
Next