Enhance customer experiences with Apollo GraphOS


<p>The Apollo Router features three customization options out of the box to adjust behavior: <a href="https://www.apollographql.com/docs/router/configuration/overview">configuration options</a>, <a href="https://www.apollographql.com/docs/router/customizations/rhai">Rhai scripting</a>, and <a href="https://www.apollographql.com/docs/router/customizations/coprocessor">external coprocessing</a>. We refer to these as extensibility options.</p>



<p>Extensibility of the Router is one of its primary strengths and is a critical feature for many organizations large and small. Understanding the performance implications of these options is important as the Router is designed for performance and serves as the entry point to your supergraph. We’re excited to share results with you covering these extensibility options, as well as a public GitHub repository where you can test them yourself.&nbsp;</p>



<p>Before diving into the methodology and detailed results, the high-level results are:&nbsp;</p>



<ul class="wp-block-list">
<li>When you’re able to choose, configuration options perform better than Rhai scripts, and Rhai scripts perform better than external coprocessing.&nbsp;</li>



<li>When you must use an external coprocessor for more complex logic, we see about 350μs overhead.&nbsp;</li>



<li>The choice of programming language, complexity of task, network proximity, type of host machine, and other factors will have varying impacts on the real-world performance of your Router and coprocessor, so ensure you consider the best tool for the job, so to speak, and keep the coprocessor as close as possible to the Router. Ideally this would be within the same Pod (if using Kubernetes) or similar for other orchestration systems.&nbsp;</li>
</ul>



<p><a href="https://github.com/apollosolutions/router-extensibility-load-testing">The GitHub repository</a> includes more information about the results, including specific numbers, as well as the ability to run these tests yourself.&nbsp;</p>



<h3 class="wp-block-heading"><a href="#methodology" aria-hidden="true" class="aal_anchor" id="methodology"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Methodology</strong></h3>



<p>As with any test, determining how to best isolate the variable you want is critical to ensure you’re not introducing other factors into the results. For the tests we ran, we wanted to isolate the overhead for each extensibility option, not the performance impact of each test we ran.</p>



<p>To help isolate overhead latency, we ensured that we pre-warmed the query plan cache to remove the initial overhead of query planning to the first request (which may have skewed results by adding a request with a higher maximum latency value). We do this by running curl commands after the Router restart with both of the test queries.&nbsp;</p>



<p>We also limited the resources for each running Docker image to use only 1 CPU core and 1GB of RAM as a way to ensure a consistent result from run to run. While this may have impacted some programming language implementations, we wanted to give a level playing field for each test.&nbsp;</p>



<p>With that in mind, we designed three different tests for the extensibility options. This allowed us to show various overheads, as well as showing the impact of not only the complexity, but potential impact of the coprocessor stages.&nbsp;</p>



<ul class="wp-block-list">
<li>Setting a static subgraph header, which is supported by all three extensibility options. This was the simplest example available, and helped test the overhead of a SubgraphRequest coprocessor stage. Since this logic is run per-subgraph on both the coprocessor and in Rhai, we wanted to understand the overhead of multiple invocations.&nbsp;</li>



<li>Setting 10 GUID headers on a response, which is supported by only Rhai and external coprocessing. This test was designed to have slightly more complex logic, while also showing something that wasn’t possible with a basic configuration.&nbsp;</li>



<li>Lastly, we tested setting the client awareness headers using claims within a JSON Web Token (JWT) which can only be done in a coprocessor as of writing. This was the most complex logic we tested, and was a way to show how complexity could affect results as well as being an example of something only possible with external coprocessing.&nbsp;</li>
</ul>



<p>Each of these tests were run against a baseline Router (that is, a basic Router without any configured extensibility options), and a Router with the respective extensibility options configured with coprocessors being implemented in Node.js, Python, Go, Java, and C#. Between each run, we restart the Router and pre-warm the query plan cache.&nbsp;</p>



<p>For every test, we utilized<a href="https://github.com/apollosolutions/router-extensibility-load-testing/tree/main/loadtester"> a custom version</a> of<a href="https://github.com/tsenart/vegeta"> Vegeta</a>, a Go-based load testing utility. Our version handles GraphQL errors within the payload and treats them as non-2XX results for the purposes of reporting failures appropriately. Each run was set to 100 requests per second for 60 seconds to adequately load-test the system and provide enough data points for reference.&nbsp;</p>



<h3 class="wp-block-heading"><a href="#results" aria-hidden="true" class="aal_anchor" id="results"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Results</strong></h3>



<h4 class="wp-block-heading"><a href="#overview" aria-hidden="true" class="aal_anchor" id="overview"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Overview</strong></h4>



<p>Diving into the results, we saw that most workloads benefit by using configuration options over Rhai scripts, and Rhai scripts are better than external coprocessing in terms of overhead.&nbsp;</p>



<p>Rhai performance was excellent across the board, with latency overheads in p95s being around 100μs. This overhead makes it an excellent choice for most customization needs, as it tends to cover most use-cases that can’t traditionally be done with the basic configuration options.&nbsp;</p>



<p>External coprocessing overhead was still excellent at 350μs overhead per stage average across all tests and languages, so for those that need more complex logic (such as the client awareness test showed), it can still be an excellent fit with great performance. With that in mind, there are additional elements here that are worth diving into.&nbsp;</p>



<ul class="wp-block-list">
<li>Your runtime/programming language matters, and will affect results depending on the type of customizations you need.&nbsp;</li>
</ul>



<p>Each language has its strengths and weaknesses, so ensure you pick the right tool for the job when deciding what to use for your coprocessor.</p>



<ul class="wp-block-list">
<li>Even more important than runtime, your networking setup is critical to ensuring optimal performance.&nbsp;</li>
</ul>



<p>Since external coprocessing communicates over HTTP, reducing the latency between your Apollo Router and external coprocessor will ensure optimal performance and reduce overhead. If you are using Kubernetes, for example, running the coprocessor as a container alongside the Router in the same Pod can ensure network proximity.&nbsp;</p>



<ul class="wp-block-list">
<li>Ensure you need to use SubgraphRequest/Response stages before using them</li>
</ul>



<p>The coprocessor overhead is per stage, so if you run a query that hits 5 subgraphs, you’ll end up with 5 invocations of the SubgraphRequest/Response stages for a coprocessor. This overhead can add up quickly, so it is advisable to validate your need to use this stage before utilizing it.&nbsp;</p>



<h4 class="wp-block-heading"><a href="#per-test-results" aria-hidden="true" class="aal_anchor" id="per-test-results"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Per-Test Results</strong></h4>



<p>Looking at each test individually, we can start to make sense of some of the above information, as well as see the percentage differences from the baseline.</p>



<p>The numbers used were from the most recent test, running on Windows using WSL2, but your latency numbers may vary depending on resources available, CPU, OS, and more. The takeaways themselves should remain the same.</p>


<div class="wp-block-image is-style-default">
<figure class="aligncenter size-full"><img loading="lazy" decoding="async" width="600" height="371" src="https://wp.apollographql.com/wp-content/uploads/2023/07/Static-Subgraph-Header-Latency.png" alt="Chart showing the differences between the extensibility options." class="wp-image-21837" srcset="https://wp.apollographql.com/wp-content/uploads/2023/07/Static-Subgraph-Header-Latency.png 600w, https://wp.apollographql.com/wp-content/uploads/2023/07/Static-Subgraph-Header-Latency-300x186.png 300w" sizes="auto, (max-width: 600px) 100vw, 600px" /><figcaption class="wp-element-caption">Results for setting a static subgraph header</figcaption></figure></div>


<p>For the static subgraph header test, we added a &#8220;coprocessor&#8221; header with the value of the type of extensibility (e.g. rhai for Rhai scripts). This leveraged the SubgraphRequest stage for a coprocessor, and we aimed to test the overhead of making multiple requests. For this test, we averaged 1.5 subgraphs per request.</p>



<p>We can start to see that we are running into minute variance issues. In the above chart, we show that both Python and the config option are faster than the baseline- but the raw data shows that they are only 40μs and 10μs difference, respectively. Regardless, the actual overhead was measured around 330μs averaged across all coprocessor types. For specific details on each option, see the<a href="https://github.com/apollosolutions/router-extensibility-load-testing/tree/main#static-subgraph-header"> full results in the GitHub repository</a>.</p>


<div class="wp-block-image">
<figure class="aligncenter size-full"><img loading="lazy" decoding="async" width="600" height="371" src="https://wp.apollographql.com/wp-content/uploads/2023/07/10-GUID-Response-Headers-Latency.png" alt="" class="wp-image-21839" srcset="https://wp.apollographql.com/wp-content/uploads/2023/07/10-GUID-Response-Headers-Latency.png 600w, https://wp.apollographql.com/wp-content/uploads/2023/07/10-GUID-Response-Headers-Latency-300x186.png 300w" sizes="auto, (max-width: 600px) 100vw, 600px" /><figcaption class="wp-element-caption">Results for adding 10 GUIDs to a header in the response</figcaption></figure></div>


<p>In the GUID response headers test, we wanted to add a level of logic that the configuration couldn&#8217;t provide, as well as test the impact of a very basic looping logic to add dynamically generated GUIDs to the Router&#8217;s response. To do so, we leveraged the RouterResponse stage for a coprocessor.</p>



<p>Breaking down the numbers, we see clearly that Rhai is far more performant- only 11% higher p95 latency than baseline, or only 10μs in our tests versus the minimum 17% change when using a Go-based coprocessor, and the others being higher.</p>


<div class="wp-block-image">
<figure class="aligncenter size-full"><img loading="lazy" decoding="async" width="600" height="371" src="https://wp.apollographql.com/wp-content/uploads/2023/07/Client-Awareness-Headers-from-JWT-Latency.png" alt="" class="wp-image-21835" srcset="https://wp.apollographql.com/wp-content/uploads/2023/07/Client-Awareness-Headers-from-JWT-Latency.png 600w, https://wp.apollographql.com/wp-content/uploads/2023/07/Client-Awareness-Headers-from-JWT-Latency-300x186.png 300w" sizes="auto, (max-width: 600px) 100vw, 600px" /><figcaption class="wp-element-caption">Results for identifying clients based on JWT claims</figcaption></figure></div>


<p>For our last test, we wanted to test something logically complex as well as something only a coprocessor could do. This test decoded a JWT using an HMAC256 secret, and then used the claims to populate the apollographql-client-name and apollographql-client-version headers<a href="https://www.apollographql.com/docs/router/managed-federation/client-awareness/"> to enable client awareness</a> for visibility in Apollo Studio.</p>



<p>This test shows how, while the HTTP overhead itself is minimal, the actual logic is far more important. Introducing complex logic can affect performance in a meaningful way, so using other extensibility options whenever possible will help reduce the overall overhead.</p>



<h3 class="wp-block-heading"><a href="#conclusion" aria-hidden="true" class="aal_anchor" id="conclusion"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Conclusion</strong></h3>



<ol class="wp-block-list">
<li>Prefer Router configuration options over other customization options whenever possible.</li>



<li>External coprocessors add, on average, only 350μs of overhead plus the overhead of your logic. This number can change depending on other factors, identified above.</li>



<li>Keep coprocessors as close as possible to the Router to ensure the overhead doesn&#8217;t become too burdensome.</li>
</ol>



<p>View the full results and implementation on the<a href="https://github.com/apollosolutions/router-extensibility-load-testing"> public GitHub page</a>.</p>


Text that reads "Performance impacts of Apollo Router customizations" with an astronaut to the right of the text.

Performance impacts of Apollo Router customizations


<p>A healthy graph is never still — new features appear, old ones are pruned, and regretful decisions are repaired (“at last, my schema is perfect” &#8211; said no one ever).</p>



<p>But planning for change requires graph maintainers to tread carefully &#8211; the more you know about how your schema is being used, the more confidently and quickly you can move. With the new Field Insights page in Apollo Studio, safe and rapid innovation of your graph just got a whole lot easier.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="600" height="450" src="https://wp.apollographql.com/wp-content/uploads/2022/09/image-3.png" alt="My schema is perfect - said no one ever." class="wp-image-15888" srcset="https://wp.apollographql.com/wp-content/uploads/2022/09/image-3.png 600w, https://wp.apollographql.com/wp-content/uploads/2022/09/image-3-300x225.png 300w" sizes="auto, (max-width: 600px) 100vw, 600px" /></figure>



<h2 class="wp-block-heading"><a href="#why-measure-fields" aria-hidden="true" class="aal_anchor" id="why-measure-fields"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Why measure fields?</h2>



<h3 class="wp-block-heading"><a href="#fields-as-features" aria-hidden="true" class="aal_anchor" id="fields-as-features"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Fields as <em>features</em></h3>



<p>Fields are the last stop for querying data from the graph. They represent the atoms of information that are exposed to the outside world. They are also functions &#8211; when used in a mutation, they are the entry points that trigger actions in the graph. Every field is literally a “feature” of the graph, and if we want to measure the usage or health of our graph’s features, the field is a great place to start. Maintaining a graph comes with a mindset of continuous curation, and knowing where to focus our improvements is key.</p>



<h3 class="wp-block-heading"><a href="#fields-as-relationships" aria-hidden="true" class="aal_anchor" id="fields-as-relationships"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Fields as <em>relationships</em></h3>



<p>Crucially, fields also model the relationships that bind objects together. This is what makes GraphQL uniquely powerful — queries can move freely about the graph, jumping from one entity to the next and fetching as much or as little data as they need along the way. But if one of these links breaks down, entire subtrees can go dark. This is of special concern in a federated context where fields provide the bridge between the business domains in your API.</p>



<h2 class="wp-block-heading"><a href="#stewardship-through-understanding" aria-hidden="true" class="aal_anchor" id="stewardship-through-understanding"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Stewardship through understanding</h2>



<p>Viewing usage data through the lens of your fields unlocks some essential graph health workflows. With field insights, we are bringing these capabilities into focus for your graph:</p>



<ol class="wp-block-list"><li><strong>Understand usage and adoption patterns.</strong> Client usage breakdowns make it easy to see how client applications and operations are using the field, and field usage over time tells you whether field usage is ramping up or down.</li><li><strong>Identify impact on user-facing apps.</strong> Understand earlier in the schema lifecycle the changes that are likely to impact end-users and other team members based on observed usage.</li><li><strong>Guide change workflows</strong>. Confidently deprecate and remove outdated or unused fields. See the first and last time a field was used, then verify changes with <a href="https://www.apollographql.com/docs/studio/schema-checks">schema checks</a> at release time.</li><li><strong>Assess field-level health.</strong> Per-field error rates and latency histograms help you pinpoint health issues in your graph to target optimization work and validate improvements.</li><li><strong>Understand federation dependencies.</strong> Discover which subgraphs participate in resolving a field as teams work together to build the graph. (Hint: use the <code>@contact</code> directive in your subgraphs to let teams know how to get in touch with one another!)</li></ol>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="584" src="https://wp.apollographql.com/wp-content/uploads/2022/09/image-4-1024x584.png" alt="" class="wp-image-15890" srcset="https://wp.apollographql.com/wp-content/uploads/2022/09/image-4-1024x584.png 1024w, https://wp.apollographql.com/wp-content/uploads/2022/09/image-4-300x171.png 300w, https://wp.apollographql.com/wp-content/uploads/2022/09/image-4-768x438.png 768w, https://wp.apollographql.com/wp-content/uploads/2022/09/image-4-1200x684.png 1200w, https://wp.apollographql.com/wp-content/uploads/2022/09/image-4.png 1371w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /><figcaption>New field-level insights for your graph.</figcaption></figure>



<p>The amazing flexibility of GraphQL has been game-changing for app developers, but it does present an observability challenge. With operations of all shapes and sizes, how can we best understand which parts of the graph are being used, how much, and by whom? What parts of the schema are unhealthy or causing performance bottlenecks for the graph? With the new Field Insights page, you’ll have a powerful new tool to answer these kinds of questions.</p>



<p>Field Insights are available to all organizations, with longer data retention offered on paid Apollo plans. As always, we’d <a href="https://docs.google.com/forms/d/1lC-7aWNe4pnxcAPTB-W4-N8jZoyD_xQXzVByD6HOZpc/edit">love your feedback</a> on how we could further improve your graph workflows.</p>



<p>⚠️ <em>Note that field latency and execution stats require the use of field-level instrumentation in your server or subgraph &#8211; for details on how to configure your server, see </em><a href="https://www.apollographql.com/docs/studio/metrics/field-usage"><em>our documentation.</em></a></p>


Continuously improve your graph with field-level usage and health metrics from Apollo Studio.


<p>With the release of Apollo Server 3.6, we introduced new features and configuration to improve usage reporting to Apollo Studio. They are companion features to Inline Tracing for federated GraphQL APIs and OpenTelemetry support. With operation shape-based usage reporting, you get a clearer picture of how your graph is used with a much smaller impact on overall performance.</p>



<h2 class="wp-block-heading"><a href="#a-brief-history-of-tracing-and-usage-reporting" aria-hidden="true" class="aal_anchor" id="a-brief-history-of-tracing-and-usage-reporting"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>A brief history of tracing and usage reporting</h2>



<p>One of my favorite benefits of GraphQL is the fine-grained usage information you get when clients must specify the fields they need in the query language. When I started using Apollo Studio, one of the features I was most excited about was access to all this data:</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="647" height="727" src="https://wp.apollographql.com/wp-content/uploads/2022/02/screenshots2.png" alt="" class="wp-image-12222" srcset="https://wp.apollographql.com/wp-content/uploads/2022/02/screenshots2.png 647w, https://wp.apollographql.com/wp-content/uploads/2022/02/screenshots2-267x300.png 267w" sizes="auto, (max-width: 647px) 100vw, 647px" /><figcaption>Screenshots of the Fields page showing field usage and the Operations page showing resolver traces</figcaption></figure>



<p>At the time, both of these views were powered by <em>tracing</em>.</p>



<p>Tracing is a means of collecting detailed, low-level information for investigating and diagnosing issues and performance. In a GraphQL service, we usually refer to “resolver tracing”, which tracks the time spent and the errors that occur with in field resolver functions.</p>



<p>In a monolithic GraphQL API, resolver tracing is pretty simple: instrument each resolver function call, then package it up and ship it to Studio.</p>



<p>It’s harder in a federated graph. Apollo Gateway doesn’t execute resolvers. Instead, that happens within subgraph services on other machines. So Apollo invented “Inline Tracing”, where subgraphs trace their requests and send the data back to the gateway <em>inline</em> — as extensions on the response. The gateway then combines traces from all subgraph responses into a complete trace and sends it to Studio.</p>



<pre class="wp-block-prismatic-blocks"><code class="language-json">{
  &quot;data&quot;: {
    &quot;topProducts&quot;: [
      {
        &quot;title&quot;: &quot;Dune&quot;,
        &quot;author&quot;: &quot;Frank Herbert&quot;
      }
    ]
  },
  &quot;extensions&quot;: {
    &quot;ftv1&quot;: &quot;676f6f64206a6f6220696e7665737469676174696e67207468697320737472696e672120636f6d6520776f726b20666f72207573212068747470733a2f2f7777772e61706f6c6c6f6772617068716c2e636f6d2f63617265657273&quot;
  }
}</code></pre>



<p><em>The trace data is a base64-encoded protocol buffer message.</em></p>



<p>I was so excited to get this data that I ended up contributing the first ports of <em><code>ftv1</code></em> (”federated tracing version one”) to both <a rel="noreferrer noopener" href="https://github.com/Gusto/apollo-federation-ruby#tracing" target="_blank">graphql-ruby</a> and <a rel="noreferrer noopener" href="https://github.com/apollographql/federation-jvm#federated-tracing" target="_blank">federation-jvm</a> so I could see federated traces in the graphs I worked on at Square.</p>



<h2 class="wp-block-heading"><a href="#the-challenges-of-inline-tracing" aria-hidden="true" class="aal_anchor" id="the-challenges-of-inline-tracing"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>The Challenges of Inline Tracing</h2>



<p>Apollo Federation allows you to build subgraphs in <a href="https://www.apollographql.com/docs/federation/other-servers/">many languages and platforms</a>, not just Ruby, Java, and Node. However, not all of the subgraph implementations support <code>ftv1</code> tracing yet, meaning that your choice of language could affect the availability of traces and field usage data.</p>



<p>On top of that, inline trace data adds overhead to subgraph requests. You may see a performance impact on the individual subgraph responses to the gateway.</p>



<p>Other systems like OpenTelemetry don’t use inline tracing. Instead, they send data out-of-band to external systems, which collate tracing data from multiple services.</p>



<p>Why doesn’t Inline Tracing work like OpenTelemetry? Because it’s a lot more work to set up a system of collector sidecars and exporter services! Inline Tracing is so easy to set up that it’s on by default—you can’t say that about most tracing systems!</p>



<p>However, the performance impact of inline tracing is unacceptable for some. This is especially true for GraphQL developers supporting very large requests. The size of the trace scales linearly with the size of the request (making the total response size grow super-linearly!)</p>



<p>Prior to Apollo Server 3.6, you had only two options to mitigate the performance impact.</p>



<ul class="wp-block-list"><li>Disabling inline tracing entirely. This avoids extra work during execution and avoids bloating the response payload. Unfortunately, it also removed both the trace data in Operations tab and the field usage data in the Fields page.</li><li>Sampling using <code>includeRequest</code>. This would affect <em>all</em> statistics reported to Studio, so you’ll lose accurate request counts, client usage data, and more.</li></ul>



<p>Given these downsides, we wanted to enhance Apollo Server’s performance while collecting as much useful data as we could. To achieve this goal, we built a new feature that uses static analysis for usage reporting.</p>



<h2 class="wp-block-heading"><a href="#usage-reporting-with-static-operation-analysis" aria-hidden="true" class="aal_anchor" id="usage-reporting-with-static-operation-analysis"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Usage reporting with static operation analysis</h2>



<p>With GraphQL, it’s critical to know which fields are referenced in operations, even if they’re never resolved. This data powers the breaking change detection in Apollo Studio’s <a href="https://www.apollographql.com/docs/studio/schema-checks/">Checks</a> feature and is the inspiration for this new usage reporting based on the static analysis of “operation shapes”.</p>



<p>Operation shape-based reporting doesn’t rely on tracing. Instead, it statically analyzes the operation document to count the types and fields. It’s much cheaper to run this algorithm over the document than it is to trace every single field resolver. And most importantly, we run this algorithm only in the gateway, avoiding the need to send tracing data between servers. This means we support detailed field usage information for all subgraph servers regardless of language or framework.</p>



<p>Usage reporting based on operation shape analysis is available in Apollo Server as of version 3.6, <a href="https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md#v360">released in December 2021</a>. The new field usage metrics improvements apply to monolithic graphs as well as federated graphs that use Apollo Gateway.</p>



<h2 class="wp-block-heading"><a href="#the-nuances-of-operation-shape-based-analysis" aria-hidden="true" class="aal_anchor" id="the-nuances-of-operation-shape-based-analysis"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>The nuances of operation shape-based analysis</h2>



<p>Consider this operation:</p>



<pre class="wp-block-prismatic-blocks"><code class="language-graphql">query TopProducts {
  topProducts {
    title
    ... on Book {
      author
    }
    ... on Album {
      artist
    }
  }
}</code></pre>



<p>When you statically analyze this operation, you know that these fields are <em>referenced</em>:</p>



<ul class="wp-block-list"><li><code>Query.topProducts</code></li><li><code>Product.title</code> (the interface field!)</li><li><code>Book.author</code></li><li><code>Album.artist</code></li></ul>



<p>Notably, it’s not possible to know if <code>Book.title</code> or <code>Album.title</code> are used!</p>



<p>However, by tracing all the resolvers, you know exactly how many times a field is resolved. When <code>topProducts</code> returns the possible types of the <code>Product</code> interface, you’ll know which items are Books and which ones are Albums. This is how you can know the number of times the <code>Book.title</code> and <code>Album.title</code> fields are <em>executed</em>.</p>



<p>The difference between “field executions” (from tracing) and “referencing operations” (from operation shape-based usage reporting) is important. We decided to make this crystal clear by showing two different metrics for each field.</p>



<figure class="wp-block-image size-large"><img decoding="async" src="https://www.apollographql.com/docs/studio/cb24a5b0244106da154eb781a08c99c1/fields-page.jpg" alt=""/><figcaption>Screenshot of the Fields page with the new &#8220;Referencing Operations&#8221; column.</figcaption></figure>



<p>The Apollo Server <a href="https://www.apollographql.com/docs/studio/metrics/field-usage/">documentation</a> goes into even more detail about the differences between “Field Execution” and “Referencing Operations”.</p>



<p>If you disable Inline Tracing, you won’t get the data that powers the “Field executions” metric in the Fields page, but you’ll still see the “Referencing operations” metric.</p>



<h2 class="wp-block-heading"><a href="#trace-sampling" aria-hidden="true" class="aal_anchor" id="trace-sampling"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Trace Sampling</h2>



<p>Instead of disabling Inline Tracing, Apollo Server 3.6 also includes a new option, <code>fieldLevelInstrumentation</code>, that makes it super easy to enable Inline Tracing for a small percentage of requests:</p>



<pre class="wp-block-prismatic-blocks"><code class="language-javascript">plugins: [
  ApolloServerPluginUsageReporting({
    fieldLevelInstrumentation: 0.01
  })
]</code></pre>



<p>With the above configuration, the gateway will enable <code>ftv1</code> tracing on subgraphs for 1% of requests. This means you’ll pay a performance penalty for a small subset of requests, but with enough traffic that 1% of requests will give you plenty of data to power the Operations tabs and help you find and diagnose issues in your resolvers.</p>



<p>Crucially, you will still get accurate metrics on field usage, overall operation latency, and error rates for 100% of requests.</p>



<p>The&nbsp;<code>includeRequest</code>&nbsp;option is still available but is no longer recommended for configuring tracing. It could still be useful if you want to exclude bot traffic from data in Studio with a configuration similar to:</p>



<pre class="wp-block-prismatic-blocks"><code class="language-javascript">ApolloServerPluginUsageReporting({
  includeRequest({ request }) {
    // set by a security scanner at the ingress
    if (request.headers[&quot;X-PROBABLY-A-BOT&quot;]) {
      return false;
    }
    return true;
  }
})</code></pre>



<h2 class="wp-block-heading"><a href="#wrapping-up" aria-hidden="true" class="aal_anchor" id="wrapping-up"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Wrapping Up</h2>



<p>Apollo Server 3.6 is a big win for GraphQL and Federation. It directly addresses the limitations of Inline Tracing while still allowing you to capture a crystal-clear picture of graph usage and performance in Apollo Studio. And when combined with OpenTelemetry (<a href="https://www.apollographql.com/blog/backend/architecture/introducing-open-telemetry-for-apollo-federation/">as discussed previously</a>), you get an even more complete picture of your API&#8217;s performance across your entire stack.</p>



<p>Personally, I’ve enjoyed learning about the differences between field references and executions from <a href="https://www.apollographql.com/docs/studio/metrics/field-usage/">the great documentation we’ve put together on Field Usage</a>. It’s given me a whole new appreciation for GraphQL and graph usage data!</p>



<h3 class="wp-block-heading"><a href="#resources" aria-hidden="true" class="aal_anchor" id="resources"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Resources</h3>



<ul class="wp-block-list"><li><a href="https://www.apollographql.com/docs/studio/metrics/field-usage/">Field Usage in Apollo Studio</a></li><li><a href="https://www.apollographql.com/docs/apollo-server/api/plugin/usage-reporting/">Usage Reporting Plugin documentation</a></li><li><a href="https://www.apollographql.com/docs/federation/metrics/">Federated Tracing documentation</a></li><li><a href="https://www.apollographql.com/docs/federation/opentelemetry/">OpenTelemetry support in Apollo Gateway documentation</a></li></ul>



<p></p>


Faster federated graphs and better usage data with Apollo Server 3.6


<p>How do you debug or optimize a request as it travels through all the layers of your stack: from the client, through the Apollo Gateway, through your subgraph services, all the way to your databases?</p>



<p>First, you need visibility into what your code is doing during that request, regardless of how that work is distributed across machines. Fortunately, this is easier than ever with a Cloud-Native Computing Foundation project called <a href="https://opentelemetry.io/">OpenTelemetry</a>!</p>



<p>As of version 0.31.1, <a href="https://www.apollographql.com/docs/federation/api/apollo-gateway">@apollo/gateway</a> natively supports OpenTelemetry as part of a complete picture of the performance and reliability of a federated GraphQL request. You can find <a href="https://www.apollographql.com/docs/federation/opentelemetry/">the documentation here</a>.</p>



<p>In this article, I’ll cover some OpenTelemetry basics, talk about the new support for OpenTelemetry in Apollo Gateway, and explain how Apollo Studio and OpenTelemetry work together to give you a complete picture of your graph.</p>



<h2 class="wp-block-heading"><a href="#what-is-opentelemetry" aria-hidden="true" class="aal_anchor" id="what-is-opentelemetry"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>What is OpenTelemetry?</h2>



<p>OpenTelemetry (or OTel) is “an observability framework for cloud-native software”. It is a collection of libraries, systems, and applications for instrumenting, collecting, and exporting information about workloads that span multiple applications.</p>



<p>A unit of work in OTel is a “span”. You can wrap a function call in a span, recording its duration and other metadata. You can also install instrumentation that automatically hooks into modules and functions, like the <a href="https://www.npmjs.com/package/@opentelemetry/instrumentation-http">HTTP instrumentation</a> for Node.js.</p>



<p>A collection of related spans is called a “trace”. In a single system, a trace is very similar to a stack trace. But what makes OpenTelemetry (and its predecessors, OpenTracing and OpenCensus) powerful is the ability to correlate spans across systems. It does this by propagating trace identifiers in remote calls. When making HTTP requests, for example, OTel instrumentation adds trace IDs as HTTP headers.</p>



<p>Once you have a trace, you can aggregate them in a “collector” and efficiently send them to your observability tools with an “exporter”.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="605" src="https://wp.apollographql.com/wp-content/uploads/2021/07/OpenTelemetryApolloFederation-1024x605.png" alt="" class="wp-image-8203" srcset="https://wp.apollographql.com/wp-content/uploads/2021/07/OpenTelemetryApolloFederation-1024x605.png 1024w, https://wp.apollographql.com/wp-content/uploads/2021/07/OpenTelemetryApolloFederation-300x177.png 300w, https://wp.apollographql.com/wp-content/uploads/2021/07/OpenTelemetryApolloFederation-768x454.png 768w, https://wp.apollographql.com/wp-content/uploads/2021/07/OpenTelemetryApolloFederation.png 1144w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /><figcaption>A diagram of a request flowing through a federated GraphQL architecture with OpenTelemetry and Apollo traces published to an OTel collector and Apollo Studio, respectively.</figcaption></figure>



<p>There’s obviously a lot more to OpenTelemetry (<a href="https://opentelemetry.io/docs/concepts/what-is-opentelemetry/">see the docs!</a>), but that covers it in the context of Apollo Federation.</p>



<h2 class="wp-block-heading"><a href="#opentelemetry-in-apollo-server-and-apollo-gateway" aria-hidden="true" class="aal_anchor" id="opentelemetry-in-apollo-server-and-apollo-gateway"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>OpenTelemetry in Apollo Server and Apollo Gateway</h2>



<p>Apollo Server has always supported OpenTelemetry via the <a href="https://www.npmjs.com/package/@opentelemetry/instrumentation-graphql">@opentelemetry/instrumentation-graphql</a> library, which instruments the underlying <a href="https://www.npmjs.com/package/graphql">graphql.js</a> library.</p>



<p>However, Apollo Gateway doesn’t use graphql.js to execute a GraphQL request; instead, it executes a “query plan”, coordinating smaller requests to your subgraph services and assembling them into a single response. The Gateway executor has a few phases:</p>



<ul class="wp-block-list"><li>Validation</li><li>Planning</li><li>Execution</li><li>Fetching</li><li>Postprocessing</li></ul>



<p>Apollo Gateway now instruments these phases as spans in an OTel trace. Combined with HTTP instrumentation, which propagates the trace ID as headers in requests to subgraphs, we can now connect the gateway query planning and execution to the actual GraphQL field resolution work in your subgraph services.</p>



<p>Check out our <a href="https://www.apollographql.com/docs/federation/opentelemetry/#1-install-required-libraries">documentation on setting up OpenTelemetry</a> in your Apollo Gateway and Apollo Server apps.</p>



<p>Even if you’re using GraphQL framework other than Apollo Server or a language other than JavaScript, you can still use OTel to instrument your application. There are OTel instrumentation libraries for many other languages and frameworks, and <a href="https://opentelemetry.io/registry/?s=graphql&amp;component=&amp;language=">GraphQL-specific OTel support</a> is small but growing.</p>



<h2 class="wp-block-heading"><a href="#opentelemetry-in-your-infrastructure" aria-hidden="true" class="aal_anchor" id="opentelemetry-in-your-infrastructure"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>OpenTelemetry in your Infrastructure</h2>



<p>Instrumenting a request is cool, but it’s not very useful on its own. You can export traces into many different systems to view, inspect, or search traces.</p>



<p>The open-source projects Zipkin and Prometheus both support OpenTelemetry. Here’s an example of a federated GraphQL request in Zipkin.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="567" src="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-1024x567.png" alt="" class="wp-image-8205" srcset="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-1024x567.png 1024w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-300x166.png 300w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-768x425.png 768w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-1536x850.png 1536w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-2048x1134.png 2048w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-1200x664.png 1200w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-23-at-9.55.49-AM-1980x1096.png 1980w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /><figcaption>A screenshot of a trace in the Zipkin web UI, showing spans for the gateway and multiple subgraphs.</figcaption></figure>



<p>The OpenTelemetry project has a number of exporters for various Application Performance Monitoring (APM) tools, including SaaS products like Datadog and Honeycomb, or cloud provider solutions like AWS X-Ray. You can find a complete list in the <a href="https://opentelemetry.io/registry/?s=&amp;component=exporter&amp;language=">OpenTelemetry Registry</a>.</p>



<h2 class="wp-block-heading"><a href="#opentelemetry-doesnt-replace-apollo-tracing" aria-hidden="true" class="aal_anchor" id="opentelemetry-doesnt-replace-apollo-tracing"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>OpenTelemetry doesn’t replace Apollo Tracing</h2>



<p>OpenTelemetry is great at recording what happened, but it doesn’t tell you much about what <em>could</em> happen. Apollo Studio uses the declarative nature of GraphQL to help you catch potential errors before they happen. Consider this GraphQL operation:</p>



<pre class="wp-block-prismatic-blocks"><code class="language-graphql">query ProductsPage($categoryId: ID!) {
  products(category: $categoryId) {
    nodes {
      sku
      price
      reviews { # could return null
        nodes {
          id
          body
          rating
          author {
            name
            photoUrl 
          }
        }
      }
    }
  }
}</code></pre>



<p>If the <code>Product.reviews</code> field returns null, then the GraphQL executor won’t run resolvers for any of the fields nested inside that selection like <code>Review.id</code>, <code>Review.body</code>, or <code>Author.name</code>. OTel instrumentation won’t record any spans, so you’ll have no data about those fields.</p>



<p>However, if you remove or change the <code>Author.name</code> field, you could break all the clients that include that field in their operations.</p>



<p>We designed <a href="https://www.apollographql.com/docs/federation/metrics/">Apollo Federated Tracing</a> to collect field usage regardless of what actually happens (or doesn’t happen!) in your services. Our tracing data powers the <a href="https://www.apollographql.com/docs/studio/schema-checks/">Schema Checks</a> feature, alerting you of potentially breaking changes and giving you confidence as you evolve your API.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="638" src="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-1024x638.png" alt="" class="wp-image-8207" srcset="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-1024x638.png 1024w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-300x187.png 300w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-768x478.png 768w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-1536x957.png 1536w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-2048x1276.png 2048w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-1200x748.png 1200w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.53.37-AM-1980x1233.png 1980w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /><figcaption>A screenshot of a failed check in Apollo Studio.</figcaption></figure>



<p>Apollo Studio also knows schema-related information like the <code>@deprecated</code> directive, so we can help you understand how clients are using deprecated fields.</p>



<figure class="wp-block-image size-large is-resized"><img loading="lazy" decoding="async" src="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM-1024x210.png" alt="" class="wp-image-8209" width="610" height="125" srcset="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM-1024x210.png 1024w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM-300x62.png 300w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM-768x158.png 768w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM-1536x316.png 1536w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM-1200x247.png 1200w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.50.03-AM.png 1732w" sizes="auto, (max-width: 610px) 100vw, 610px" /><figcaption>A screenshot of deprecation information in Apollo Studio.</figcaption></figure>



<p>It’s possible to cross-reference OTel traces and Apollo Studio traces by customizing the <a href="https://www.apollographql.com/docs/apollo-server/api/plugin/usage-reporting/">Usage Reporting plugin</a> in Apollo Server. By passing header values in trace sent to Studio that align with attributes on your OTel spans, you can jump between Studio and your OTel observability tooling to get a complete picture of a request.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="572" src="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-1024x572.png" alt="" class="wp-image-8211" srcset="https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-1024x572.png 1024w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-300x168.png 300w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-768x429.png 768w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-1536x858.png 1536w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-2048x1144.png 2048w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-1200x670.png 1200w, https://wp.apollographql.com/wp-content/uploads/2021/07/Screen-Shot-2021-07-22-at-10.52.54-AM-1980x1106.png 1980w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /><figcaption>A screenshot of a trace in Apollo Studio with metadata you can use to associate this request with an OpenTelemetry trace.</figcaption></figure>



<h2 class="wp-block-heading"><a href="#wrapping-up" aria-hidden="true" class="aal_anchor" id="wrapping-up"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Wrapping Up</h2>



<p>OpenTelemetry is a fantastic open-source project and we’re excited to participate in the growing ecosystem. Combined with Apollo Studio, OpenTelemetry gives you a clear picture of your running systems, helping you debug, optimize, and evolve your federated graphs.</p>



<p>Check out the <a href="https://opentelemetry.io/docs/">OpenTelemetry documentation</a> and our <a href="https://www.apollographql.com/docs/federation/opentelemetry/">documentation on instrumenting federated graphs</a>.</p>


Introducing Open Telemetry for Apollo Federation


<p>A common challenge that developers experience as they build products is how quickly their apps grow in complexity. For GraphQL services, your request sizes and query strings fatten over time, which in turn leads to extra weight transferred over the network. We solved this challenge by shipping&nbsp;<strong>Automatic Persisted Queries</strong>&nbsp;— a tool to that improves GraphQL network performance by reducing request size.</p>



<p>In Apollo Server 1.x, Automatic Persisted Queries and CDN caching depended on a closed-source Go component called the “Engine Proxy”. But with&nbsp;<a href="https://www.apollographql.com/docs/apollo-server/v2/whats-new.html" target="_blank" rel="noreferrer noopener">Apollo Server 2.0</a>, our goal is that it’s the only tool you need to implement a production-ready GraphQL API. So how could we take care of this issue?</p>



<p>As part of Apollo Server 2.0, we’re open sourcing all of the features of the Engine proxy!</p>



<p>Apollo Server 2.0 now has Automatic Persisted Queries (APQ) and CDN caching built right in, to greatly enhance your network performance. In the rest of the post, I’ll show you how to use them.</p>



<p>Performance will be a first-class citizen in your GraphQL services with Automatic Persisted Queries and CDN caching.</p>



<h2 class="wp-block-heading" id="a87a"><a href="#the-cost-of-graphql-queries" aria-hidden="true" class="aal_anchor" id="the-cost-of-graphql-queries"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>The cost of GraphQL queries</h2>



<p>The size of individual queries in your app can be a bottleneck for smooth performance. The larger the queries, the more time it takes to send them over the network.</p>



<p>We’ve seen GraphQL query sizes in production ranging well above 10 KB,&nbsp;<em>just for the query text</em>. This is can be a significant overhead when compared with a simple REST URL of 50–100 characters. When paired with the fact that the uplink speed from the client is typically the most bandwidth-constrained part of the chain, large queries can become bottlenecks for client performance.</p>



<p>The slower a user’s connection is, the more costly those extra bytes become, and the longer it takes for the client to transmit its request. This is exactly what&nbsp;<strong>Automatic Persisted Queries</strong>&nbsp;solves.</p>



<h2 class="wp-block-heading" id="2c49"><a href="#what-are-automatic-persisted-queries" aria-hidden="true" class="aal_anchor" id="what-are-automatic-persisted-queries"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>What are Automatic Persisted Queries?</h2>



<p>With&nbsp;<strong>Automatic Persisted Queries</strong>, the client has the ability to ask the server if it needs the full query by sharing a small, 64-byte hashed representation of it, and only sending the full query if the server hasn’t seen that request before.</p>



<h3 class="wp-block-heading" id="a031"><a href="#how-it-works" aria-hidden="true" class="aal_anchor" id="how-it-works"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>How it works</h3>



<p>Apollo Server checks if a request coming from the client contains a persisted query hash. If a hash is detected, Apollo Server looks up its registry for the corresponding query. Upon finding a match, Apollo Server expands the request with the full text of the query and executes it.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="714" height="422" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1_mMsPHtd3-9P7DQcB5B1d7g.png" alt="" class="wp-image-717" srcset="https://wp.apollographql.com/wp-content/uploads/2020/03/1_mMsPHtd3-9P7DQcB5B1d7g.png 714w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_mMsPHtd3-9P7DQcB5B1d7g-300x177.png 300w" sizes="auto, (max-width: 714px) 100vw, 714px" /><figcaption>Optimal Performance Path</figcaption></figure>



<p>On the other hand, if no query is found for a hash, the server will notify the client to resend the request with the full query so that it can store the query / hash mapping in the registry for subsequent requests to profit from.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="730" height="502" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1_1QC1Q6atKn1k7X8ZDUC9jA.png" alt="" class="wp-image-716" srcset="https://wp.apollographql.com/wp-content/uploads/2020/03/1_1QC1Q6atKn1k7X8ZDUC9jA.png 730w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_1QC1Q6atKn1k7X8ZDUC9jA-300x206.png 300w" sizes="auto, (max-width: 730px) 100vw, 730px" /><figcaption>New Performance Path</figcaption></figure>



<h3 class="wp-block-heading" id="1501"><a href="#how-to-configure-the-cache-store" aria-hidden="true" class="aal_anchor" id="how-to-configure-the-cache-store"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>How to configure the cache store</h3>



<p>By default, Apollo Server is configured to use an in-memory cache to store the query hashes. While this works great for many projects, highly-trafficked deployments or deployments where the processes are short-lived (like Amazon Lambda or Google Cloud Functions) will benefit from using a shared cache store, such as Memcached or Redis. These stores can be configured via the Apollo Server constructor:</p>



<pre class="wp-block-prismatic-blocks"><code class="language-javascript">const { MemcachedCache } = require(&#039;apollo-server-memcached&#039;);
const { ApolloServe } = require(&#039;apollo-server&#039;);

const server = new ApolloServer({
  ...
  persistedQueries: {
    cache: new MemcachedCache(
      [&#039;memcached-server-1&#039;, &#039;memcached-server-2&#039;, &#039;memcached-server-3&#039;],
      { retries: 10, retry: 10000 }, // Options
    ),
  },
});</code></pre>



<h3 class="wp-block-heading" id="a967"><a href="#client-setup" aria-hidden="true" class="aal_anchor" id="client-setup"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Client Setup</h3>



<p>Finally, you need to turn on APQ in your client to start sending hashes. You can set it up in your client app with Apollo Link, like so:</p>



<pre class="wp-block-prismatic-blocks"><code class="language-javascript">import { createPersistedQueryLink } from &quot;apollo-link-persisted-queries&quot;;
import { createHttpLink } from &quot;apollo-link-http&quot;;
import { InMemoryCache } from &quot;apollo-cache-inmemory&quot;;
import ApolloClient from &quot;apollo-client&quot;;

const link = createPersistedQueryLink().concat(createHttpLink({ uri: &quot;/graphql&quot; }));

const client = new ApolloClient({
  cache: new InMemoryCache(),
  link: link,
});</code></pre>



<p>Ready to give it a try? Check out&nbsp;<a href="https://www.apollographql.com/docs/guides/performance.html#automatic-persisted-queries" target="_blank" rel="noreferrer noopener">the documentation here</a>&nbsp;to get your app set up and running with Automatic Persisted Queries.</p>



<h2 class="wp-block-heading" id="656f"><a href="#bring-data-closer-to-clients-with-cdn-caching" aria-hidden="true" class="aal_anchor" id="bring-data-closer-to-clients-with-cdn-caching"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Bring data closer to clients with CDN caching!</h2>



<p>Another important optimization that can be really helpful for commonly-executed queries is CDN caching. Placing your server behind a CDN allows you to deliver data faster to the clients your users interact with.</p>



<p>Most GraphQL requests are large POST requests. Therefore, improving network performance with CDN integration will involve you following these steps:</p>



<ol class="wp-block-list"><li><a rel="noreferrer noopener" href="https://www.apollographql.com/docs/guides/performance.html#setup-cdn" target="_blank">Set up a CDN and configure it to send requests to Apollo Server.</a></li><li><a rel="noreferrer noopener" href="https://www.apollographql.com/docs/guides/performance.html#cache-hints" target="_blank">Add cache hints to your schema so that Apollo Server knows which fields and types are cacheable and for how long.</a></li></ol>



<pre class="wp-block-prismatic-blocks"><code class="language-graphql">type Dog @cacheControl(maxAge: 120) {
  id: String!
  breed: String!
  displayImage: String @cacheControl(maxAge: 240)
  images: [Image]
  subbreeds: [String]
}</code></pre>



<p>For example, the schema above indicates that all fields that return a&nbsp;<code>Dog</code>&nbsp;type should be cached for 120 seconds, and that the&nbsp;<code>displayImage</code>&nbsp;field itself should be cached for 240 seconds.</p>



<p><a href="https://www.apollographql.com/docs/guides/performance.html#enable-apq" target="_blank" rel="noreferrer noopener">3. Finally, enable automatic persisted queries.</a></p>



<p>Apollo Server makes it straightforward to use CDNs with GraphQL queries to cache full responses while still executing more dynamic queries as usual.</p>



<h2 class="wp-block-heading" id="1b2e"><a href="#one-more-thing" aria-hidden="true" class="aal_anchor" id="one-more-thing"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>One More thing!</h2>



<p>Over the past few months, we’ve worked very hard in making Apollo Server 2.0 your go-to API server in your GraphQL toolbox. And to help you get over your fear of missing out, we’re&nbsp;<a href="https://www.apollographql.com/docs/apollo-server/v2/whats-new.html" target="_blank" rel="noreferrer noopener">constantly updating the docs</a>&nbsp;to give you a healthy pathway in learning to consume&nbsp;<a href="https://www.apollographql.com/docs/apollo-server/v2/" target="_blank" rel="noreferrer noopener">Data Sources, File uploads, metrics handling, subscriptions and a plethora of features.</a></p>



<p>Furthermore, feel free to check out our<a href="https://www.apollographql.com/docs/" target="_blank" rel="noreferrer noopener">&nbsp;new best practices guides about topics like schema design, versioning, access control, and more.</a></p>



<hr class="wp-block-separator"/>



<p>Finally, I hope you’ll also&nbsp;<a href="https://summit.graphql.com/" target="_blank" rel="noreferrer noopener">join us at the 3rd annual GraphQL Summit on Nov 7–8 in San Francisco</a>. With over 800 attendees expected, it’s the largest GraphQL developer event in the world. Super early bird tickets are selling fast, so&nbsp;<a href="https://www.eventbrite.com/e/graphql-summit-2018-tickets-46601841362" target="_blank" rel="noreferrer noopener">register today</a>&nbsp;to reserve your spot!</p>


Performance

Performance impacts of Apollo Router customizations

Continuously improve your graph with field-level usage and health metrics from Apollo Studio.

Faster federated graphs and better usage data with Apollo Server 3.6

Introducing Open Telemetry for Apollo Federation

Automatic Persisted Queries and CDN caching with Apollo Server 2.0

Optimizing Your GraphQL Request Waterfalls

Performance

Performance impacts of Apollo Router customizations

Continuously improve your graph with field-level usage and health metrics from Apollo Studio.

Faster federated graphs and better usage data with Apollo Server 3.6

Introducing Open Telemetry for Apollo Federation

Automatic Persisted Queries and CDN caching with Apollo Server 2.0

Optimizing Your GraphQL Request Waterfalls

Stay in our orbit