Enhance customer experiences with Apollo GraphOS


<p>Apollo doesn’t just build Federation – we use it internally as well. Like many of our customers, we recently upgraded a fleet of Apollo Gateway instances to Apollo Router. This fleet ran the superschema for our &#8220;Apollo Studio&#8221; graph in multiple variants: development, staging, and production.</p>



<p>Our team recently upgraded to the Apollo Router with very positive results which we’d like to share alongside some helpful tips.</p>



<h2 class="wp-block-heading"><a href="#replacing-a-gateway-with-the-apollo-router" aria-hidden="true" class="aal_anchor" id="replacing-a-gateway-with-the-apollo-router"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Replacing a Gateway with the Apollo Router</strong></h2>



<p>The Apollo Router is Apollo&#8217;s replacement for the Gateway @apollo/gateway, and provides higher performance with lower resource consumption. It was an easy choice to replace the existing gateway fleets with new router fleets to serve the various Apollo Studio graph variants.</p>



<p>Migrating from gateway to router requires a bit of a mindset shift. The gateway is essentially a JavaScript toolkit intended to be highly customizable with all users providing their own customizations for their own particular execution environment. The Apollo Router is a product. It leverages the experience and knowledge of Apollo to provide a performant, scalable GraphQL Federation experience while still offering customization to users.</p>



<p>Apollo had deployed a highly customized gateway and migrating to the router was largely a task of mapping custom gateway functionality to standard router functionality. Most of this was driven by router configuration (header forwarding). We also wrote a Rhai script for simple header and cookie manipulation and a coprocessor that provided custom logging in a different language (TypeScript).</p>



<p>Once we had figured out how to migrate our custom gateway functionality, it was then simple to adopt the standard router container image and deploy this via a custom Helm chart which leveraged the standard router Helm Chart to do the heavy lifting. You can read more details about this in the Kubernetes section below.</p>



<p>Because the router provides a standard, configuration-driven execution environment, it is simpler to utilize standard tools such as Helm to manage deployments. This drive to standardization delivers benefits to Apollo in terms of product feature development speed and for customers, since it is much simpler to deploy a standard router than it is to develop and deploy your own custom gateway.</p>



<h2 class="wp-block-heading"><a href="#performance-comparison" aria-hidden="true" class="aal_anchor" id="performance-comparison"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Performance Comparison</strong></h2>



<p>When Apollo migrated to the Apollo Router, we were looking for improvements in resilience, scalability, responsiveness, and resource consumption. Let&#8217;s take a look at some of the details of our migration.</p>



<h3 class="wp-block-heading"><a href="#resources" aria-hidden="true" class="aal_anchor" id="resources"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Resources</strong></h3>



<p>Our router CPU consumption is approximately 60% that of the gateway. Router memory consumption is approximately 13% of the gateway. That&#8217;s a substantial operational cost-saving reduction.</p>



<h4 class="wp-block-heading"><a href="#gateway" aria-hidden="true" class="aal_anchor" id="gateway"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Gateway</strong></h4>



<p>Our existing gateway configuration is fairly consistent across our three environments. Since there is no standard Helm Chart for deploying the gateway, we have a custom deployment mechanism and this doesn&#8217;t take advantage of features such as affinity, pod disruption budgets, or autoscaling. There is minimal variation in CPU and memory requests with a varying number of fixed deployments of each according to environment.</p>



<figure class="wp-block-table"><table><tbody><tr><td><strong>Environment</strong></td><td><strong>Instances</strong></td><td><strong>CPU (Request, Limit)</strong></td><td><strong>Memory (Request, Limit)</strong></td></tr><tr><td>Development</td><td>1</td><td>0.25, 1</td><td>512Mi, 512Mi</td></tr><tr><td>Staging</td><td>30</td><td>0.25, 1</td><td>2Gi, 2Gi</td></tr><tr><td>Production</td><td>40</td><td>0.5, 1</td><td>2Gi, 2Gi</td></tr><tr><td><strong>Totals</strong></td><td><strong>71</strong></td><td><strong>27.75, 71</strong></td><td><strong>140.5Gi, 140.5Gi</strong></td></tr></tbody></table></figure>



<p>The &#8220;Totals&#8221; figures are approximate and based on the observation that resource consumption measurements indicate that for most of the time requested resource is sufficient.</p>



<p>There are no affinity, pod disruption budgets or autoscaling configuration.</p>



<h4 class="wp-block-heading"><a href="#router" aria-hidden="true" class="aal_anchor" id="router"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Router</strong></h4>



<p>The Apollo Router comes with a maintained Helm chart which makes it simple to set features such as pod disruption budgets, affinities, autoscaling, and many other enterprise Kubernetes features. We take advantage of the router Helm chart to vary the resource requirements across environments with more flexibility than we have with our gateway.</p>



<figure class="wp-block-table"><table><tbody><tr><td><strong>Environment</strong></td><td><strong>Instances</strong></td><td><strong>CPU (Request, Limit)</strong></td><td><strong>Memory (Request, Limit)</strong></td></tr><tr><td>Development</td><td>2-12</td><td>0.7, 1.5</td><td>1.5Gi, 1.5Gi</td></tr><tr><td>Staging</td><td>2-12</td><td>2, 3.5</td><td>1.5Gi, 2Gi</td></tr><tr><td>Production</td><td>3-12</td><td>4, 6</td><td>4Gi, 5Gi</td></tr><tr><td><strong>Totals</strong></td><td><strong>7-36</strong></td><td><strong>17.4, 28</strong></td><td><strong>18Gi, 22Gi</strong></td></tr></tbody></table></figure>



<p>I&#8217;ve aggregated the requests and limits for the main containers within a router pod: router and the coprocessor. The resources are approximately 50/50 for each container.</p>



<p>Because we are using horizontal pod autoscaling (HPA) with the router, it&#8217;s harder to calculate the &#8220;Totals&#8221; figure. However, we note that autoscaling is never triggered in development or staging. In production it is triggered, but fairly infrequently, so that the net result is approximately the same as running 3 routers constantly.</p>



<p>The router is configured to have affinity with our Istio mesh ingresses, anti-affinity with other router pods (to improve node throughput and reliability) and we have a pod disruption budget set to a maximum unavailability of 1 (to improve reliability). We have configured horizontal pod autoscaling from a minimum of 2 (3 in production) and a maximum of 12, with triggering set for 75% CPU consumption.</p>



<h3 class="wp-block-heading"><a href="#responsiveness" aria-hidden="true" class="aal_anchor" id="responsiveness"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Responsiveness</strong></h3>



<p>The primary measure of responsiveness is request duration, as measured from arriving at the router from a client to sending the response to the client.</p>



<figure class="wp-block-table"><table><tbody><tr><td><strong>Federator</strong></td><td><strong>Average Response Time (ms)</strong></td></tr><tr><td>Gateway</td><td>510</td></tr><tr><td>Router</td><td>165</td></tr></tbody></table></figure>



<p>We couldn&#8217;t directly compare gateway and router responsiveness over the exact same time interval since we were not mirroring traffic but directing traffic to one or the other. For purposes of this comparison, we collected the data on multiple, separate days across the same time period (8 hours) in each day and averaged the results.</p>



<h2 class="wp-block-heading"><a href="#sizing-and-tuning" aria-hidden="true" class="aal_anchor" id="sizing-and-tuning"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Sizing and Tuning</strong></h2>



<p>Our production workload varies over the course of 24 hours, with a peak during U.S. working hours and quieter times during the rest of the day. In a typical week, we are processing about 70M requests.</p>



<p>In this scenario, we&#8217;re measuring Client Request Rate per Second (RRS). We have sized our router fleet based on the following data:</p>



<p>Average RRS: 115 (70M / 604800) Peak RRS: 815 Baseline Subgraph Latency: 150ms Average Client Request Size: 1k Average Client Response Size: 1m Number of Instances: 3 &#8211; 12</p>



<h2 class="wp-block-heading"><a href="#kubernetes" aria-hidden="true" class="aal_anchor" id="kubernetes"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Kubernetes</strong></h2>



<p>We use Helm to manage our Kubernetes router fleet. We have an Apollo in-house Helm chart which provides Apollo-specific configuration details such as:</p>



<ul class="wp-block-list">
<li>Rhai Scripts</li>



<li>Istio configuration</li>
</ul>



<p>The in-house Helm chart makes use of the Router Helm Chart to provide most of the heavy lifting for the configuration of:</p>



<ul class="wp-block-list">
<li>Router (header propagation, max request sizes, introspection, etc&#8230;)</li>



<li>Standard Kubernetes Functionality (Resources, Affinity, Pod Disruption Budgets, Sidecars, etc&#8230;)</li>
</ul>



<p>We separate our Helm configuration files (values.yaml) into a common file and an environment-specific file. We make use of the fashion in which Helm progressively applies values, by ordering the files for application with common values first and environment-specific values following later in the command.</p>



<h2 class="wp-block-heading"><a href="#observability" aria-hidden="true" class="aal_anchor" id="observability"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Observability</strong></h2>



<p>The router ships with a lot of observability for tracing, metrics, and logging. Most of this is easily configurable or can be enhanced with Rhai scripts or external coprocessors.</p>



<p>We use <a href="https://www.datadoghq.com/" data-type="link" data-id="https://www.datadoghq.com/">DataDog</a> to visualize the captured metrics.</p>



<h2 class="wp-block-heading"><a href="#getting-started-with-the-apollo-router" aria-hidden="true" class="aal_anchor" id="getting-started-with-the-apollo-router"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a><strong>Getting started with the Apollo Router</strong></h2>



<p>Apollo has many resources to help you get up and running with the Apollo Router.&nbsp;</p>



<ul class="wp-block-list">
<li>Watch this Apollo tech talk to <a href="https://www.apollographql.com/events/tech-talk/extending-apollo-router-3/">learn more about the Apollo Router’s extensibility model</a>.</li>



<li><a href="https://www.apollographql.com/events/champions-corner/how-wayfair-slashed-costs-simplified-infra-and-cut-latency-in-half-with-apollo-router/">This Champions Corner with Wayfair</a> discusses how they scaled their router during Black Friday.</li>



<li><a href="https://www.apollographql.com/docs/router/">The updated docs</a> now include more how-tos and reference guides for <a href="https://www.apollographql.com/docs/router/containerization/kubernetes/">deploying the Apollo Router in Kubernetes</a>.</li>
</ul>



<p>You can also connect with us as well as other users in our <a href="https://events097286.typeform.com/joincommunity">Apollo Champions Slack community</a>. You can also <a href="https://discord.com/invite/VZNZV2ujaY" data-type="link" data-id="https://discord.com/invite/VZNZV2ujaY">find us on Discord</a> if you prefer that channel.  Let us know if you have any questions!</p>


Engineering

Deploying the Apollo Router at Apollo

How we deploy the Apollo GraphOS monorepo

Engineering

Deploying the Apollo Router at Apollo

How we deploy the Apollo GraphOS monorepo

Stay in our orbit