HTTP Callback Subscriptions Best Practices

Architecture patterns and operational guidance for running subscriptions with the HTTP callback protocol


This guide covers architecture patterns and operational best practices for running GraphQL subscriptions using the HTTP callback protocol in a federated supergraph.

See code examples in the Subscriptions Best Practices Examples repository for each pattern.

Why HTTP callbacks over WebSockets?

While WebSockets may seem appealing for their lower overhead and built-in ping/pong mechanisms, they have a critical limitation: subscription state can't survive connection failures.

The persistence problem

Connections can drop for many reasons—subgraph deployments, pod evictions, load balancer timeouts, network partitions. The difference is how each protocol handles these failures when they happen.

HTTP callbacks

  • Subscription state (callback URL, subscription ID, query, variables) can be persisted in Redis

  • New subgraph instances load subscription state from Redis and resume delivery

  • Clients never know the subgraph restarted

WebSockets

  • Connection state exists only as an ephemeral TCP socket

  • When the subgraph restarts or a load balancer severs the connection, the WebSocket drops

When you must use WebSockets

WebSockets can work when HTTP callbacks aren't an option for your infrastructure. Provision your Router instances with higher resources to handle persistent connection overhead:

  • Memory: 8-16 GB RAM

  • CPU: 2-4 cores

Your requirements depend on subscription volume, payload sizes, and connection patterns. Monitor memory usage during load testing to determine what your deployment needs.

See the WebSocket code example.

Choosing an architecture

The HTTP callback protocol requires managing heartbeats to detect whether Router instances still have an open connection to clients. Backend services send subscription events to those Router instances when updates are available.

The key architectural decision is where heartbeat management lives and how events flow from your PubSub system to clients.

Which pattern should you use?

Your requirementsRecommended pattern
Complex PubSub, high scale, event queuing, and backpressureSubscription Broker Pattern
Zero message loss, historical backfill, persistent storageGuaranteed Delivery Pattern

Key decision factors

Choosing a subscription broker

  • Events come from complex PubSub systems requiring consumer group coordination

  • You need event queuing, backpressure, or fan-out to thousands of subscribers per event

  • Message loss is acceptable but high throughput is required

  • High event volumes would strain subgraph resources

  • You want to completely isolate subscription infrastructure from GraphQL query processing

  • Multiple subgraphs handle subscriptions and you want centralized subscription logic

Choosing a guaranteed delivery pattern

  • Zero message loss is required (messaging, notifications, email, financial transactions)

  • Clients need historical events, not just live updates

  • Cursor-based resumption and backfill are necessary

  • You can operate persistent event storage (event store database)

  • You need audit trails or event replay capabilities

  • You can accept to have more complex infrastructure for delivery guarantees

Framework support

The two patterns in this guide work with any GraphQL framework because your subgraph offloads HTTP callback logic to a separate broker service.

If you want a simpler approach without a dedicated broker, check whether your framework natively supports the HTTP callback protocol.

Other frameworks (Python, Ruby, Go, etc.) don't currently support the HTTP callback protocol. If your framework lacks native support and you prefer not to build a broker service, use WebSocket passthrough mode instead.

Shared concerns

These considerations apply to all patterns.

Configure the load balancer

The HTTP callback protocol requires long-lived multipart HTTP connections between your client and Router. Configure your load balancers with these requirements:

  • The load balancer must support multipart HTTP

  • Disable response buffering for subscription endpoints

  • Set idle timeout high enough for subscription duration, ensuring that the connection doesn't time out between heartbeat intervals

  • Set max connection lifetime appropriately based on how long the connection should remain open

Refer to your load balancer's documentation for specific configuration options.

Session affinity

When you establish a subscription, your client opens a persistent HTTP multipart connection to a specific Router instance. Events must be delivered over this exact connection; a different instance of the Router can't deliver events because it doesn't hold the client's connection.

Use session affinity to route callbacks to the correct instance. Configure your load balancers for sticky sessions using the subscription ID or callback URL. This ensures all callback traffic—heartbeats and event delivery—reaches Router instance holding the client connection.

Alternative: Direct instance URLs

Your broker/subgraph/sidecar can send callbacks directly to a Router instance-specific URLs (for example, https://router-instance-3.internal:4000/callback):

  • Eliminates need for load balancer session affinity

  • Configure Router to provide instance-specific callback URLs to subgraphs

  • Increases broker complexity (must track individual instance addresses)

  • May simplify routing but adds operational overhead

Handle client resilience

HTTP connections can fail, so your clients must handle disconnections gracefully.

Reconnection strategy

A client-side connection can terminate for multiple reasons: a Router restart, a networking issue, or a supergraph schema update (which triggers Router to reload). When this happens, your client should:

  1. Detect connection loss (when the HTTP response stream closes or errors)

  2. Reconnect using the same subscription parameters and subscription ID

  3. Resume from the last received event

note
Apollo Client, Apollo Kotlin, and Apollo iOS all support HTTP subscriptions with built-in retry and reconnection.

Resume subscriptions

Client-side

  • Generate a unique subscription ID (UUID) when first creating a subscription

  • Include the ID in all subscription requests

  • Track the sequence number or timestamp of the last received event

  • On reconnection, send subscription ID and last event marker

Broker/subgraph-side

  • Store subscription metadata in Redis keyed by subscription ID

  • Include last delivered event sequence/timestamp, query, and variables

  • On reconnection request with existing ID, look up the subscription in Redis:

    • Resume from the stored checkpoint if found

    • Start a new subscription if not found or expired

  • Set Redis TTL (for example, 1 hour) to clean up abandoned subscriptions

Event tracking

  • Events should include sequence numbers or timestamps

  • Store last delivered event marker after successful delivery

  • On reconnection, query PubSub for events after stored marker

Handling duplicate subscriptions and events

Duplicate subscriptions
  • Client-generated IDs: Use the same subscription ID on reconnection to resume

  • Broker/subgraph deduplication: Recognize existing subscription IDs and resume instead of creating new subscriptions

Duplicate events
  • Application-level handling: Design application logic to handle duplicate events gracefully

  • Deduplication window: Client stores recently delivered event IDs to ignore duplicates

Router lifecycle and failover

During normal operation

  • Router maintains open HTTP multipart connections to clients

  • Broker/sidecar sends heartbeat checks at configured intervals

  • Router responds with 204 status to confirm connection is alive

  • Events flow through callbacks to the Router instance holding the client connection

When your Router instance shuts down

  1. Broker/sidecar detects heartbeat failures

  2. Mark affected subscriptions as closed in Redis

  3. Clients detect connection loss

  4. Clients reconnect with subscription ID

  5. New connection might route to a different Router instance

  6. Broker/sidecar resumes from last delivered event (if Redis entry hasn't expired)

After the new Router instance starts up

  • New subscription requests route to healthy instances

  • Existing subscriptions not automatically migrated; clients must resubscribe

  • With subscription resumption, clients don't miss events during transition

  • Implement health checks for broker/sidecar to detect Router availability

Redis state management

What to store

  • Subscription ID (key)

  • Callback URL

  • Query and variables

  • Last delivered event (sequence number or timestamp)

  • Subscription creation timestamp

  • Heartbeat metadata (last success, failure count)

Configuration

Monitoring and observability

Track these metrics to ensure your subscription operations remain healthy:

Subscription metrics

  • Active subscription count per Router instance

  • Subscription creation rate and failure rate

  • Subscription duration distribution

Heartbeat metrics

  • Heartbeat success/failure rate

  • Heartbeat latency (time to receive acknowledgment)

  • Failed heartbeat count per Router instance

Event delivery metrics

  • Event delivery latency (published → callback sent)

  • Callback success/failure rate

  • Queue depth (if using queuing)

  • Events dropped due to full queues

Resource metrics

  • Broker/sidecar CPU and memory usage

  • Router instance connection count

  • Redis memory usage and command latency

  • PubSub consumer lag

Set up alerts for these conditions:

  • Heartbeat failure rate exceeds 5%

  • Queue depth exceeds 80% of configured limit

  • Broker/sidecar CPU exceeds 80% or memory exceeds 85%

  • Event delivery latency increases over time

  • Callback failure rate exceeds 10%

Feedback