Subscription Broker Pattern

This pattern uses a dedicated service (or serverless functions) to manage all subscription logic, completely separating it from subgraphs.

When to use this pattern

• Complex PubSub systems: Using systems requiring consumer group coordination

• High scale: Thousands of concurrent subscriptions with high event throughput

• Event queuing needed: Must buffer events during Router slowdowns or network issues

• Resource isolation: Isolate subscription processing from subgraph performance

• Multiple subgraphs: Want to avoid duplicating subscription infrastructure across subgraphs

• Complex fan-out: One event needs to reach thousands of subscribers

Architecture diagram

Component responsibilities

Subgraph

• Receive subscription request from Router

• Make HTTP API call to the broker with subscription details

• Return success/failure to Router

• No further involvement in subscription lifecycle

Subscription broker

• Manage all subscription state and lifecycle

• Send heartbeats to Router instances

• Subscribe to PubSub topics/channels

• Receive events from the PubSub system

• Queue events per subscription with backpressure handling

• Deliver events to Router via callback URLs

• Handle subscription resumption using Redis state

Redis

• Store subscription metadata

• Track last delivered event for resumption

• Coordinate broker instances (for distributed deployments)

PubSub system

• Event backbone (examples: SNS/SQS, Google Pub/Sub, custom message queue)

• Often owned by platform or business teams

• Enables business systems to publish events independently of subscription details

Broker architecture options

Serverless subscription brokers (recommended for scale)

Use serverless functions where each function manages one subscription

• One function instance per active subscription

• Function triggered by subgraph's HTTP API call

• Maintains single heartbeat timer

• Subscribes to relevant PubSub topics

• Terminates when subscription closes

• Automatically scales to any number of subscriptions

• Simplifies timer coordination

When to use

• When you have unpredictable or highly variable subscription volumes

• When you want automatic scaling without capacity planning

• When you want each subscription can be managed independently

• When you prefer operational simplicity over cost optimization

Tradeoffs

• Cold start latency on subscription creation (typically 100-500ms)

• Higher cost per subscription than shared infrastructure (evaluate based on volume)

Centralized subscription broker

Single service managing all subscriptions:

• Simpler to operate initially

• Must manage timers for all active subscriptions

Considerations for deployment and scaling

• Ensure your broker is able to leverage horizontal scaling

  • Configure auto-scaling based on CPU exceeding 70%, memory exceeding 80%, or response time exceeding 100ms

When to use

• When working in development or testing environments

• When you have predictable, moderate subscription volumes

• When subscriptions can't be managed separately

  • Need to batch subscriptions that receive updates from the same events

• Need horizontal scaling but want to avoid serverless

Trade-offs

• Can become bottleneck at scale

• Requires capacity planning—you must estimate subscription volumes and provision resources accordingly

Managing heartbeats

Heartbeat interval

The recommended heartbeat interval to set in Router configuration is 5000 milliseconds (5 seconds). This value balances keeping connections alive while minimizing overhead. Adjust the interval based on your needs:

• Higher connection counts: Increase the interval to reduce heartbeat traffic (for example, 10000ms)

• Network latency concerns: Decrease the interval to detect failures faster (for example, 3000ms)

• Load balancer timeout: Ensure the interval is shorter than your load balancer's idle timeout

• Note: This value is set in your Router configuration file and must be enforced by the broker. If the broker takes longer to send a heartbeat Router will terminate the subscription.

Queueing events and managing backpressure

Your broker might receive events faster than it can deliver to Router.

• Queue events per subscription (in-memory or Redis-backed)

• Set maximum queue depth (for example, 1000 events)

• When full, choose a strategy:

  • Drop oldest (real-time data where freshness matters)

  • Drop newest (sequential data where order matters)

• Pause the PubSub consumption (if system supports backpressure)

• Monitor queue depths and alert when approaching limits

Next steps

See the Subscription Broker code example.