Gmail API Push Notifications: Pub/Sub Architecture Explained

"Design a system that reacts to new email in near real time" is a deceptively deep system design interview prompt, and Gmail's own push-notification API is one of the cleanest real-world answers to it. It is a textbook example of event-driven, webhook-based design built on a message broker — so understanding it teaches you the exact patterns interviewers probe in any "design a notification or webhook system" round. This deep dive walks the real Gmail API Pub/Sub push notifications architecture end to end, using only the actual API surface: users.watch, users.history.list, and users.stop.

Polling vs Push: Why Gmail Built This

The naive way to detect new mail is to poll: hit the API every few seconds and ask "anything new?" Polling is simple but wasteful. At low frequency it is slow — you might wait 30 seconds to see a message. At high frequency it burns quota and CPU on requests that almost always return "nothing changed." Multiply that by millions of mailboxes and the cost is enormous.

Push inverts the flow: the source tells you the instant something changes. Gmail publishes a notification to a Google Cloud Pub/Sub topic the moment a watched mailbox changes, so you do zero idle work and learn about changes in roughly a second. The trade-off is operational complexity — you now run a topic, a subscription, and a webhook, and inherit the hard parts of distributed messaging (duplicates, ordering, retries, expiry). That trade-off, push complexity for freshness and efficiency, is exactly the discussion an interviewer wants to hear.

The Components

There are five moving parts. Get the responsibilities straight and the rest follows.

• users.watch() on a label — your registration call. You tell the Gmail API which mailbox to watch, which Cloud Pub/Sub topic to publish to, and which labels to scope to (for example INBOX, or only UNREAD). This is what arms the notifications. It returns a starting historyId and an expiration timestamp.
• A Cloud Pub/Sub topic — the broker Gmail publishes to. It decouples Gmail (the producer) from your service (the consumer). The Gmail API's service account must hold pubsub.publisher rights on this topic, or publishing silently fails.
• A Pub/Sub subscription — how your service receives messages from the topic. A topic can have many subscriptions; each gets its own copy of every message. The subscription is where you choose delivery style.
• Push webhook (HTTPS) vs pull — a push subscription POSTs each message to an HTTPS endpoint you own; a pull subscription has your workers call Pub/Sub to fetch and acknowledge messages. Push is lower-latency and simpler to start; pull gives you flow control and is easier to scale horizontally with a worker pool.
• Your processor + datastore — the code that receives the notification, calls back into Gmail to fetch what changed, and persists the result. Critically, it also stores the last historyId it has processed per mailbox.

The Notification Payload: a Pointer, Not the Data

The single most important thing to understand — and the detail that separates someone who has read the docs from someone who hasn't — is that the notification carries no email content. The Pub/Sub message data field is base64-encoded JSON with exactly two fields:

{
  "emailAddress": "[email protected]",
  "historyId": "9876543210"
}

That is the whole payload. emailAddress identifies the mailbox; historyId is a monotonic cursor marking the mailbox's new state. The notification is a doorbell, not a delivery — "something changed, the new watermark is this." You pull the actual changes yourself.

You do that by calling users.history.list with startHistoryId set to the last historyId you stored for that mailbox — not the one in the notification. Gmail returns the ordered list of changes (messages added, deleted, labels added or removed) between your stored cursor and now; you apply them, then advance your stored historyId. This stored-cursor design is what makes the system resilient: even if you miss notifications, the next one lets you replay everything since your last known position.

Architecture Diagram

Here is the full flow, from arming the watch to durable storage:

  ┌─────────────┐   1. users.watch(label, topic)   ┌──────────────────┐
  │ Your Service│ ───────────────────────────────▶ │   Gmail API      │
  │ (registrar) │ ◀─── returns {historyId, exp} ─── │  (the mailbox)   │
  └─────────────┘                                   └────────┬─────────┘
                                                             │ 2. mailbox
                                                             │    changes
                                                             ▼
                                              ┌──────────────────────────┐
                                              │  Cloud Pub/Sub  TOPIC    │
                                              │  msg = {emailAddress,    │
                                              │         historyId}       │
                                              └────────────┬─────────────┘
                                                           │ 3. fan-out
                                                           ▼
                                              ┌──────────────────────────┐
                                              │  Pub/Sub  SUBSCRIPTION   │
                                              │  (push HTTPS  |  pull)   │
                                              └────────────┬─────────────┘
                                                           │ 4. deliver + OIDC JWT
                                                           ▼
                                              ┌──────────────────────────┐
                                              │   YOUR WEBHOOK (HTTPS)   │
                                              │  verify token → ack 200  │
                                              └────────────┬─────────────┘
                                                           │ 5. history.list(startHistoryId =
                                                           │       stored cursor)
                                                           ▼
                                              ┌──────────────────────────┐
                                              │   Gmail API  history     │  ◀── returns the delta
                                              └────────────┬─────────────┘
                                                           │ 6. apply changes,
                                                           │    advance cursor
                                                           ▼
                                              ┌──────────────────────────┐
                                              │   YOUR DATABASE          │
                                              │  messages + lastHistoryId│
                                              └──────────────────────────┘

Read it as a loop: watch() arms it → Gmail publishes to the topic → the subscription delivers to your webhook → you call history.list from your stored cursor → you write to your DB and move the cursor forward. Every arrow is a place where an interviewer can ask "what happens if this fails?"

Idempotency: Duplicates and Out-of-Order Delivery

Cloud Pub/Sub guarantees at-least-once delivery. That implies two things you must design for: the same notification can arrive more than once, and notifications can arrive out of order. If your handler is not idempotent, a redelivered message could double-process a change or overwrite newer state with older.

The defense is the stored-cursor pattern plus idempotent writes:

• Persist the largest fully-processed historyId per mailbox and never move it backward. Re-fetching from that cursor is always safe because history.list returns only the genuine delta.
• Apply message changes as upserts keyed by message ID. Re-applying "message abc123 got label UNREAD" twice yields the same final state, so replays become a no-op.
• Advance the cursor only after the delta is committed. Crash mid-batch and the next notification replays from the old cursor — the same checkpoint discipline you would design into any consumer.

Notice the architecture turns the hard "exactly-once" problem into an easier "at-least-once + idempotent" one — worth stating explicitly in an interview, since true exactly-once delivery across a network is effectively impossible.

Ack Deadlines and Retries

Pub/Sub holds each message as outstanding until your subscriber acknowledges it. For a push subscription, returning an HTTP 2xx from your webhook is the ack; any other status (or a timeout) is a nack, and Pub/Sub redelivers after a backoff. For pull, you call acknowledge explicitly. Each subscription has an ack deadline (commonly tuned to seconds to minutes); if you do not ack within it, the message is considered unacknowledged and redelivered.

This has a clean design implication: ack fast, work durably. Don't do the slow history.list call and database writes inside the request you must ack. A robust pattern is to validate the notification, drop it onto an internal queue, ack immediately, and let workers do the fetch-and-persist. That keeps you under the ack deadline and protects you from a slow Gmail call triggering a storm of redeliveries. Pair retries with a dead-letter topic so a permanently poisonous message doesn't loop forever.

The 7-Day Watch Expiry

A watch is not permanent. It expires after seven days, and once it lapses Gmail simply stops publishing — no error, just silence, which makes this a favorite source of "why did notifications mysteriously stop" bugs. users.watch() returns the expiration timestamp, so the fix is to re-call watch() on a schedule well before expiry; once per day is the standard recommendation, which both refreshes the window and self-heals any transient gap. When you want to stop receiving notifications for a mailbox entirely, call users.stop(). Treat watch renewal as a first-class cron job, not an afterthought — its absence is invisible until mail stops flowing.

Security: Lock Down the Endpoint

Your webhook is a public HTTPS URL, so you must assume the open internet will find it. Two layers protect it:

• Verify the Pub/Sub OIDC/JWT token. Configure the push subscription with a service account so Pub/Sub attaches a signed OpenID Connect JWT in the Authorization header of every request. Your endpoint verifies the signature against Google's public keys, checks the aud claim matches your endpoint URL, and confirms the token's email is the service account you authorized. Reject anything that fails — this is what stops a forged POST from injecting fake notifications.
• Lock down publish rights on the topic. Grant only the Gmail API service account permission to publish to your topic, so Gmail is the sole legitimate producer.

And because the payload is only a pointer, even a spoofed notification can't leak data: the worst it does is make you call history.list, which is authenticated and returns only that mailbox's real changes. Treating the body as an untrusted trigger and re-fetching from the source API is the defensive default.

Scaling

The design scales naturally because the broker absorbs spikes. A few notes for the "now make it handle millions of mailboxes" follow-up:

• Pull subscriptions for horizontal scale. A pool of stateless workers pulling one subscription gives you flow control and easy autoscaling; Pub/Sub load-balances across them.
• The topic buffers bursts. A marketing blast hitting a million inboxes queues in the topic, and consumers drain it at a sustainable rate instead of toppling over.
• Stagger watch renewals across the day so you don't fire a thundering herd of watch() calls at midnight.
• Mind Gmail API quota. Each notification triggers a history.list call, so respect rate limits to avoid 429s under load.

What Interviewers Probe in a "Design a Notification/Webhook System" Round

Gmail's design maps almost one-to-one onto the generic webhook-system question. If you can explain this flow, you can answer that round. Here's what the interviewer is really listening for:

• Push vs poll, and why. Can you justify the broker instead of a tight polling loop, and name the trade-off (freshness and efficiency for operational complexity)?
• Delivery guarantees. Do you know at-least-once means duplicates and reordering, and do you reach for idempotency and a stored cursor rather than hand-waving "exactly once"?
• Acks, retries, and dead-letter queues. Do you separate "acknowledge fast" from "process durably," and handle the poison message?
• Thin payload + fetch-on-demand. Do you see why sending a pointer (a cursor) beats stuffing data into the notification — smaller messages, no stale content, security by re-fetch?
• Subscription lifecycle. Do you account for expiry/renewal and a clean teardown path?
• Security of the endpoint. Do you authenticate the sender (signed token) rather than trusting any POST that arrives?

Walking through Gmail's watch → topic → subscription → webhook → history.list → DB pipeline gives you a concrete, defensible reference design for all of those — which beats reciting abstract principles. For more on structuring this kind of answer, see our guide on designing a notification system, and the companion piece on using AI in system design interviews as a backup brain while you reason out loud.

FAQ

How do Gmail API push notifications work?

You call users.watch() on a mailbox, naming a Cloud Pub/Sub topic and the labels you care about. When a matching mailbox change occurs, Gmail publishes a small message to that topic containing the emailAddress and a new historyId. A Pub/Sub subscription delivers that message to your HTTPS webhook, and your service calls users.history.list starting from the last historyId you stored to pull the actual changes. There is no email body in the notification — it is only a pointer telling you to go fetch.

What is in a Gmail push notification payload?

The Pub/Sub message data is a base64-encoded JSON object with just two fields: emailAddress (the mailbox that changed) and historyId (a cursor marking the new state of that mailbox). It deliberately contains no message content. You diff against the historyId you previously persisted by calling users.history.list, which returns the added, deleted, and label-changed messages since that point.

How do you handle duplicate or out-of-order Gmail notifications?

Cloud Pub/Sub guarantees at-least-once delivery, so duplicates and reordering are normal and must be designed for. Make processing idempotent: store the largest historyId you have fully processed and never move it backward. Because users.history.list returns the full delta from your stored cursor, replaying an older notification simply re-fetches changes you already applied, so applying updates idempotently (upserts keyed by message ID) keeps mailbox state correct.

How long does a Gmail watch last and how do you renew it?

A watch expires after seven days, and Gmail stops publishing once it lapses. The expiration timestamp is returned by users.watch(), so you schedule a re-call of watch() well before it — daily is the common recommendation — to refresh the subscription. To stop receiving notifications entirely, call users.stop(). Renewing also re-establishes delivery if your topic or permissions ever changed.

How do you secure a Gmail Pub/Sub push webhook?

Verify the OIDC JWT that Pub/Sub attaches to each push request: confirm the token's signature against Google's public keys, check the aud claim matches your endpoint, and confirm the service account email is the one you authorized. Serve the endpoint over HTTPS, reject anything unauthenticated, and grant the Gmail API service account publish rights on your topic so only Gmail can post to it. Treat the historyId as untrusted input and re-fetch via the API rather than trusting any body content.