# Designing a Usage-Based Billing Pipeline for SaaS > **Source:** [https://lorbic.com/designing-a-usage-based-billing-pipeline-for-saas/](https://lorbic.com/designing-a-usage-based-billing-pipeline-for-saas/) > **Author:** [Vikash Patel](https://vikashpatel.net) > **Published:** August 01, 2026 > **Reading Time:** 14 min > > *This is the raw Markdown source of the article from the [Lorbic Technical Journal](https://lorbic.com/).* --- In [How Multi-Tenant SaaS Actually Works](/posts/2026-07-01-how-multi-tenant-saas-works/), I left one major promise unfulfilled at the end: *"Billing is an entire system design on its own. The billing pipeline gets its own post."* This is that post. Building a billing pipeline is different from building almost any other backend component. If your caching layer has a bug, latencies increase by 50ms. If your log aggregator drops a packet, a dashboard chart dips for a minute. But if your billing pipeline has a bug, it operates silently in the dark. It either undercharges your customers (and your business bleeds cash without throwing a single 500 status code) or overcharges them (and you wake up to customer support tickets, disputes, and ruined trust). Relay pays upstream LLM providers per token, marks up the cost by 20%, and bills tenants monthly based on their total usage. In this post, we will design the complete billing system that makes this work reliably. Whether you use Stripe, Razorpay, Paddle, PayU, or PayPal, the fundamental engineering problems are identical. They all emit webhooks, they all encounter payment failures, and they all model subscriptions using the same underlying mechanics. We are going to design this system using clean, provider-agnostic architecture. --- ## How Relay's Billing Model Works Before diving into webhooks and database schemas, let us lay out how data moves through Relay's billing pipeline across three distinct layers: 1. **Real-Time Cost Tracking**: Every API request passing through Relay logs token usage and cost calculation into the `requests` table. 2. **Asynchronous Aggregation**: A background worker continuously aggregates raw request data into the `usage_daily` rollup table. 3. **Monthly Metered Invoicing**: At the end of a billing cycle, Relay's billing worker queries the daily usage, compiles invoice line items, and submits them to the payment gateway. ```mermaid graph TB RELAY[Relay API Gateway] --> CUST["Gateway Customer\n(Created per tenant at signup)"] CUST --> SUB["Subscription\n(Monthly cycle)"] SUB --> ITEM1["Item: OpenAI Token Usage"] SUB --> ITEM2["Item: Anthropic Token Usage"] ITEM1 --> UR["Usage Reports\n(Flushed hourly by worker)"] ITEM2 --> UR CUST --> INV["Invoice\n(Generated at period end)"] INV --> PAY["Payment Attempt\n(Automatic charge)"] ``` ### Local Database State vs Gateway State The payment gateway is always the source of truth for payment status, customer card details, and invoice settlements. Relay maintains a local replica in Postgres to serve API authorization checks and dashboard views without making remote HTTP calls on every incoming API request. ```sql CREATE TABLE tenant_billing ( tenant_id TEXT PRIMARY KEY REFERENCES tenants(id), gateway TEXT NOT NULL, -- stripe | razorpay | paddle | payu gateway_customer_id TEXT NOT NULL UNIQUE, -- cus_... | cust_... | ctm_... gateway_subscription_id TEXT, -- sub_... | null until subscribed subscription_status TEXT, -- active | past_due | canceled | ... current_period_start TIMESTAMPTZ, current_period_end TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), updated_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE TABLE tenant_subscription_items ( tenant_id TEXT NOT NULL REFERENCES tenants(id), model_family TEXT NOT NULL, -- openai_gpt4 | anthropic_claude | ... gateway_item_id TEXT NOT NULL, -- subscription item ID from gateway gateway_price_id TEXT NOT NULL, updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), PRIMARY KEY (tenant_id, model_family) ); ``` Always create the gateway customer object eagerly during tenant provisioning rather than lazily when they add a card. Every downstream flow (adding payment methods, applying credits, emitting metered usage) assumes a valid `gateway_customer_id` exists. Handling lazy customer creation forces conditional logic across dozens of handlers, creating unnecessary surface area for bugs. --- ## Subscription Lifecycle Management Payment gateways model recurring billing using finite state machines. While vendor documentation uses different terms for these states, their operational behavior is identical. ```mermaid stateDiagram-v2 [*] --> pending : Subscription created (payment pending) pending --> active : First payment succeeded pending --> expired : Payment not completed within gateway window active --> past_due : Renewal payment fails past_due --> active : Retry payment succeeds past_due --> suspended : All retries exhausted active --> canceled : Tenant cancels suspended --> active : Payment method updated + retried suspended --> canceled : Grace period expires expired --> [*] canceled --> [*] ``` Here is how common gateway states map to Relay's internal subscription model: | Operational State | Stripe | Razorpay | Paddle | PayU | Relay Action | |---|---|---|---|---|---| | **Pending** | `incomplete` | `created` | `trialing` | `PENDING` | Allow trial limits; prompt for card | | **Active** | `active` | `authenticated` | `active` | `ACTIVE` | Normal API access | | **Renewal Failing** | `past_due` | `halted` | `past_due` | `PAUSED` | Grace period; send dunning notices | | **Exhausted** | `unpaid` | `cancelled` | `canceled` | `FAILED` | Suspend access (HTTP 402 on API calls) | ### Subscription Creation and Initial Payment When a customer signs up for a paid plan, Relay creates a subscription object at the gateway. 1. **Synchronous Card Verification**: For traditional card payments without extra verification, the initial charge processes immediately. The subscription transitions directly from `pending` to `active`. 2. **Asynchronous 3D Secure / SCA**: Under regulations like PSD2 in Europe or RBI guidelines in India, payments frequently require multi-factor authentication (3D Secure). The gateway returns a pending status alongside a redirect URL. Relay keeps the subscription state as `pending` until the customer completes authentication on their bank's portal. ### Renewal Cycles and Invoice Draft Windows At the end of a billing cycle, the gateway automatically generates a renewal invoice. Most payment gateways provide a temporary draft window (typically 1 hour) between invoice creation and charge execution. This draft window acts as an operational buffer. If Relay's hourly usage worker experienced a transient delay near midnight, the system uses the `invoice.created` webhook event to perform a final reconciliation pass, injecting any missing line items before the gateway finalizes the bill. --- ## Idempotent Webhook Processing Webhooks are delivered over public networks using **at-least-once delivery guarantees**. This means your system *will* receive identical events multiple times, and events *will* occasionally arrive out of sequence. A webhook processing pipeline must obey three strict engineering rules: 1. **Verify signature and enqueue instantly**: Validate signatures against raw payload bytes, insert the event ID into a storage table, enqueue a worker task, and return HTTP 200 immediately. 2. **Deduplicate atomically**: Use database constraints to prevent double-processing. 3. **Re-fetch live state**: Do not rely on event payload snapshots. Fetch the latest state directly from the gateway API before applying changes. ```mermaid sequenceDiagram participant Gateway as Payment Gateway participant Endpoint as API Endpoint (/webhooks) participant DB as Postgres DB participant Queue as Worker Queue participant Worker as Background Worker Gateway->>Endpoint: POST /v1/webhooks/billing (Raw Body + Signature) Endpoint->>Endpoint: Verify HMAC Signature on Raw Bytes Endpoint->>Queue: Enqueue Event Processing Job Endpoint-->>Gateway: HTTP 200 OK (Fast Ack < 100ms) Queue->>Worker: Dequeue Job (Event ID) Worker->>DB: INSERT INTO webhook_events ON CONFLICT DO NOTHING alt Event Already Exists (RowsAffected == 0) Worker-->>Queue: Acknowledge Job (Skip duplicate) else New Event Worker->>Gateway: GET /v1/subscriptions/{id} (Fetch Live State) Gateway-->>Worker: Return Current Subscription Object Worker->>DB: Update Local Tenant State in Transaction Worker->>DB: UPDATE webhook_events SET status = 'done' Worker-->>Queue: Acknowledge Job Complete end ``` ### Raw Signature Verification Always verify HMAC signatures on the raw byte stream from the HTTP request body. Never parse JSON first and verify second; JSON parsers do not preserve key ordering or whitespace formatting, which invalidates the computed signature digest. ```go package billing import ( "crypto/hmac" "crypto/sha256" "encoding/hex" "errors" "net/http" ) type GatewayEvent struct { ID string Type string ObjectType string ObjectID string RawPayload []byte } type PaymentGateway interface { Name() string VerifyWebhook(body []byte, headers http.Header) (GatewayEvent, error) FetchEvent(ctx context.Context, eventID string) (GatewayEvent, error) GetSubscription(ctx context.Context, subID string) (Subscription, error) GetInvoice(ctx context.Context, invoiceID string) (Invoice, error) ReportUsage(ctx context.Context, record UsageRecord) error CreateCreditNote(ctx context.Context, req CreditNoteRequest) (string, error) } func VerifyHMACSignature(rawBody []byte, signatureHeader, secret string) bool { mac := hmac.New(sha256.New, []byte(secret)) mac.Write(rawBody) expectedMAC := hex.EncodeToString(mac.Sum(nil)) return hmac.Equal([]byte(signatureHeader), []byte(expectedMAC)) } ``` ### Atomic Database Deduplication We track processed events using a dedicated database table. Using PostgreSQL's `ON CONFLICT (event_id) DO NOTHING`, we guard against concurrent or repeated event processing inside a single database transaction. ```sql CREATE TABLE webhook_events ( event_id TEXT PRIMARY KEY, gateway TEXT NOT NULL, event_type TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'pending', -- pending | done | failed payload JSONB, processed_at TIMESTAMPTZ ); ``` ```go func (w *BillingWorker) ProcessEvent(ctx context.Context, eventID, eventType string, payload []byte) error { return w.db.WithTx(ctx, func(tx *sql.Tx) error { res, err := tx.ExecContext(ctx, ` INSERT INTO webhook_events (event_id, gateway, event_type, status, payload) VALUES ($1, $2, $3, 'pending', $4) ON CONFLICT (event_id) DO NOTHING`, eventID, w.gateway.Name(), eventType, payload) if err != nil { return err } rows, err := res.RowsAffected() if err != nil { return err } if rows == 0 { // Event already processed by another worker; exit safely. return nil } // Re-fetch the live object from the gateway API to handle out-of-order events. event, err := w.gateway.FetchEvent(ctx, eventID) if err != nil { return err } if err := w.applyEvent(ctx, tx, event); err != nil { return err // Rollback transaction so retry can attempt later } _, err = tx.ExecContext(ctx, `UPDATE webhook_events SET status = 'done', processed_at = now() WHERE event_id = $1`, eventID) return err }) } ``` If your business logic fails, the database transaction rolls back, reverting both the tenant state update and the `webhook_events` insert. When the gateway retries the webhook delivery later, the worker can safely try processing it again. --- ## Dunning Logic and Failed Payment Handling Dunning is the automated process of managing failed payments. Involuntary churn (failures caused by expired cards, temporary bank holds, or insufficient funds) accounts for a large percentage of revenue loss in SaaS. Dunning consists of two synchronized layers: 1. **Technical Dunning**: Automated retry schedules managed by the payment gateway. 2. **Communication & Access Dunning**: In-app banners, email notifications, feature restrictions, and eventual account suspension managed by Relay. ```mermaid flowchart TD A[Webhook: payment.failed] --> B{Decline Classification} B -->|Soft Decline: Insufficient Funds| C[Keep API Active\nSchedule Retry] B -->|Hard Decline: Stolen / Invalid Card| D[Immediate Access Restriction] C --> E{Attempt Count} E -->|Attempt 1| F[Send Email: Card Retry Notice] E -->|Attempt 2| G[In-App Dashboard Warning Banner] E -->|Attempt 3| H[Restrict New API Key Generation] E -->|Final Attempt Failed| I[Update Tenant Status to SUSPENDED\nReturn HTTP 402 on Gateway Requests] D --> I ``` ### Soft vs Hard Declines Payment declines fall into two broad categories: - **Soft Declines** (`insufficient_funds`, `network_timeout`, `processing_error`): These indicate temporary issues. The gateway retries the charge automatically over a multi-day schedule. Relay grants a grace period during soft declines. - **Hard Declines** (`stolen_card`, `lost_card`, `invalid_account`): These failures are permanent. Retrying will not succeed. Relay immediately prompts the customer for a new payment method and restricts administrative operations. ### Graduated Grace Period Implementation Relay handles dunning progressively without immediately shutting down a customer's production traffic on the first payment failure: ```go func (w *BillingWorker) OnPaymentFailed(ctx context.Context, tx *sql.Tx, payment FailedPayment) error { tenant, err := w.tenants.GetByGatewayCustomer(ctx, payment.CustomerID) if err != nil { return err } mrr, err := w.billing.GetMRR(ctx, tenant.ID) if err != nil { return err } if isHardDecline(payment.DeclineCode) { w.notifyOps(ctx, tenant.ID, payment.DeclineCode, "Hard decline encountered") } switch payment.AttemptCount { case 1: portalURL := w.gateway.BillingPortalURL(ctx, tenant.GatewayCustomerID) w.mailer.Send(ctx, tenant.BillingEmail, "payment_failed_initial", map[string]string{ "portal_url": portalURL, }) case 2: w.mailer.Send(ctx, tenant.BillingEmail, "payment_failed_reminder", nil) if mrr > 50000 { // High value customer (> $500 MRR) w.notifySales(ctx, tenant.ID, "medium_priority") } default: w.mailer.Send(ctx, tenant.BillingEmail, "payment_failed_final", nil) if mrr > 50000 { w.notifySales(ctx, tenant.ID, "high_priority") } // Restrict administrative setup while keeping runtime proxy endpoints working w.features.Restrict(ctx, tenant.ID, "create_api_keys", "upgrade_models") } return w.tenants.UpdateBillingStatus(ctx, tx, tenant.ID, "PAYMENT_FAILING", map[string]any{ "attempt_count": payment.AttemptCount, }) } ``` If payment retries fail completely and the gateway marks the subscription as uncollectible, Relay transitions the tenant status to `SUSPENDED`. The API gateway middleware immediately begins rejecting runtime API requests with HTTP status `402 Payment Required`. --- ## Credit Notes and Accounting Adjustments A credit note is a formal accounting document that offsets a previously issued invoice. You should never edit or delete an existing finalized invoice in your database. Instead, issue a credit note to maintain a clean financial audit trail. Common scenarios requiring credit notes include: 1. **Outage Credits**: SLA compensation issued after an upstream provider outage. 2. **Partial Overcharge Refunds**: Correcting token calculation overcharges. 3. **Bad Debt Write-Offs**: Clearing unpaid invoices for canceled accounts. ```sql CREATE TABLE billing_adjustments ( id TEXT PRIMARY KEY, tenant_id TEXT NOT NULL REFERENCES tenants(id), gateway_credit_note_id TEXT NOT NULL, type TEXT NOT NULL, -- service_credit | refund | write_off amount_cents BIGINT NOT NULL, currency TEXT NOT NULL DEFAULT 'USD', memo TEXT, issued_at TIMESTAMPTZ NOT NULL DEFAULT now(), issued_by TEXT NOT NULL ); ``` ### The Paid Invoice Allocation Rule When issuing a credit note against a paid invoice, the sum of all credit allocations must equal the total invoice value being adjusted: ``` Refund Amount + Customer Balance Credit + Write-off Amount = Invoice Amount Adjusted ``` ```go type CreditNoteRequest struct { InvoiceID string AmountCents int64 Type string // "balance_credit" | "refund" | "write_off" Memo string IdempotencyKey string } func (s *BillingService) IssueServiceCredit(ctx context.Context, tenantID string, amountCents int64, memo string) error { billing, err := s.db.GetTenantBilling(ctx, tenantID) if err != nil { return err } invoice, err := s.gateway.GetLastPaidInvoice(ctx, billing.GatewayCustomerID) if err != nil { return err } idempotencyKey := fmt.Sprintf("credit-%s-%d", tenantID, time.Now().UnixMilli()) creditNoteID, err := s.gateway.CreateCreditNote(ctx, CreditNoteRequest{ InvoiceID: invoice.ID, AmountCents: amountCents, Type: "balance_credit", Memo: memo, IdempotencyKey: idempotencyKey, }) if err != nil { return err } return s.db.InsertBillingAdjustment(ctx, BillingAdjustment{ TenantID: tenantID, GatewayCreditNoteID: creditNoteID, Type: "service_credit", AmountCents: amountCents, Memo: memo, IssuedBy: "system", }) } ``` --- ## Usage-Based Invoice Generation Unlike fixed monthly software subscriptions, metered billing requires reporting cumulative consumption metrics to the gateway throughout the billing cycle. ```mermaid sequenceDiagram participant Proxy as Relay API Proxy participant Log as Request Logger participant DB as Postgres DB participant Worker as Hourly Billing Worker participant GW as Payment Gateway Proxy->>Log: Request Complete (1,500 Tokens) Log->>DB: INSERT INTO requests Log->>DB: UPSERT INTO usage_daily Note over Worker: Runs Hourly via Cron Job Worker->>DB: SELECT SUM(total_tokens) WHERE hour = LAST_HOUR Worker->>GW: ReportUsage(item_id, quantity, idempotency_key) GW-->>Worker: Usage Record Acknowledged Note over GW: Billing Period Closes GW->>GW: Compile Invoice Line Items (Quantity * Tier Rate) GW->>Proxy: Webhook: invoice.created (Draft Status) Proxy->>DB: Run Final Reconciliation Check Proxy->>GW: Inject Any Unreported Residual Usage Items GW->>GW: Finalize & Charge Customer Card ``` ### Reporting Hourly Usage Safely To report consumption without double-counting usage during retries, construct a deterministic **idempotency key** for every usage payload. Format: `usage-{tenant_id}-{model_family}-{hour_timestamp}` ```go type UsageRecord struct { ItemID string Quantity int64 At time.Time IdempotencyKey string } func (w *BillingWorker) ReportHourlyUsage(ctx context.Context) error { hourBucket := time.Now().UTC().Truncate(time.Hour).Add(-time.Hour) rows, err := w.db.QueryHourlyUsage(ctx, hourBucket) if err != nil { return err } for _, row := range rows { itemID, err := w.db.GetGatewayItemID(ctx, row.TenantID, row.ModelFamily) if err != nil { if errors.Is(err, ErrItemNotFound) { if refreshErr := w.refreshSubscriptionItems(ctx, row.TenantID); refreshErr != nil { return refreshErr } itemID, err = w.db.GetGatewayItemID(ctx, row.TenantID, row.ModelFamily) } if err != nil { return err } } idempotencyKey := fmt.Sprintf("usage-%s-%s-%d", row.TenantID, row.ModelFamily, hourBucket.Unix()) err = w.gateway.ReportUsage(ctx, UsageRecord{ ItemID: itemID, Quantity: row.TotalTokens, At: hourBucket, IdempotencyKey: idempotencyKey, }) if err != nil { w.log.Error("Failed to report usage", "tenant", row.TenantID, "error", err) // Continue processing other tenants } } return nil } ``` If the hourly job fails midway through processing 1,000 tenants, re-running the job is completely safe. The gateway uses the deterministic idempotency key to ignore duplicate reports. --- ## End-to-End Billing Architecture Here is the complete blueprint showing how real-time request tracking, hourly aggregation, webhook processing, and dunning workflows fit together: ```mermaid graph TB subgraph HotPath["1. Real-Time Hot Path"] API[API Request] --> LOG[Request Logger] LOG --> REQS[(requests Table)] LOG --> DAILY[(usage_daily Rollup)] end subgraph HourlyWorker["2. Metered Usage Pipeline"] HW[Hourly Cron Worker] --> DAILY HW -->|ReportUsage with Idempotency Key| GW_USAGE[Gateway API] end subgraph WebhookPipeline["3. Asynchronous Webhook Processor"] WH[POST /webhooks/billing] --> SIG[HMAC Signature Check] SIG --> Q[Worker Queue] Q --> WW[Webhook Worker] WW --> IDEM[(webhook_events Table)] WW -->|Fetch Live State| GW_API[Gateway API] GW_API --> STATE[Update Postgres Tenant Status] end subgraph DunningPipeline["4. Dunning Engine"] FAIL_EVENT[Event: payment.failed] --> DW[Dunning Worker] DW --> EMAIL[Email Provider API] DW --> RESTRICT[Restrict Tenant Permissions] end subgraph Reconciliation["5. Period-End Reconciliation"] INV_EVENT[Event: invoice.created] --> RECON[Reconciliation Worker] RECON --> REQS RECON -->|Inject Missing Line Items| GW_API end ``` --- ## FAQ **Why create gateway customer objects eagerly at signup instead of lazily when adding a payment method?** Creating customer objects at signup guarantees that every tenant record in your database has a valid `gateway_customer_id`. This eliminates conditional checks across your codebase when attaching payment methods, processing trials, issuing promotional credits, or fetching invoices. **What happens if our webhook processor drops offline for 12 hours?** Payment gateways store undelivered webhooks and retry delivery for 3 to 7 days using exponential backoff schedules. When your processor comes back online, it processes accumulated events. Because your worker re-fetches live state from the gateway API before making database updates, processing stale events remains safe and correct. **How do we handle mid-cycle plan upgrades?** When a tenant upgrades their plan, the payment gateway replaces existing subscription items with new ones. Your system must listen for `subscription.updated` webhooks and update local `tenant_subscription_items` mappings immediately. Otherwise, hourly usage reports will target deleted item IDs and fail. **Can daily usage rollups fall behind during traffic spikes?** Yes. Aggregation jobs can lag during heavy traffic surges. For this reason, period-end invoice reconciliation queries the primary `requests` log table directly rather than relying solely on rollups.