Shopify execution technical guide

This site documents the backend Shopify event pipeline delivered in PR #22 and PR #23. It covers the code path from webhook registration and Pub/Sub delivery through parsing, per-client Supabase persistence, failure capture, and replay.

Document fact	Value
Integration branch	redash_api
PR #22	Complete Shopify subscriber execution, merged June 16, 2026
PR #23	Storage provisioning and retry handling, merged June 22, 2026
Tracked Shopify topics	39 allowlisted topics
Target storage	1 event ledger plus 12 resource snapshot tables
Application scope	Backend only; no frontend code changed

Scope boundary

The documentation describes the behavior introduced or completed by PR #22 and PR #23. Existing Shopify connection APIs and unrelated database features are mentioned only where the event pipeline reads them.

What the system does

The backend consumes Shopify events from a Google Cloud Pub/Sub subscription. It normalizes Shopify headers, resolves the owning organization, selects a resource-specific handler and service, validates the event against the allowlist, and writes two records to that client's Shopify Supabase database:

1. An immutable-style event ledger row in shopify_events, updated only when the same event identifier is delivered again.
2. A current resource snapshot in a module table such as shopify_orders or shopify_customers.

Failures are copied into the application database before the Pub/Sub message is acknowledged. A separate replay command retries due records with bounded exponential backoff.

Key design decisions

• Green-only event scope. A single allowlist is used for webhook registration, routing, and parser validation. Events outside that set are not processed.
• Per-client data isolation. The normal path resolves a Shopify database and encrypted credential for the organization. Each client can point to a separate Supabase project.
• Idempotent writes. Event and snapshot writes use PostgreSQL ON CONFLICT DO UPDATE with stable composite keys.
• Module ownership. Orders, customers, products, refunds, returns, and the other resources have separate handler and service classes while sharing parser and storage infrastructure.
• Durable failure handoff. A failed event is acknowledged only after the failure envelope and traceback have been written to the application database. If that write fails, the Pub/Sub message is negatively acknowledged.
• Soft deletion. Shopify delete topics mark a resource snapshot with is_deleted = true; the pipeline does not physically delete target rows.

Read this next

• System architecture explains the VM, Pub/Sub, application database, and client Supabase boundary.
• Event processing traces header extraction, routing, parsing, acknowledgement, and persistence.
• Database CRUD is the operation-by-operation database reference.
• Schema reference lists every table, key, index, and lifecycle field.
• Client onboarding explains how a new client database is configured and provisioned.
• Retry and recovery covers failure states, backoff, and operational gaps.
• PR #22 and PR #23 maps the two deliveries to concrete files and behavior.

Current implementation limits

The pipeline is functional, but four boundaries are important for operators and future development:

• Only customer create, update, and delete events receive field-level dataclass parsing. Other tracked events are validated and normalized, then retain the original JSON as parsed_payload.
• The retry command exists, but no scheduler is included in these PRs. Production must invoke it through cron, systemd, a job runner, or another scheduler.
• Missing handlers and missing or duplicate ShopifyConnection records currently return without raising. The Pub/Sub message is therefore acknowledged and no SubscriberFailedEvent is created.
• SHOPIFY_SUPABASE_DATABASE_URL is an all-organizations override. Leaving it set bypasses per-client database resolution and should be limited to controlled testing or a deliberate single-target deployment.