Skip to main content

System architecture

The Shopify execution spans an external webhook gateway, Google Cloud Pub/Sub, a long-running Django management command on a VM, the backend application database, and a client-specific Supabase/PostgreSQL database.

Component topology​

Control plane and data plane​

The architecture deliberately separates operational metadata from client event data.

BoundaryDatabase responsibilityTables read or written by this work
Application PostgreSQLOrganization ownership, Shopify connection metadata, target database credentials, direct webhook audit rows, durable failuresShopifyConnection, Database, DatabaseCredential, ShopifyEvent, SubscriberFailedEvent
Client Shopify SupabaseRaw normalized event history and latest resource snapshots for one client's Shopify integrationshopify_events and 12 shopify_* resource tables

The standard multi-client path selects the target database by organization. A global SHOPIFY_SUPABASE_DATABASE_URL environment value takes precedence and sends every event handled by that process to one database.

Runtime placement and the VM​

The VM is the always-on execution host for python manage.py run_subscriber. The Django web process does not need to hold the Pub/Sub stream open.

The worker:

  1. Reads PROJECT_ID and SUBSCRIPTION_ID.
  2. Opens a streaming pull with google.cloud.pubsub_v1.SubscriberClient.
  3. Dispatches callbacks to the event processing path.
  4. Restarts the streaming pull after process-level failures using delays of 5, 10, 20, 40, then 60 seconds.
  5. Handles SIGTERM and SIGINT by cancelling the streaming future and exiting cleanly.

The VM does not own event data. Durable state remains in Pub/Sub, the application database, and client Supabase, so the worker process can be restarted or redeployed.

Two webhook ingress paths​

The codebase currently exposes two related but different paths.

Registered gateway and subscriber path​

shopify.webhooks.register_webhooks() registers all allowlisted topics against SHOPIFY_WEBHOOK_CALLBACK_URL. Its default points to an external gateway. That gateway is expected to publish the body and Shopify headers to Pub/Sub. This is the path that reaches module handlers and client Supabase.

Direct Django webhook path​

POST /api/shopify/webhook/<org_id>/ verifies X-Shopify-Hmac-Sha256 using the organization's api_secret_key, parses the JSON body, and inserts one ShopifyEvent row into the application database. The endpoint is excluded from global JWT enforcement because Shopify authenticates through HMAC.

This direct endpoint does not publish to Pub/Sub and does not write the client Supabase resource tables. It is a separate audit/ingress path and was used in PR #22's live VM smoke test.

Organization and database resolution​

The subscriber first maps shop_domain to ShopifyConnection.myshopify_domain. That yields organization_id, which is carried as OrgContext through the handler and storage layers.

Target resolution then follows this order:

  1. Use SHOPIFY_SUPABASE_DATABASE_URL when it is non-empty.
  2. Find the first non-deleted Database for the organization whose type equals shopify, case-insensitively.
  3. If absent, find the first non-deleted Database whose database_name equals shopify.
  4. Load its one-to-one DatabaseCredential and decrypt the password with DB_CREDENTIAL_FERNET_KEY.
  5. Construct a PostgreSQL DSN with SHOPIFY_SUPABASE_SSLMODE, defaulting to require.

Missing database rows, missing credentials, and incomplete host/database/user fields raise StorageConfigurationError and enter the durable failure path.

Client isolation model​

The intended production layout is one Supabase project/database per client application. For Shopify, each organization's Database row points to that client's Shopify Supabase project. The same table names can exist in every project without mixing tenant data.

Composite keys still include organization_id and shop_domain. This provides defense in depth, supports controlled shared-database testing, and prevents one shop's duplicate event identifier from overwriting another shop's row.

Isolation prerequisite

Do not set the global Supabase URL on a worker intended to route multiple organizations independently. The environment override wins before the organization-specific Database lookup.

Frontend relationship​

Neither PR changed the frontend repository. The pipeline operates behind existing backend connection and database configuration flows. Any future onboarding UI should create or update Database and DatabaseCredential records, then trigger the provisioning command or an equivalent backend endpoint.