Subscriber runbook

This runbook covers the long-running VM process and the supporting replay command. It assumes backend dependencies and Django migrations are already installed.

Required runtime configuration

At minimum, the subscriber process needs:

export PROJECT_ID='full-google-cloud-project-id'
export SUBSCRIPTION_ID='shopify-events-sub'
export GOOGLE_APPLICATION_CREDENTIALS='/secure/path/service-account.json'
export DB_CREDENTIAL_FERNET_KEY='deployment-secret'

The Google identity needs permission to consume the configured subscription. The external publisher/gateway separately needs permission to publish to the topic.

Preflight checks

python manage.py check
python manage.py showmigrations subscriber
python manage.py migrate --noinput

Confirm subscriber.0001_initial is applied. Then verify at least one client target with:

python manage.py provision_shopify_storage \
  --org-id 123 \
  --shop-domain client-store.myshopify.com

Start the subscriber

python manage.py run_subscriber

Expected startup output includes the full subscription path:

Listening on projects/<project-id>/subscriptions/<subscription-id>

The process should be supervised by the VM's service manager. SIGTERM and SIGINT request a clean stop and cancel the active streaming future.

Recommended process split

Run the web API and subscriber as separate processes from the same code release:

Process	Command	Responsibility
Web API	Existing Gunicorn command	Connection APIs, direct HMAC webhook endpoint, application requests
Event worker	python manage.py run_subscriber	Streaming Pub/Sub consumption and client Supabase writes
Retry job	python manage.py retry_failed_events --limit 50	Periodic durable-failure replay

Smoke test sequence

1. Publish a controlled order event with a unique webhook ID and the client's permanent shop domain.
2. Confirm the subscriber logs a dispatch for orders.create.
3. Query client Supabase for one matching shopify_events row.
4. Query shopify_orders for the matching Shopify resource ID.
5. Redeliver the same webhook ID and confirm row counts do not increase for the composite keys.
6. Publish a controlled malformed payload and confirm a pending SubscriberFailedEvent is created.
7. Correct the root cause, run retry_failed_events, and confirm the row becomes resolved.

Log interpretation

Log text	Meaning
Listening on ...	Streaming pull started
Dispatching ... resource_type=...	Handler and parser succeeded; persistence is beginning
No handler registered	Source/topic is not routed; message will be acknowledged
No ShopifyConnection	Shop domain cannot resolve; message will be acknowledged
Multiple ShopifyConnections	Domain is ambiguous; message will be acknowledged
Stored subscriber dead-letter event id=...	Failure is durable and Pub/Sub will be acknowledged
Failed to persist subscriber dead-letter event; nacking	Application DB could not store the failure; Pub/Sub redelivery requested
Subscriber crashed, restarting in ...	Streaming worker-level backoff is active

Troubleshooting

Subscriber starts against the wrong GCP project

Verify PROJECT_ID is the complete project ID, not a truncated prefix or project display name. The command constructs the subscription resource path directly from this value.

Authentication fails

Check the active Application Default Credentials source and confirm the service account has Pub/Sub Subscriber access to the subscription. Keep the JSON file outside tracked source files.

Events are acknowledged but absent from Supabase

Check logs for missing handlers or Shopify connections. Those conditions currently return without raising and are therefore acknowledged without durable failure capture.

If dispatch logs appear, inspect SubscriberFailedEvent for target configuration, credential, network, parser, or SQL exceptions.

All clients write to one database

Unset SHOPIFY_SUPABASE_DATABASE_URL. That variable intentionally takes precedence over per-organization configuration.

Provisioning reports missing configuration

Confirm the organization owns a non-deleted Database with type = shopify or database_name = shopify, and that its DatabaseCredential has non-empty host, db_name, and username fields.

Password decryption fails

The deployed DB_CREDENTIAL_FERNET_KEY must be the same key used by DatabaseCredential.set_password(). Rotating the key requires re-encrypting stored passwords.

Retries remain pending

Check next_retry_at. --event-id does not bypass the due-time predicate. Also confirm a scheduler is actually invoking the command.

Direct webhook smoke test

The Django endpoint accepts only POST, requires X-Shopify-Hmac-Sha256, and returns:

Response	Meaning
200 EVENT_RECEIVED	HMAC and JSON valid; ShopifyEvent inserted in application DB
400 Bad Request	Invalid JSON
401 Unauthorized	Missing or invalid HMAC
404 Not found	No Shopify connection for URL organization
405 Method Not Allowed	Non-POST request

Remember that this endpoint stores only the application ShopifyEvent; it does not exercise Pub/Sub or per-client Supabase persistence.