PR #22 and PR #23 delivery map

The complete implementation is the result of two dependent pull requests. PR #22 establishes the end-to-end Shopify execution path. PR #23 makes that path maintainable across modules and operationally recoverable.

Delivery timeline

Pull request	Merge	Scope	Result
#22 - Complete Shopify subscriber execution	June 16, 2026 into redash_api	Green-only topics, parser/router, Supabase storage, webhook middleware path	Working Pub/Sub-to-Supabase execution foundation
#23 - Add Shopify storage provisioning and retry handling	June 22, 2026 into redash_api	Module handlers/services, schema provisioning, headers, durable failures and replay	Maintainable multi-module pipeline with recovery controls

PR #23 is not standalone architecture. It extends and refactors the parser and storage foundation introduced by PR #22.

PR #22: execution foundation

PR #22 contains four commits ending at f127f80 and changes 18 files.

Topic registration and allowlist

• Added shopify/topics.py with the 39 green-only topics.
• Made webhook registration consume the same allowlist.
• Added slash/dot normalization helpers.
• Added tests rejecting non-green events.

Generic parser and storage

• Added subscriber/parsers/shopify/events.py with ParsedShopifyEvent and resource-table mapping.
• Added the generic ShopifyEventHandler path.
• Added ShopifyEventStorage for per-organization Supabase/PostgreSQL writes.
• Wrote both a normalized event row and a latest resource snapshot.
• Added deterministic event IDs and idempotent conflict updates.

Storage contract and credential handling

• Made the handler storage contract explicit through BaseHandler.store().
• Reused existing Database and DatabaseCredential records to resolve per-client destinations.
• Correctly decoded URL-encoded username/password components from a Supabase database URL during the implementation stage.

PR #23 later replaced the original URL-based credential handling with direct field-based DSN construction while retaining the same per-organization model.

Direct webhook authentication path

• Kept the Shopify callback exempt from global JWT because Shopify authenticates with HMAC.
• Added unauthenticated prefixes for Shopify webhook and OAuth callback routes.
• Added middleware regression coverage.

The direct endpoint still verifies X-Shopify-Hmac-Sha256; exempting it from JWT does not make it unauthenticated.

PR #23: modularity, onboarding, and recovery

PR #23 is commit 8ad4b0b, changing 41 files with 1,045 additions and 94 deletions.

Separate handlers and services

Added one handler and one service per resource module:

• Orders
• Checkouts
• Carts
• Products
• Customers
• Customer groups
• Payment terms
• Refunds
• Selling plan groups
• Returns
• Reverse deliveries
• Reverse fulfillment orders

Each handler owns module dispatch. Each service validates its resource_type. Shared parsing and SQL remain centralized.

Client storage provisioning

• Added ShopifyEventStorage.provision_schema().
• Added reusable table/index creation helpers.
• Added python manage.py provision_shopify_storage for onboarding.
• Changed the raw event table name to the explicit shopify_events ledger.
• Provisioned all 12 snapshot tables up front while keeping on-write schema checks.

Durable failures and replay

• Added SubscriberFailedEvent plus migration subscriber.0001_initial.
• Added failure envelope and traceback capture.
• Changed message handling to acknowledge after durable failure storage and negatively acknowledge if failure storage itself fails.
• Added retry_failed_events with due-time filtering, bounded backoff, and terminal state.

Header extraction

• Added Shopify header constants and case-insensitive extraction from Pub/Sub attributes.
• Added raw-body wrapping when attributes identify a Shopify message.
• Preserved original and merged headers for diagnostics and replay.
• Added event, shop, source, and Pub/Sub message identifiers to the enriched envelope.

Combined code path

File ownership map

Layer	Primary files	Responsibility
Topic policy	shopify/topics.py, shopify/webhooks.py	Green-only registration and normalization
Direct webhook	shopify/views/webhook.py, config/settings.py	HMAC ingress and JWT middleware exemption
Subscriber lifecycle	subscriber/management/commands/run_subscriber.py	Pub/Sub stream, envelope enrichment, dispatch, ack/nack
Routing	subscriber/router.py	Source/event to resource handler mapping
Parsing	subscriber/parsers/shopify/events.py	Validation, metadata, IDs, table selection, deletion flag
Module handlers	subscriber/handlers/shopify/*.py	Resource-specific entry points
Module services	subscriber/services/shopify/*.py	Resource ownership validation and persistence delegation
Target storage	subscriber/storage/shopify.py	Database resolution, DDL, event and snapshot upserts
Durable failure	subscriber/failures.py, subscriber/models.py	Failure ownership and persistence
Operations	provision_shopify_storage.py, retry_failed_events.py	Onboarding and replay commands
Verification	subscriber/tests/test_*.py, schema/tests/test_middleware.py	Topic, parser, router, storage, header, retry, and middleware behavior

Review-driven corrections

The PR #22 sequence includes follow-up commits that clarified the handler storage contract, corrected Supabase credential decoding, and fixed the global JWT middleware conflict discovered during live deployment. PR #23 had no GitHub review comments recorded after opening.

Historical live validation

PR #22's recorded live test deployed head f127f80 to the VM, passed Django checks and migrations, accepted a signed orders/create webhook through HTTPS, inserted one application ShopifyEvent, and verified one Supabase event row plus one order snapshot. Temporary rows from that test were removed.

PR #23's integration run used a real GCP Pub/Sub topic/subscription and Supabase target. It demonstrated:

• Successful order event persistence.
• A customer event that encountered a PostgreSQL deadlock, entered SubscriberFailedEvent, and resolved through replay.
• A malformed payload retained as pending after a failed replay.
• A target connection failure retained durably and later resolved after restoring the DSN.

The PR #23 test records were intentionally retained during the debugging session, unlike the PR #22 smoke data.

Net result

Together, the two PRs deliver a backend-only Shopify pipeline that can:

• Register exactly the selected webhook topics.
• Consume wrapped or raw-body Pub/Sub messages.
• Resolve organizations and per-client Shopify databases.
• Route every tracked resource to its own handler and service.
• Persist idempotent event history and current snapshots.
• Provision a new client's standard schema.
• Capture parse, code, lock, credential, network, and SQL failures.
• Replay due failures without requiring the original queue message.

The verification page separates automated coverage, live evidence, and remaining operational risks.