Concurrency Conflicts
Overlapping claim, redeem, and release requests for the same campaign and voucher must resolve to one processing order.
Design and Implementation Visual · Issue #538
Event-Sourced Promotion Voucher Campaign
`commerce/event-sourced-promotion-voucher-campaign` records every change to campaigns and vouchers in sequence in a PostgreSQL event store. A basic CRUD implementation cannot reliably handle concurrent requests, retries, read-model lag, or operational recovery. This example shows how to solve those problems by separating command processing from read-model updates.
Command Processing
Idempotency check, aggregate rehydration, expected-version check, response storage
System of Record for State Changes
Stream-head lock, append fence for global sequence allocation, immutable events
Read-Model Updates
Lease, fencing token, deduplication, checkpoint, read model
Failure Recovery
Failed-event replay, generation-based rebuild, validation, active-generation swap
Example Concept
The example issues, redeems, and releases vouchers within one promotion campaign while addressing four production concerns.
Retry After a Lost Response
When a client retries after losing a response, the system must return the previous result without storing duplicate events.
Projection Lag
A completed write does not guarantee an up-to-date read model. The API must report its current processing position and lag.
Online Recovery
Poison events, projector restarts, full rebuilds, and personal-data erasure must be handled without taking the service offline.
Create and Activate CampaignRecord budget, duration, and issuance policy in the event stream.
→
Claim, Redeem, and Release VoucherValidate business rules and the expected version before changing state.
→
Query, Audit, and RecoverRead and operational features use the same event history.
A discount voucher is a familiar business concept, but a production system must also
account for optimistic concurrency control, idempotency, eventual consistency,
projection rebuilds, and personal-data erasure. This example shows how these technical
concerns connect within one business flow.
Design Direction
Instead of coordinating simultaneous writes across multiple tables, the system records every state change as an event first. Current state and read models are derived from that event stream.
Limits of Basic CRUD
Concurrent UpdatesA later write can overwrite a state change that was processed first.
Dual WritesEither the business state or the query table can be stored without the other.
Non-Idempotent RetriesA lost response can cause the same command to run twice.
Direct History EditsErasing only personal data while preserving the audit trail is difficult.
Design Principles
Events FirstUse the PostgreSQL event stream as the system of record for state changes.
Atomic StorageStore events and idempotency results in the same transaction.
Asynchronous ProjectionRebuild read models from events and expose the current projection position.
Generation-Based RebuildBuild a new projection generation separately, validate it, and then make it active.
Sensitive-Data SeparationDo not retain raw user details or voucher codes in immutable events.
Implementation Direction
Separate business rules from storage mechanisms while making transaction boundaries and event-processing positions explicit in code.
Aggregate and Reducer
Replay events to restore current state, then validate the command and determine which new events to record.
Command Service
Perform the idempotency check, state rehydration, business decision, event storage, and final-response storage in one use case.
PostgreSQL Event Store
Record the stream lock, `append fence`, event payload, and global processing position in one transaction.
Lease and Checkpoint
Read stored events in batches, then deduplicate events, update the read model, and advance the checkpoint in one transaction.
Recovery and Observability
Provide failed-event replay, consistency checks, and generation-based rebuilds while measuring connection-pool usage and processing lag.
Snapshots optimize event replay; they are not the final source of data. If snapshot
metadata is invalid or its HMAC key version is unavailable, discard the snapshot and
replay the full event stream.
Run and Verify the Example
Start with the Testcontainers integration tests to verify the core behavior. To exercise the full flow, connect the application to PostgreSQL and send HTTP mutation and query requests in sequence.
Run the Integration Tests
Verify command processing, idempotency, read-model updates, and failure recovery without configuring a separate local PostgreSQL instance.
./gradlew \
:commerce-event-sourced-promotion-voucher-campaign:integrationTest \
--console=plain
Run the Application
Set the PostgreSQL data source and HMAC secret in environment variables. The default key included with the example is for local execution only and must not be used in production.
export SPRING_DATASOURCE_URL='jdbc:postgresql://localhost:5432/<database>'
export SPRING_DATASOURCE_USERNAME='<username>'
export SPRING_DATASOURCE_PASSWORD='<password>'
export VOUCHER_HMAC_ACTIVE_VERSION=2
export VOUCHER_HMAC_ACTIVE_KEY_BASE64='<base64-secret>'
./gradlew :commerce-event-sourced-promotion-voucher-campaign:bootRun
Every API request requires the `X-Workshop-Tenant` and `X-Workshop-Principal` headers.
State-changing requests also require `Idempotency-Key`. The `/operator/**` APIs must be
called from localhost and additionally require `X-Workshop-Operator-Secret`,
`X-Workshop-Guard`, and `X-Workshop-Operator-Role: OPERATOR`.
Create a Campaign
Create a campaign through the operator API with the common authentication headers, an idempotency key, and `If-None-Match: *`.
POST /operator/api/v1/campaigns
{ campaignId, startsAt, endsAt, capacity, perUserLimit, redemptionTtlSeconds }
Activate the Campaign
Send the `revision` from the create response as `expectedRevision` to detect whether another request changed the state first.
POST /operator/api/v1/campaigns/{campaignId}/activate
{ "expectedRevision": 0 }
Claim a Voucher
Convert the user reference to an HMAC-based pseudonym, then validate the overall issuance capacity and per-user limit in the aggregate.
POST /api/v1/campaigns/{campaignId}/claims
{ "userRef": "member-42" }
Check the Read Model
Send the stream position from the mutation response as the minimum required projection position. This ensures the read model includes the preceding change.
GET /api/v1/campaigns/{campaignId}
X-Min-Stream-Position: {position}
Check Recovery Status
If projection processing is delayed, retry only the GET request. If the entire read model diverges from the event history, rebuild it through the operator API and run the consistency check.
POST /operator/api/v1/projections/{projection}/rebuilds
GET .../rebuilds/{generation}
Command ResultUse `Idempotency-Replayed` and `X-Stream-Position` to determine whether the request was processed or a previous result was replayed.
Query Result`200` means the read model is current. For `202`, inspect `Retry-After` and `X-Projection-Lag`, then retry only the GET request.
Operational StateInspect `/actuator/health`, Prometheus metrics for lag, failed events, and connection-pool usage, along with rebuild status and audit records.
Processing Flow in the Code
Command processing, read-model updates, rebuilds, and security and operations are implemented as separate concerns. Every state change passes through the PostgreSQL event history.
Timing Gap Between Commands and the Read Model
When a mutation succeeds, the event and idempotency result are stored first. The read model becomes current only after its projection checkpoint advances through that event.
Event Stream
Read-Model Freshness
When read-model processing is delayed, the API returns `202 PROJECTION_PENDING`.
S
X-Stream-Position
0
P
X-Projection-Position
0
L
X-Projection-Lag
0
Implemented Structure
The following Kotlin classes implement the transaction scopes and state-change rules described above.
Event Storage
- Lock `StreamHeads` first and verify that the expected stream version matches the current version.
- Use `AppendFences` to assign gap-free global sequence numbers to committed events.
- Append events and update the stream head in the same transaction.
Command Idempotency
- Claim the idempotency key and store the final response within the command-service transaction.
- When the request fingerprint matches, return the previously stored final response unchanged.
- If a key required to restore the response is unavailable, fail closed and return `503`.
Read-Model Worker
- Renew a projection lease only after validating its owner and fencing token.
- Deduplicate events, update the read model, and advance the checkpoint in one transaction.
- Quarantine poison events for recovery instead of skipping them.
Read-Model Rebuild
- Make a newly built read model active only after validating it.
- Use revisions and fencing tokens to support cancellation and resumption and to reject writes from stale workers.
- Run the currently served read model independently from the generation being rebuilt.
Read-Model Projection Status
Read the event-stream position and read-model position together. If the requested position has not been projected, wait for the configured interval, then return `202 PROJECTION_PENDING` with lag information in the response headers.
Personal-Data Erasure and State Rehydration
Store raw user data in an erasable mapping to HMAC-based pseudonyms. Use a snapshot as the starting point for event replay only after it passes validation.
Technical Problems the Example Solves
The API responses and operational data make failure states explicit. Callers can interpret response codes and processing positions, while operators can recover through controlled retries and rebuilds.