Designing a reliable event pipeline
A fictional case study in retries, duplicate events, and recovery.
The scenario
A fictional operations team receives repeated alarms from connected devices. It needs a useful incident stream without losing the original signals.
The design to explore
Start with durable ingestion, then separate correlation from incident creation. Give each event a clear identity and make downstream handling safe to repeat.
The important tradeoff
A longer correlation window can group more related signals, but it can also delay a useful alert. Choose the policy around the product’s response-time needs.
How I would validate it
Replay a known event set. Inject duplicates, delayed messages, and processing failures. Measure incident accuracy, end-to-end delay, backlog recovery, and the ability to trace a decision.
What the final write-up should include
An architecture diagram, event examples, failure behavior, the options considered, and measured results from a reproducible test. No company implementation details are required.