Delayed Conversion Processing
- investigating
We are currently investigating an issue causing delays in conversion processing. Data is being captured, but reporting may lag behind real-time.
- monitoring
A fix has been implemented, and we are monitoring the system as processing returns to normal. We have begun backfilling any delayed conversions to ensure your reporting is 100% accurate. No data was lost during this incident.
- resolved
This incident has been resolved.
- postmortem
On February 3rd, starting at approximately 10:00 AM EST, a misconfiguration during a feature release impacted the processing speed of click ingestion. While clicks were being redirected at the usual speed, the process that allows for storing clicks and attributing conversions was delayed for roughly 90 minutes, until 11:30 AM EST. We have since isolated the conversions that were potentially dropped due to this delay. At 2:20 PM EST, we began "replaying" these events to ensure all data is accurately captured and recorded; this process is currently underway and is expected to be completed by 11:00 PM EST tonight. **What was not impacted:** Our endpoints for Clicks, Impressions, and Conversions continued to respond at standard speeds throughout the event. **What was possibly impacted:** Because of the processing delay, some Conversion, Payout, and Revenue caps may have exceeded their allowed limits. Additionally, the detection of tiered commission events may have been delayed in certain cases. **What was definitely impacted:** Customers utilizing our Firehose, as well as those using "New Conversion Registered" and "New Event Registered" notifications or webhooks, likely saw elevated error rates starting around 9:30 AM EST. During this window, some events may not have fired, or were delivered with missing properties. As these specific services \(Firehose and Webhooks\) cannot always be automatically replayed, we will be reviewing these instances on a case-by-case basis. If you have been affected by these specific service disruptions, please contact our support team so we can work with you to address your data needs.