Delayed automations and rules

Resolved

Degraded performance

Started over 4 years agoMarch 29, 2022Lasted about 15 hoursMarch 29, 20225:11 PMMarch 30, 20228:08 AMUTC

Affected

Rules

Updates

Resolved
March 30, 2022 at 8:08 AMUTC
Resolved
March 30, 2022 at 8:08 AMUTC
This incident has been resolved.
Monitoring
March 29, 2022 at 9:34 PMUTC
Monitoring
March 29, 2022 at 9:34 PMUTC
We've implemented a fix that has resolved the underlying problem and is allowing us to clear the backlog more quickly.
Identified
March 29, 2022 at 7:26 PMUTC
Identified
March 29, 2022 at 7:26 PMUTC
We've identified the underlying issue that led to degradation of our automation system and have begun remediation. At this time, new incoming events are being processed close to real time, and we still have a backlog of events to process.
Investigating
March 29, 2022 at 5:11 PMUTC
Investigating
March 29, 2022 at 5:11 PMUTC
Rule executions and sequence automation are currently delayed up to 3 hours. No data has been lost. We're investigating the underlying cause.

- Delayed automations and rules – Incident details