On April 29, 2026, at 09:12 UTC, our internal monitoring and validation checks detected workflow failures in eu3: Cloud - Europe. Customers in this region were unable to execute workflows during the incident window.
Root cause
During a planned update, a misconfiguration altered the workflow routing settings, causing workflow requests in eu3 to be directed incorrectly and resulting in service degradation.
Recovery
Our engineering team identified the issue and took immediate action to resolve it. We reverted the incorrect configuration through a controlled release, removed any residual routing settings, and validated service recovery through internal testing. The issue was fully resolved and service was restored by 10:01 UTC.
Corrective and preventative actions
We are implementing the following actions to prevent recurrence:
We are implementing clearer controls and procedures for configuration changes. This reduces the risk of configurations changes being inadvertently applied.
We are implementing clearer visual labeling to distinguish these changes. This reduces the likelihood of unintended modifications being applied.
We are implementing workflow monitoring and smoke test improvements, including faster health checks. This helps us detect and recover from workflow issues more quickly.
We are implementing stronger smoke test standards and tracking across teams. This improves early detection and drives faster follow-through on test failures.
We apologize for the impact this issue has had on your operations. We are committed to the improvements outlined above to prevent similar disruptions. If you have questions or concerns, please contact our Support team.