Platform Alerts
Incident Report for Anaplan
Postmortem

On May 15, at approximately 08:57 UTC, we identified an issue with our CloudWorks service that impacted customers in Anaplan Data Centers - US East, US West, Germany, and the Netherlands. This issue caused CloudWorks integrations to take longer than expected to complete.

Initial investigations identified that integrations were being picked up but were not progressing to completion. We restarted a sub-component which initially had a positive impact; however, the issue continued to persist. Additional investigation indicated that the issue was linked to a recent change that was completed on Thursday, May 11, 2023. As a result, the change was rolled back. However, this did not resolve the underlying issue that was impacting the CloudWorks integrations.

Further investigation into upstream components identified high resource utilization. Although the resource utilization had not reached capacity, we identified that there was an unbalance of load between the underlying sub-components. As a result, we isolated the impacted sub-component to restart the service.  This resolved the underlying issue, and normal service levels resumed at 20:03 UTC. Though, due to the duration of the incident, some customers may have experienced some backlog jobs taking up to an additional 8 hours to complete.

We are currently reviewing the change and its impact on the underlying component, to identify what caused this unbalance to occur. We have also identified an initiative to improve the resource usage and steps to tune our monitoring to avoid a similar issue from reoccurring.

We deeply apologize for the impact this had on your business operations. We understand the disruption these issues can cause to your business and users, and we are continuously strengthening our systems and procedures to avoid similar issues from happening again in the future.

If you have any further questions or concerns, please contact Anaplan Support. Thank you for your patience during this situation and thank you for being an Anaplan customer.

Posted May 19, 2023 - 07:55 UTC

Resolved
Service has now been restored; you should now be able to resume normal activities. As a result of this incident, there is a backlog of jobs that need to process, so you may experience up to an 8-hour delay in backlog job completing.

We deeply apologize for any impact this issue may have caused. We appreciate your patience and partnership as we worked through this issue.  

We will follow up within 7 business days with a detailed root cause analysis (RCA) that will be shared on our Status Page. If you have any questions or concerns, please do not hesitate to contact us at Anaplan Support.
Posted May 15, 2023 - 20:36 UTC
Identified
We are progressing mitigation steps to alleviate the issue and are focused on restoring service as quickly as possible. Currently, we do not yet have a time to resolution. We will provide further updates in 30 minutes or upon resolution.
Posted May 15, 2023 - 20:09 UTC
Update
We have identified mitigation steps to alleviate the issue. We are working on progressing these steps. Customers will notice integrations with progress, but at a degraded service at this time. Will will provide a further update in 30 mins.
Posted May 15, 2023 - 19:24 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, we do not yet have a time to resolution. We will continue to provide updates every 30 minutes as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 17:59 UTC
Update
Thank you for your patience. Our engineers are still investigating the root cause of the issue.
Customers may experience slow execution of the Integration task.
We will continue to provide updates as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 16:44 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, we do not yet have a time to resolution. We will continue to provide updates every 30 minutes as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 15:19 UTC
Update
Thank you for your patience. Our engineers are still investigating the root cause of the issue.
Customers may experience slow execution of the Integration task.
We will continue to provide updates every 30 minutes as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 14:01 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, we do not yet have a time to resolution. We will continue to provide updates every 1 Hour as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 13:14 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, we do not yet have a time to resolution. We will continue to provide updates every 30 minutes as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 12:08 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, engineering teams are we are working through the mitigation process, and we will send you further update shortly.
Posted May 15, 2023 - 11:08 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, we do not yet have a time to resolution. We will continue to provide updates every 30 minutes as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 10:15 UTC
Update
Thank you for your patience as we continue to investigate this issue. Currently, we do not yet have a time to resolution. We will continue to provide updates every 30 minutes as we work to resolve this issue as quickly as possible.
Posted May 15, 2023 - 09:36 UTC
Investigating
We are currently investigating an issue impacting customers’ ability to run Cloudworks integrations.

We are working to resolve this issue as quickly as possible and will provide updates every 30 minutes or upon resolution.
Posted May 15, 2023 - 09:08 UTC
This incident affected: Anaplan Data Center - US East, Anaplan Data Center - US West, Anaplan Data Center - Netherlands, and Anaplan Data Center - Germany.