Platform Alerts
Incident Report for Anaplan
Postmortem

On May 23, 2024, at 12:14 UTC, we started to receive errors indicating that models were inaccessible in Anaplan Amazon Cloud – Europe.

Our team established that a change to authentication interactions within the platform had been deployed around 10:35 UTC. This change was identified as the probable cause of the incident and was rolled back. The rollback was completed by 12:47 UTC. All local testing and associated monitoring were confirmed as healthy, and the incident was closed at 12:56 UTC.

At 13:10 UTC, we received feedback that some customer models remained inaccessible. The incident was re-opened, and we found that a workspace migration had automatically started once the change had been reverted. This migration was required to move workspaces to fresh components; however, workspaces were inaccessible until the migration was completed.

We closely monitored the migration and by 13:24 UTC only two workspaces remained in progress. These workspaces were reviewed and at 13:43 UTC it was decided that manual intervention was required to expedite the process. The migration job was manually restarted and by 14:03 UTC all workspaces were available. We continued to monitor the platform to ensure no further issues were identified. The incident was resolved at 14:31 UTC.

To prevent a recurrence of this issue, we are updating our approach when implementing these types of changes. Our aim is to make the process more efficient, as well as, improving the rollback mechanism to avoid disruption in future. In addition, we are implementing additional alerting and monitoring for this scenario to ensure we know about this even earlier if the same scenario were to arise in future.

We apologize for any impact this issue may have had on your business operations. We are continuously strengthening our systems and procedures to ensure we avoid future disruptions to your business and users.

If you have further questions or concerns, please visit the Anaplan Support website. Thank you for your patience during this situation and thank you for being an Anaplan customer.

Posted May 30, 2024 - 19:13 UTC

Resolved
We have confirmed that the issue is now resolved.

We deeply apologize for any impact this issue may have caused. We appreciate your patience and partnership as we worked through this issue.

We will follow up within 7 business days with a detailed root cause analysis (RCA) that will be shared on our Status Page. If you have any question or concerns, please do not hesitate to contact us at Anaplan Support.
Posted May 23, 2024 - 14:47 UTC
Monitoring
Service has now been restored; you should now be able to resume normal activities.

We will continue to monitor the platform to ensure no additional issues arise. If you have any questions, concerns, or continue to experience issues, please do not hesitate to contact Anaplan Support. We will provide a final update to you when we consider this situation fully resolved.
Posted May 23, 2024 - 14:31 UTC
Update
We are aware that two models remain unavailable. The remaining migration jobs are ongoing and we are viewing options to expedite the recovery of these models.

Currently, we do not yet have a time to resolution. We will provide further updates in 30 minutes or upon resolution.
Posted May 23, 2024 - 13:56 UTC
Identified
We are aware that some models are currently unavailable. This is due to an internal workspace migration triggered by the recent release rollback.

Currently, we do not yet have a time to resolution. We will provide further updates in 30 minutes or upon resolution.
Posted May 23, 2024 - 13:25 UTC
This incident affected: Anaplan Amazon Cloud - Europe.