On December 4, 2024, at 11:55 UTC, our internal monitoring identified a CloudWorks™ integration issue. This prevented integrations from completing in Anaplan Data Center - U.S. West.   

During the initial investigation, our team found that a CloudWorks™ subcomponent was experiencing connectivity issues with an upstream component. This caused integrations to fail. However, our monitoring system quickly found the problem and fixed it. The auto-recovery system fixed the connection problem at 12:01 UTC. By 12:27 UTC, the integration backlog was cleared and normal service resumed.     

Further investigation identified that an earlier patch to an upstream service contributed to the connectivity issue. Patching activities are non-service-disrupting and were completed successfully in all regions. However, for Anaplan Data Center- U.S. West, we found that a specific automated operation was triggered during patching. This operation only happens when the component being patched is under a lot of load. For CloudWorks™ this was a result of the top-of-the-hour integration load. The action that happened stopped writes for a short time while the upstream service fixed itself. This caused a short interruption to connectivity. Once the operation had completed, CloudWorks™ auto-recovery process initialized, and connectivity and service was restored.   

We recognize that even minor service interruptions can cause frustrations. We have completed an in-depth review of the issues that led to this disruption. As a result, we have made the patching operation better. The automated trigger will no longer stop requests from being processed while it runs. We have also set up a priority working group to improve the resiliency of CloudWorks™. Several improvements are being made, including a review of how queries are optimized for the CloudWorks™ service. This can make integration processing more efficient and improve application recovery after problems.   

We deeply apologize for any impact this issue may have had on your business operations. We are continuously strengthening our systems and procedures to ensure we avoid future disruptions to your business and users. If you have any further questions or concerns, please contact Anaplan Customer Care. Thank you for your patience during this situation and thank you for being an Anaplan customer.