On December 9, 2024, at 01:55 UTC, our internal monitoring detected an interruption in our CloudWorks™ services. During this period, integration processes in Anaplan Google Cloud – Dedicated environment were unable to be completed.
Our team reviewed the source of the alerts and found that a CloudWorks™ subcomponent was experiencing connectivity issues with an upstream component. Due to this issue, integrations were unable to run. Our engineers initially performed a rolling restart of the CloudWorks™ service, but the issue persisted. The team then conducted a deep dive into the service and upstream components, and everything appeared to be functioning correctly. The service self-healed, and integrations started processing normally. Full service was restored at 03:34 UTC.
After further investigation, we identified two critical processing queues that had stopped working. This caused memory usage in an upstream component to gradually increase. Although memory was increasing, the component appeared to be functioning correctly. When it reached its maximum capacity at 03:30 UTC, the component automatically restarted. This restored normal operations and resolved the integration processing disruption.
We are taking proactive steps to improve our CloudWorks™ services reliability. A dedicated working group has reviewed this issue, with a particular focus on how our service interacts with upstream dependencies. To prevent a recurrence, we removed the failed processing queues. Memory utilization has remained stable since we implemented this change. Also, we are implementing enhanced alerting mechanisms and conducting a comprehensive review of our integration processes. Our goal is to increase system efficiency and strengthen our ability to handle potential upstream challenges more effectively.
We deeply apologize for any impact this issue may have had on your business operations. We are continuously strengthening our systems and procedures to ensure we avoid future disruptions to your business and users.
If you have any further questions or concerns, please contact Anaplan Customer Care. Thank you for your patience during this situation and thank you for being an Anaplan customer.