On December 21, 2022, at approximately 09:30 UTC, we noticed unusually elevated API errors within Google Single Tenant (GST), which can cause degraded service with Anaplan connect integration workspaces.

An initial investigation revealed a decrease in request rate to the API service at 01:00 UTC. The decrease was likely due to a delay in saving model tasks, and it resulted in our API service running with fewer threads. Subsequently, our API service began returning timeout errors (status 524) as it could not process some incoming requests.

Our engineering team restarted the API service to clear the queue and allow requests to be processed again. That resolved the issue at 10:45 UTC.

To limit the likelihood of this issue recurring, we have added extra detection measures to the API service to allow immediate mitigations. At the same time, we continue to work with our vendor to resolve the issue with occasional delays to saving models.

We sincerely apologize for any impact this situation had on your business operations. We understand the disruption these issues can cause to your business and users, and we are continuously strengthening our systems and procedures to avoid similar issues in the future. If you have any further questions or concerns, please contact Anaplan Support.

Thank you for your patience during this situation and thank you for being an Anaplan customer.