On December 30, 2023, at 10:20 UTC, we identified an issue that caused some new authentication requests to fail when logging on to Anaplan Google Cloud Public – Canada. The impacted authentication requests would have been successful upon retry. Customers already logged on to the Anaplan platform were not affected.
Initial investigation identified a connectivity issue on a backend component of the authentication service. Further investigations revealed that the backend component had breached its threshold causing this incident. This was a result of an internal service’s requests not timing out and becoming stuck. To mitigate the issue, a rolling restart to the authentication service was performed, and the issue was resolved at 10:30 UTC.
To avoid a reoccurrence of this issue, we are implementing a timeout mechanism within the authentication service to prevent these requests from becoming stuck. We have also enhanced our monitoring to ensure we identify any potential spikes to the threshold much quicker in the future to prevent customer impact.
We apologize for any impact this issue might have had on your business operations. We’re continuously strengthening our systems and procedures to ensure we avoid future disruption to your business and users.
If you have further questions or concerns, please visit our Anaplan Support website. Thank you for your patience during this situation and thank you for being an Anaplan customer.