On April 22 at 05:22 UTC, Anaplan engineers were alerted to a network issue by our automated monitoring systems and began investigating immediately.
The investigation determined that our Amsterdam data center was experiencing packet loss on the network connection to our Washington data center. Once we identified the backbone connection between Amsterdam and Washington was at fault, Anaplan engineers switched the network connection to our failover link at 08:00 UTC, allowing traffic to route through a different network circuit.
The issue with the network connection was caused by maintenance being performed by our external providers. While we had earlier been informed of configuration work scheduled by our data center provider, it was classified as "no impact" maintenance, and we have since followed up with our data center and network providers to ensure they investigate the issues that occurred.
As additional preventive measures, Anaplan is improving its automated alerting to more quickly identify network degradation. We are also improving the tooling used to debug network issues during an outage such as this one. The aim is to more promptly distinguish a network issue from an application issue, which will lead to quicker resolution. A final change we are investigating is the potential for an automated switchover to the failover network circuit when connection issues arise.
We understand the impact to your experience when these events occur, and we are working diligently to improve our systems and processes to mitigate further recurrence of these issues. We appreciate your patience, and as always thank you for being a valued Anaplan customer.