On December 18, 2024, at 05:18 UTC, we became aware of an access issue for customers that have IP allowlist enabled. Customers that don’t have the IP allowlist enabled weren't impacted. The service disruption was observed across multiple regions: Anaplan Data Center - US East, Anaplan Data Center - US West, Anaplan Data Center – Germany, Anaplan Data Center – Netherlands, Anaplan Google Cloud Public - US East, Anaplan Google Cloud Public – Japan, Anaplan Amazon Cloud Public – US and Anaplan Amazon Cloud Public - Europe.
During our initial investigation, we attempted to resolve the issue through a standard restart of the impacted service, but this wasn't successful. Further investigation identified the problem may have been linked to a release that was implemented earlier that day. Part of the release was to update a configuration within the IP authorization system. We strategically decided to re-apply the previous configuration to the IP authorization system. This solution was applied successfully in one region to ensure its effectiveness. After confirming that users could log in without errors, we applied the solution to all impacted regions. At 07:17 UTC, access to the platform was fully restored across the affected regions.
Upon completing a thorough investigation, we identified that the configuration update to the IP authorization system had unexpectedly affected a downstream service's ability to verify user connections. This impacted the IP allowlist feature. While this update was intended to strengthen our IP authorization process, it caused the header, which is used to find the IP address, to be applied incorrectly in the downstream service. Once the configuration was changed back to its original value, the issue was resolved.
We are engaged in several activities to prevent a recurrence. We are analyzing the IP authorization system and the downstream services to identify what occurred that caused the wrong header to be applied in the downstream service following the configuration update. Once this has been fully understood, the appropriate mitigations will be implemented prior to this update being reapplied. In parallel, we are also reviewing our testing, monitoring, and alerting to ensure more robust coverage for the IP allowlist feature.
We deeply apologize for any impact this issue may have had on your business operations. We are continuously strengthening our systems and procedures to ensure we avoid future disruptions to your business and users.
If you have any further questions or concerns, please contact Anaplan Customer Care. Thank you for your patience during this situation and thank you for being an Anaplan customer.