On October 29th 2023, between 07:10 and 07:30 UTC, a small percentage of Pantheon sites started reporting downtime per internal monitoring. Upon further investigation it was determined that a single malicious site was receiving a massive amount of requests, which in turn overloaded certain internal services. This prevented other traffic from being served. The site was immediately blocked at the edge and put into maintenance mode due to its abusive/bad actor nature. We are actively working to reduce our exposure to this sort of incident in the future.
Posted Nov 08, 2023 - 08:19 PST
Resolved
The issue has been successfully resolved and there should be no further disruptions. If you come across any unexpected issues, please don't hesitate to get in touch with our support team for immediate assistance.
We appreciate your for your patience and continued support during this incident.
Posted Oct 29, 2023 - 03:00 PDT
Monitoring
A fix has been implemented and we are actively monitoring the situation as we continue to observe a full recovery in progress.
Posted Oct 29, 2023 - 02:30 PDT
Identified
The issue has been identified and a fix is being implemented.
Posted Oct 29, 2023 - 01:47 PDT
Update
Our investigation into this issue is ongoing, and we are dedicated to resolving it at the earliest opportunity.
Rest assured, we will provide updates on this page as soon as new information becomes available.
Should you encounter urgent matters or require immediate assistance, please don't hesitate to reach out to us at helpdesk@pantheon.io. We are here to address your urgent concerns.
Posted Oct 29, 2023 - 01:10 PDT
Investigating
We are investigating an infrastructure failure that is affecting customer sites. The current symptoms identified are 503 errors leading to site unavailability.