Infrastructure Issue Affecting Customer Sites

Incident Report for Pantheon Operations

Postmortem

On June 28 at approximately 3:38PM PT, a portion of the underlying Pantheon infrastructure became unavailable in a single zone due to networking issues, which lead to some sites being unreachable. Sites in the zone covered by our Disaster Recovery protection were unaffected and due to our various caching layers most sites remained able to serve traffic. We have conducted an internal review of the incident in order to increase the platform resilience to such outages.

Posted Jul 08, 2019 - 12:42 PDT

Resolved

This incident has been resolved.
Posted Jun 28, 2019 - 19:25 PDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jun 28, 2019 - 18:39 PDT

Update

We have identified the problem and are working to resolve it.
Posted Jun 28, 2019 - 18:12 PDT

Update

We are continuing to work on a fix for this issue.
Posted Jun 28, 2019 - 17:35 PDT

Update

We are continuing to work on a fix for this issue.
Posted Jun 28, 2019 - 16:27 PDT

Identified

The issue has been identified and a fix is being implemented.
Posted Jun 28, 2019 - 15:56 PDT

Investigating

We are investigating an infrastructure issue that is affecting a small portion of customer sites.
Posted Jun 28, 2019 - 15:52 PDT
This incident affected: Customer Sites.