Degraded Performance - UPDATE
Incident Report for Pantheon Operations
Postmortem

On March 14th, 2019 at approximately 11:05 AM Pacific, one of our distributed filesystems experienced increased latency which resulted in read-only filesystems for less than 0.5% of paid Pantheon Sites. At 11:50, our engineers identified and implemented a remediation. By 12:00, the number of sites affected dropped by half, and by 12:20 all sites were operating as usual. We have since implemented an improvement to reduce load on our distributed filesystems, we have updated our internal documentation on how to mitigate this issue, and we are updating our alerts to be more sensitive to distributed filesystem latency.

Posted Mar 20, 2019 - 14:45 PDT

Resolved
This update is regarding: https://status.pantheon.io/incidents/bpqnd83z5br6

We are planning to release a post-mortem for this incident on Wednesday, March 20th by 5 pm Pacific Time. Check our status page after that time for the details.
Posted Mar 15, 2019 - 14:56 PDT
This incident affected: Customer Sites.