Email Disruption
Incident Report for Pantheon Operations
Postmortem

A loophole was found in our invitation system that allowed sending unsolicited mail from our transactional email system by adding users to a strategically named organization. To alleviate this issue, outgoing messages were temporarily suspended. We have mitigated the problem by disabling our invite to team email functionality. All other transactional emails have been restored.

Posted Nov 30, 2021 - 11:09 PST

Resolved
This incident has been resolved.
Posted Nov 22, 2021 - 07:03 PST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 22, 2021 - 05:34 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 22, 2021 - 05:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 22, 2021 - 04:20 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 22, 2021 - 03:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 22, 2021 - 02:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 22, 2021 - 01:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 22, 2021 - 00:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 21, 2021 - 23:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 21, 2021 - 22:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 21, 2021 - 21:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 21, 2021 - 20:19 PST
Update
No new information at this time. We will post an update in 60 minutes.
Posted Nov 21, 2021 - 19:19 PST
Update
We are continuing to work on a fix for this issue.
Posted Nov 21, 2021 - 18:18 PST
Identified
The issue has been identified and a fix is being implemented.
Posted Nov 21, 2021 - 17:17 PST
Update
We are continuing to investigate this issue.
Posted Nov 21, 2021 - 16:09 PST
Investigating
We are aware of an issue that is preventing emails from being sent when performing certain actions on the Pantheon dashboard. This is impacting the deliverability of password reset emails and notification emails when adding new team members to accounts.

This service disruption does not affect any site-specific email functionality.

Our Engineering team is actively working towards a resolution.
Posted Nov 21, 2021 - 15:09 PST