At 4:49pm PDT on April 2 2024, Amplitude’s US data center experienced a service interruption after a series of metadata tables were accidentally deleted. Customers on the US data center were unable to access the Amplitude platform—including Analytics, CDP, Experiment, and Session Replay. Our EMEA data center was not impacted. The service came back online at ~11.30pm PDT the same day, and we began processing the data received—but not ingested—during the outage. As a result, there was a lag in performance and/or limited availability of a small amount of metadata for some users as we worked to fully restore the service. As of 12:15am PDT on April 4 2024, Amplitude is running at full capacity and we are conducting a full root cause analysis to ensure this doesn’t happen again.
Posted 1 year ago. Apr 04, 2024 - 00:33 PDT
Update
A small percentage of the metadata, including event types or property names introduced between 12:30pm and 5:00pm PDT on April 2, may still be temporarily absent from charts and dashboards. We expect this issue to be resolved by Thursday, 4/4. We will provide another update by 9am PT.
Posted 1 year ago. Apr 03, 2024 - 17:55 PDT
Update
A small percentage of the metadata, including event types or property names, introduced between 7:10am and 5:00pm PDT on April 2 may still be temporarily absent from charts and dashboards. We are continuing to work on this issue and will provide another update at 6:00pm PDT.
Posted 1 year ago. Apr 03, 2024 - 16:03 PDT
Update
We are done processing the data collected during the outage, and platform performance should be back to normal. A small percentage of the metadata, including event types or property names, introduced between 7:10am and 5:00pm PDT on April 2 may still be temporarily absent from charts and dashboards. We are working on this issue and will provide another update at 4:00pm PDT.
Posted 1 year ago. Apr 03, 2024 - 14:14 PDT
Update
We are still processing the data collected during the outage. We expect this to be complete within 2 hours. Our next update will be at 2:00pm PDT.
Posted 1 year ago. Apr 03, 2024 - 11:58 PDT
Update
We are still processing the data collected during the outage. We expect this to be complete within 1-2 hours. Our next update will be at 12:00pm PDT.
Posted 1 year ago. Apr 03, 2024 - 10:46 PDT
Update
We are still processing the data collected during the outage. We expect this to be complete within 1-2 hours. Our next update will be at 12:00pm PDT.
Posted 1 year ago. Apr 03, 2024 - 10:45 PDT
Update
We are still processing the data collected during the outage. We expect this to be complete within 1-2 hours. Our next update will be at 12:00pm PDT.
Posted 1 year ago. Apr 03, 2024 - 10:04 PDT
Update
We are still processing the collected data during the outage. We estimate the data processing will catch up before 10 AM PDT on Apr 3rd. Next update will be on 10:00 AM PDT.
Posted 1 year ago. Apr 03, 2024 - 07:31 PDT
Update
We are now processing the collected data during the outage. We estimate the data processing will catch up before 7AM PDT on Apr 3rd. Next update will be on 7:30 AM PDT.
Posted 1 year ago. Apr 03, 2024 - 03:21 PDT
Update
We are continuing to monitor for any further issues.
Posted 1 year ago. Apr 02, 2024 - 23:30 PDT
Monitoring
We have successfully restored our database and are working on resuming services. Most our services should become available momentarily.
Given that several hours' worth of data was received during the outage but not processed, there may still be a minor lag in performance and/or availability for some users as we work to fully restore the service.
Small percentage of data between 7:10 AM PDT and 5 PM on Apr 2nd might not show in charts and dashboards temporarily.
We are still working on data processing and recovery and we will provide the next update in 2 hours.
Posted 1 year ago. Apr 02, 2024 - 23:24 PDT
Update
We are still in the process of restoring the data. Next update will be posted here in 2 hours.
Posted 1 year ago. Apr 02, 2024 - 20:55 PDT
Update
At 4:49 PM PDT, a series of metadata tables were deleted that caused a service interruption to Amplitude. At this time, we believe we have a fix and we will continue to update the Status Page with new details as they are available.
Note that we are still receiving data and it will be available when we come back online. We apologize for the disruption to your service experience.
Next update will be posted here in 2 hours.
Posted 1 year ago. Apr 02, 2024 - 19:10 PDT
Update
We are continuing to work on a fix for this issue.
Posted 1 year ago. Apr 02, 2024 - 17:48 PDT
Identified
On 16:49 PM PDT Apr 2nd, all our customers experienced a wide service outage across our analytics, experiment, CDP and session replay. We have identified the root cause and working on the remediation actions. Our EU customers are not affected.
Next update will be in the next 2 hours.
Posted 1 year ago. Apr 02, 2024 - 17:14 PDT
This incident affected: Analytics (Web Reporting, Web Application), Experiment (Evaluation API, Management API, Web Application), Audiences (Profile API, Web Application), and Data (Web Application, CLI/Codegen, Data Processing, Data Reception, Cohort Export, Event Streaming, Data Export).