Realtime lag is fully catch up. All systems should be recovered now.
Posted Apr 30, 2024 - 08:34 PDT
Update
Realtime lag is coming down to less than 1 hour now, we expect it fully recover in ~20 mins
Posted Apr 30, 2024 - 08:08 PDT
Monitoring
We identified the root cause and had a fix, the system should start to recovering now. Realtime data is delaying for almost 2 hours, it's catching up now.
Posted Apr 30, 2024 - 07:15 PDT
Update
We are still investigating the root cause, did some mitigation, so some customers could see recovery, but realtime data is still missing from query result, no data lost. During the investigation, we disabled cohort sync from 6:45 am to 7:20am PT to test a theory.
Posted Apr 30, 2024 - 06:52 PDT
Investigating
Since 2024-04-30 05:20 PT (12:20 UTC), we are currently experiencing issues with our query capabilities and users may see their charts/dashboards slow to load or unavailable.
Our team is working on a fix and no data is lost during this incident. No action is needed from customers at this time.
We will post updates as we recover the systems.
Posted Apr 30, 2024 - 05:42 PDT
This incident affected: Analytics (Web Reporting, Web Application).