Data Processing is still catching up on data received through '/batch' and vacuum. Processing of data received through '/batch' and vacuum is delayed by up to 3 hours.
Posted Oct 05, 2020 - 16:11 PDT
Update
Data processing is now caught up on all the data received from sdks and httpapi. Data Processing is still catching up on data received through '/batch' and vacuum. Processing of data received through '/batch' and vacuum is delayed by up to 6 hours.
Posted Oct 05, 2020 - 15:03 PDT
Update
We are continuing to monitor for any further issues.
Posted Oct 05, 2020 - 14:16 PDT
Monitoring
We deployed the fix and data processing is recovering currently. Data received through '/batch' and vacuum is delayed by up to 5 hours. Data received through other end points is delayed by up to 1hr.
Posted Oct 05, 2020 - 13:29 PDT
Investigating
Our data processing systems are delayed. This incident started at 11:40 am PST.
Current status: a) Our ingestion systems are working as expected and we are not losing data. Our processing systems are not processing newly ingested data. b) All of our customers are impacted. c) Impacted customers will see delayed metrics for the last 30 mins.
We are investigating the issue and will post an update in an hour or earlier if we identify the issue.