Thursday 9th September 2021

Metrics / AccessLogs [Metrics / Access Logs] Ingestion lag and queries unavailability

Metrics and access logs are currently partially unavailable to query. We are investigating.

EDIT 14:35 UTC: The root cause has been identified, the ingestion lag currently sits at around 2 hours so metrics queries will be out of sync for the time being. Access logs are not ingesting and are currently kept in a separate queue. We expect the lag to start decreasing later tonight. This incident is a follow-up to the urgent maintenance of yesterday which mainly aimed at better stabilizing the cluster.

EDIT 23:34 UTC: Metrics have been fully ingested, access logs are still delayed but they are currently being written. Queries might still be slow, this is expected.

EDIT 6:30 UTC: The situation is back to normal.