Metrics queries results are lagging a bit, we have identified the underlying issue and issued a preliminary fix. We are monitoring the result. Grafana dashboards or results obtained from the metrics API might be missing some recent values until this is resolved.
EDIT 2024-04-18 10:53 UTC+2: The issue is still present. We've been force to sample incoming data until we figure out the underlying issue.
EDIT 2024-04-18 12:07 UTC+2: Our storage layer has been stabilized, we still apply a sampling on incoming data. Queries should be working properly.
EDIT 2024-04-18 21:13 UTC+2: The situation has improved, sampling on incoming data has been disabled. We continue to monitor the system but queries should now return the correct data without lag.
EDIT 2024-04-18 23:37 UTC+2: This incident is now over.