Sunday 31st May 2020

Access logs data loss and partial unavailability

Due to a backend issue, access logs between 17:05 UTC and 18:19 UTC were lost. All access logs emitted after 18:47 UTC can't be queried because they are not yet indexed. The indexation fails because of a issue with the GEO IP location feature. Once this is fixed, logs will then be indexed. This impacts accesslogs retrieval through the command line and the various stats displayed in the console (last 24 hours of requests, heatmap, live map, status code, ...).

Update 21:04 UTC: The GEO IP feature has been fixed. It seems to have initially broke with an auto update of the GEO IP library but more tests will need to be conducted to be sure of the root cause. All access logs between 18:47 UTC and now have been consumed and you should now be able to query them. We will work on improving the monitoring of the whole system to detect this kind of issue faster.