Past Incidents

Wednesday 14th October 2020

No incidents reported

Tuesday 13th October 2020

No incidents reported

Monday 12th October 2020

No incidents reported

Sunday 11th October 2020

No incidents reported

Saturday 10th October 2020

No incidents reported

Friday 9th October 2020

No incidents reported

Thursday 8th October 2020

FS Buckets Some FS Buckets are experiencing issues

Some FS Buckets addons are experiencing issues. We identified the issue and are working on its resolution.

EDIT 21:25 UTC: The issue is fixed. The PHP applications may not work correctly. We are redeploying them.

EDIT 22:30 UTC : Applications with FS Buckets have been redeployed. The incident is closed.

Post mortem: An incorrect human action conducted the FS Buckets system to follow the wrong path between different storage nodes. We applied fix to avoid this cause.

Cellar Cellar addons are experiencing issues

We identified issues on Cellar addons availability. We identified them and are working on their resolution.

EDIT 15:25 UTC: fixed. We are investigating the reasons.

EDIT 15:45 UTC: we identified the reasons and applied a fix.

Access logs Metrics / Access logs: requests error

Metrics and access logs requests might experience issues following the maintenance of a core component of those features. Requests can either take a very long time to complete or simply answer an error. We are working toward a fix.

Data won't be lost, the ingestion is simply delayed.

Impacted products:

Metrics (in console or using the API)
Access logs (charts in the console's overview or using the CLI / API)

EDIT 14:03 UTC: Ingestion is now catching up on the delay, everything looks good. Looks like it may take 30 to 40 minutes to go completely back to normal.

EDIT 14:25 UTC: Ingestion has now caught up, everything should be back to normal.

EDIT 21:26 UTC: New issues are ongoing, we are investigating.

EDIT 22:16 UTC: Ingestion is running. We are consuming queues.

EDIT 23:30 UTC: Ingestion is back to normal. Fixed.