We are currently facing an unavailability of the Metrics and access logs stack. The problem has been identified and we are working to bring it up.
Metrics through the console or Grafana or access logs query is currently affected.
EDIT 16:44 UTC: The service is back up, we are starting to process the backlog of events. You should now be able to query the data but it might lag a bit.
EDIT 17:01 UTC: The queue has been ingested. The service is now back to normal. Sorry for the inconvenience
Tuesday 10th January 2023
DeploymentsDeployments are slow
Deployments are currently slower than usual. They may take more time to start or complete. We are investigating.
EDIT 13:46 UTC: The slowness is now resolved since 13:35. The initial cause of the slowness has been found and we continue to monitor the situation.
Monday 9th January 2023
DeploymentsDeployments are unavailable
Deployments are currently unavailable and failing for unknown reasons. We are currently investigating.
EDIT 15:25 UTC: Deployments are running again. Some more operations will be done in the next few minutes to stabilize the situation. In the meantime, we continue to monitor the health of the deployment system.
EDIT 15:45 UTC: The incident is now over. If you still have troubles deploying your application, please reach out to our support team. Sorry for the inconvenience.
Sunday 8th January 2023
No incidents reported
Saturday 7th January 2023
CellarClock skew on Cellar C2 (Paris zone)
Cellar C2 is having issue with time sync. It may result with a "ClockTooSkewed" error when you try to list or access files.
We are working on fixing the clocks on the Ceph monitoring servers. (Ceph is the software we use to provide the Cellar service.)
EDIT 12:40 UTC+1: One of the reverse proxies in front of the Cellar system was desynchronized. This proxy is now out of the pool for further investigation and the issue should now be fixed.