Past Incidents

Friday 18th January 2019

API Console/API issues

We are experiencing issues on our API which is impacting the console. We are investigating.

EDIT 18/01/19 00:53 UTC: Root issue is most probably identified. The issue was coming from an internal tool. We will investigate this further. In the meantime, the tool has been deactivated and shouldn't cause any harm.

Thursday 17th January 2019

No incidents reported

Wednesday 16th January 2019

No incidents reported

Tuesday 15th January 2019

MongoDB shared cluster [MongDB] Free Shared cluster connections issues

MongoDB free shared cluster is having troubles accepting connections. We are investigating the issue.

EDIT 16/01/2019 09:45 UTC: The problem might be due to old clients drivers being used on the cluster. We have set up a new cluster (version 4.0.3) which should greatly improve things. You can create a new add-on to migrate your database.

To dump your data from your existing, you can use this command: mongodump -u "${MONGODB_ADDON_USER}" -p "${MONGODB_ADDON_PASSWORD}" -h "${MONGODB_ADDON_HOST}" -d "${MONGODB_ADDON_DB}" --archive --gzip

You can then import the data into the new database by using the mongorestore command displayed in the dashboard of your new add-on.

An automatic migration tool for mongodb should be available in the next few days.

RabbitMQ shared cluster [RabbitMQ] Network loss for one node of shared cluster

One node of the shared RabbitMQ cluster lost its connectivity during 1 minute. It then re-joined the cluster as expected. Real time logs were unavailable at the same time because of that issue.

Monday 14th January 2019

[Redis] Redis add-on upgrade at 13:00 UTC, scheduled 5 years ago

Redis add-on creation will be disabled starting 13:00 UTC. Dashboard might be unavailable too. This should not include redsmin integration which should remain available.

16:15 UTC: The maintenance is over. Add-on creation and dashboard are now fully available again.

Sunday 13th January 2019

No incidents reported

Saturday 12th January 2019

Infrastructure An hypervisor is down

(Hours in UTC)

At 22:27, one of our hypervisors lost access to parts of its disks. Amongst others, It impacted a deprecated front reverse proxy for applications and a front reverse proxy for add-ons (databases). We moved the IP of one of the proxies. The other one, related to the application reverse proxy (62.210.92.244) couldn't be moved and is now unreachable. If you still use it, you should update your DNS records: https://www.clever-cloud.com/doc/admin-console/custom-domain-names/#personal-domain-names

The situation is stabilized. We still consider the infrastructure not fully recovered.