All systems are operational

Past Incidents

Tuesday 7th February 2023

No incidents reported

Monday 6th February 2023

Infrastructure A Hypervisor is unresponsive

At 16:29 UTC, a staff member started investigating an alert on one of our hypervisors. They saw the hypervisor could not be logged into anymore.

All services running on that hypervisor are still up and running, but deployments fail to stop the obsolete VMs and we cannot connect to the host itself. We are considering a "semi" kernel crash on the hypervisor's host. We are investigating and may reboot the hypervisor in the following minutes/hours. (First, we try migrating as much important services as possible to avoid causing too much downtime to our customers.)

EDIT 16:46 UTC: We are starting to migrate add-ons on the impacted hypervisor.

EDIT 18:54 UTC: We rebooted the hypervisor, everything went well, all the remaining services are UP again.

Sunday 5th February 2023

No incidents reported

Saturday 4th February 2023

Git repositories [RETROACTIVE] Git repositories SSH remote identification changed

Between 12:39 UTC and 20:10 UTC, some users may have experienced an error message WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! when pushing code using git+ssh on our Git repositories. This was due to an update of the allowed signature algorithms of our SSH servers. Users that had an old signature algorithm stored in their known_hosts ssh file were impacted.

The change has been rolled back.

Friday 3rd February 2023

No incidents reported

Thursday 2nd February 2023

No incidents reported

Wednesday 1st February 2023

No incidents reported