Some systems are experiencing issues

Past Incidents

Monday 31st October 2022

Infrastructure SYD/SGP network issues

Our monitoring has raised network issues, we are investigating.

Status:

  • Ping does not go through between PAR (on BSO Network) and SGP/SYD (OVH Network)
  • Ping does go through between PAR and other OVH zones (RBX, MTL, WSW…)
  • Ping does go through between RBX and SGP/SYD (OVH to OVH)
  • Applications on both SGP and SYD are still UP and reachable from other networks. Deployments on these zones are still unavailable.

UPDATE 20:13 UTC: Network is kind of coming back up, but we see 80% to 90% packet loss. 21:50 UTC: still a lot (90%) of loss on the PAR -> SGP/SYD route, way less (30%) in the SGP/SYD -> PAR route. 2022-11-01 0812 UTC: >90% of loss on the PAR -> SGP/SYD route. 2022-11-01 1812 UTC: network seems fine.

Sunday 30th October 2022

No incidents reported

Saturday 29th October 2022

No incidents reported

Friday 28th October 2022

No incidents reported

Thursday 27th October 2022

Reverse Proxies PAR zone proxies unavailable

PAR reverse proxies unavailable for a short time period

Reverse Proxies Network issue on OVH

Some network issues on our provider OVH can leads to some desync of reverse proxy configurations

Access logs Performance issue on Metrics/AccessLogs

We are experiencing performance issues on our metrics/accesslogs infrastructure. We are on it.

Update 10:36 UTC: Performance has been fixed.

Wednesday 26th October 2022

No incidents reported

Tuesday 25th October 2022

Infrastructure Deployment queue / Application slowness

Some deployments can have abnormal delays to deploy. Applications may experience slowness.

** EDIT 11:55 UTC **: We have found the root cause, we have mitigated the issue. we are deploying the solution.