All Systems are Online

CHG0076999 CITZ MCS SILVER - Migrate to OVN Container Network Interface

In Progress

This change is now in progress


Posted: Mon, 20 Apr 2026 20:39:00 +0000

Scheduled

The current Container Network Interface (CNI) called OpenShift SDN is deprecated and not available in the next version of OpenShift. We need to move to OVNKubernetes CNI to unlock the upgrade path to OpenShift 4.18.

The upgrade requires a rolling restart of all nodes in the cluster twice. API nodes and Infra nodes are highly available and will fail over services as needed.

As the node drains are managed by OpenShift operator automation, Platform Ops has no way to control the speed of drains.

We may continue to see slow or delayed storage operations like mounting PVCs to pods. Pods drained from one node onto another node may see up to 45 min delay until it starts. The best solution is to have multiple replicas so that a single pod failing to start does not cause application downtime.


Posted: Mon, 20 Apr 2026 16:00:00 +0000

Metrics

Private Cloud - Silver Cluster

Uptime - %

Private Cloud - Gold Cluster

Uptime - %

Private Cloud - GoldDR Cluster

Uptime - %

Private Cloud - Emerald Cluster

Uptime - %

Registry

Uptime - %

RocketChat

Uptime - %

Vault

Uptime - %

Private Cloud - Klab Cluster

Uptime - %

Private Cloud - Clab Cluster

Uptime - %

Upcoming Events

Next 30 Days

Sunday, May 3rd 2026

CHG0076031 STMS Kamloops Data Centre Removal of Extension switch from MCS Load balancer

» Sun, 03 May 2026 11:00:00 +0000

Scheduled

The Extension switch connecting the MCS Load balancer to the Core switches will be bypassed.

More details at the TIB https://ociomysc.service-now.com/sp?id=kb_article&table=u_kb_template_technical_information_bulletin&sys_id=e51bc58bfb3b369492f6fa686eefdc00&view=ess

There will be an outage of up to 30 minutes to all Silver and Gold OpenShift traffic.

Should a rollback be necessary, there will be a second outage of up to 30 minutes during the change window.

Clients are encouraged to test services and validate that applications are functioning before the change window and after the activity has completed.


Posted: Sun, 03 May 2026 11:00:00 +0000

Wednesday, May 13th 2026

CHG0077435 STMS NET - Switch refresh MCS (OpenShift) Calgary Gold-DR

» Wed, 13 May 2026 01:00:00 +0000

Scheduled

Network traffic for the production and storage interfaces of the OpenShift clusters will be migrated to new switches.

No impact is expected.

There are two redundant connections for each traffic type from each server and they will be moved one at a time.


Posted: Wed, 13 May 2026 01:00:00 +0000

Overview

Silver
Good Service
Gold
Good Service
Golddr
Good Service
Emerald
Good Service
Vault
Good Service
Artifactory
Good Service

Recent Events

Last 7 Days

Wednesday, April 22nd 2026

There are no reported events.

Tuesday, April 21st 2026

CHG0077502 CITZ MCS GOLD - Move app-16 to a new rack

» View Event Details | Created Tue, 21 Apr 2026 16:00:00 +0000

Resolved

Node is back in service


Posted: Tue, 21 Apr 2026 19:24:00 +0000

In Progress

This change is now in progress


Posted: Tue, 21 Apr 2026 15:54:00 +0000

Scheduled

The app node will be drained and any pods running on it will be moved to other nodes, which may be disruptive.

This is to help clean up the organization of the racks in the datacenter.


Posted: Tue, 21 Apr 2026 16:00:00 +0000

Monday, April 20th 2026

CHG0076999 CITZ MCS SILVER - Migrate to OVN Container Network Interface

» View Event Details | Created Mon, 20 Apr 2026 16:00:00 +0000

In Progress

This change is now in progress


Posted: Mon, 20 Apr 2026 20:39:00 +0000

Scheduled

The current Container Network Interface (CNI) called OpenShift SDN is deprecated and not available in the next version of OpenShift. We need to move to OVNKubernetes CNI to unlock the upgrade path to OpenShift 4.18.

The upgrade requires a rolling restart of all nodes in the cluster twice. API nodes and Infra nodes are highly available and will fail over services as needed.

As the node drains are managed by OpenShift operator automation, Platform Ops has no way to control the speed of drains.

We may continue to see slow or delayed storage operations like mounting PVCs to pods. Pods drained from one node onto another node may see up to 45 min delay until it starts. The best solution is to have multiple replicas so that a single pod failing to start does not cause application downtime.


Posted: Mon, 20 Apr 2026 16:00:00 +0000

Sunday, April 19th 2026

CHG0076035 STMS Calgary Data Centre Removal of Extension switch from MCS Load balancer

» View Event Details | Created Sun, 19 Apr 2026 11:00:00 +0000

Resolved

Change has been completed and the Calgary clusters are back online.


Posted: Sun, 19 Apr 2026 11:58:00 +0000

Scheduled

The Extension switch connecting the MCS Load balancer to the Core switches will be bypassed.

More details at the TIB https://ociomysc.service-now.com/sp?id=kb_article&table=u_kb_template_technical_information_bulletin&sys_id=e51bc58bfb3b369492f6fa686eefdc00&view=ess

There will be an outage of up to 30 minutes to all Gold DR OpenShift traffic.

Should a rollback be necessary, there will be a second outage of up to 30 minutes during the change window.

Clients are encouraged to test services and validate that applications are functioning before the change window and after the activity has completed.


Posted: Sun, 19 Apr 2026 11:00:00 +0000

Saturday, April 18th 2026

There are no reported events.

Friday, April 17th 2026

There are no reported events.

Thursday, April 16th 2026

There are no reported events.

Subscribe to Updates