RWTH High Performance Computing (HPC)

GPFS is not working anymore on the login nodes. we are working on the problem

We must report that compute nodes within Claix2023, Claix2025 as well as IH and private systems are currently suffering from random operating system failures that cause the jobs to become unavailable and for Jobs to hang while finishing in a completing state.
We know that this causes the waiting times to increase and that users are unable to stop their hanging jobs.
We are currently working on solving the problem with the uttermost priority.

Until the problem is resolved, we will manually stop users jobs in a stuck state and reset the nodes to a usable state manually.
This also takes time and work, so we ask users to please have patience until it is resolved.
To minimize node downtime, we ask users to try and use single node jobs where possible.

We apologize for this inconvenience and hope to have the issue resolved as soon as possible.

Due to latent bugs in resource accounting, ssh-ing into nodes where one of your own jobs is running is temporarily disabled.

We are aware of the popularity of this feature and aim to reenable it in a future maintenance.

Im angegebenen Zeitraum führt das IT Center Wartungsarbeiten an den MySQL-Datenbanken der Multi-Faktor-Authentifizierung (MFA) durch. Im Verlauf der Wartung werden die Datenbanken für insgesamt etwa 15 Minuten nicht zur Verfügung stehen.

Während dieses Zeitraums sind keine neuen Anmeldungen an Services mit RWTH Single Sign-On möglich. Bereits bestehende Sitzungen bleiben jedoch bestehen und sind von der Wartung nicht betroffen.

Due to maintenance work and debugging, copy23-2 is unavailable at the moment. Please use copy23-1 in the meantime.

Due to unresolved security issues, the access to the CLAIX dialog systems is restricted at the moment. Updates are being prepared. Once deployed, the dialog nodes can be opened again.
Already submitted batch jobs are not affected and scheduled.

We would like to inform users of the ongoing security updates we are deploying across the Claix Cluster.
These updates aim to patch a dangerous root exploit that could affect the entire system.
In addition to the existing login nodes, all compute nodes haven been drained to apply the necessary security patches.
Login nodes are patched and should be usable for users. The compute nodes on the other hand are being drained.
As individual compute nodes go idle and available for updates, we will deploy the security patches.
Since the progress is gradual as nodes go free and idle from the draining procedure, the time require to patch the entire system might take over a week.
If you are an HPC user with active jobs, we recommend you cancel any non essential jobs manually using scancel.
This will free the susceptible nodes sooner for security updates.

up to now, 22 out of 48 GPU systems are still not updated as also 144 out of 661 HPC systems.
We are continually working on updating the systems as soon as they are empty.

Bis auf einzelne Systeme sind die Probleme behoben.

The CLAIX-2025 cluster must be shut-down due to maintenance work at the CDU.

The "DGUV certification" measurements are also conducted; the maintenance needs to be extended for that purpose.

The CDU's faulty pump was replaced. The DGUV measurements are still pending due to technical issues.

Thde DGUV testing is now in progress.

The DGUV tests are completed, the cluster will be powered-on shortly.

To resolve observed slow performance, all CLAIX-2025 nodes are under temporary maintenance during performance tuning and benchmarks. We will open the cluster again for the pilot users afterwards.

The performance tuning is ongoing, but will be limited to allow for a pilot operation on CLAIX-2025.