RWTH High Performance Computing (HPC)

We must report that compute nodes within Claix2023, Claix2025 as well as IH and private systems are currently suffering from random operating system failures that cause the jobs to become unavailable and for Jobs to hang while finishing in a completing state.
We know that this causes the waiting times to increase and that users are unable to stop their hanging jobs.
We are currently working on solving the problem with the uttermost priority.

Until the problem is resolved, we will manually stop users jobs in a stuck state and reset the nodes to a usable state manually.
This also takes time and work, so we ask users to please have patience until it is resolved.
To minimize node downtime, we ask users to try and use single node jobs where possible.

We apologize for this inconvenience and hope to have the issue resolved as soon as possible.

Due to latent bugs in resource accounting, ssh-ing into nodes where one of your own jobs is running is temporarily disabled.

We are aware of the popularity of this feature and aim to reenable it in a future maintenance.

During the specified maintenance window, the IT Center will perform maintenance on the MySQL databases used by the Multi-Factor Authentication (MFA) service. As part of the maintenance, the databases will be unavailable for a total of approximately 15 minutes.

During this time, new logins to services using RWTH Single Sign-On will not be possible. Existing sessions will remain active and will not be affected by the maintenance.

Due to maintenance work and debugging, copy23-2 is unavailable at the moment. Please use copy23-1 in the meantime.

Due to a GPFS failure on the login nodes, it is currently not possible to log in, as the home directory cannot be loaded.

Partial outage has been resolved.

Due to unresolved security issues, the access to the CLAIX dialog systems is restricted at the moment. Updates are being prepared. Once deployed, the dialog nodes can be opened again.
Already submitted batch jobs are not affected and scheduled.

We would like to inform users of the ongoing security updates we are deploying across the Claix Cluster.
These updates aim to patch a dangerous root exploit that could affect the entire system.
In addition to the existing login nodes, all compute nodes haven been drained to apply the necessary security patches.
Login nodes are patched and should be usable for users. The compute nodes on the other hand are being drained.
As individual compute nodes go idle and available for updates, we will deploy the security patches.
Since the progress is gradual as nodes go free and idle from the draining procedure, the time require to patch the entire system might take over a week.
If you are an HPC user with active jobs, we recommend you cancel any non essential jobs manually using scancel.
This will free the susceptible nodes sooner for security updates.

up to now, 22 out of 48 GPU systems are still not updated as also 144 out of 661 HPC systems.
We are continually working on updating the systems as soon as they are empty.

With the exception of a few systems, the problems have been resolved.

The CLAIX-2025 cluster must be shut-down due to maintenance work at the CDU.

The "DGUV certification" measurements are also conducted; the maintenance needs to be extended for that purpose.

The CDU's faulty pump was replaced. The DGUV measurements are still pending due to technical issues.

Thde DGUV testing is now in progress.

The DGUV tests are completed, the cluster will be powered-on shortly.

To resolve observed slow performance, all CLAIX-2025 nodes are under temporary maintenance during performance tuning and benchmarks. We will open the cluster again for the pilot users afterwards.

The performance tuning is ongoing, but will be limited to allow for a pilot operation on CLAIX-2025.