Authentication Plugins

Overview

It is important to know that the Remote Procedure Calls (RPCs) that are received by Slurm are coming from trusted sources. There are a few different authentication mechanisms available within Slurm to verify the legitimacy and integrity of the requests.

MUNGE

MUNGE can be used to create and validate credentials. It allows Slurm to authenticate the UID and GID of a request from another host that has matching users and groups. MUNGE libraries must exist when building Slurm in order for it to be able to use munge for authentication. All hosts in the cluster must have a shared cryptographic key.

Setup

1. MUNGE requires that there be a shared key on the machines running slurmctld, slurmdbd and the nodes. You can create a key file by entering your own text or by generating random data:

   dd if=/dev/random of=/etc/munge/munge.key bs=1024 count=1
   

2. This key should be owned by the "munge" user and should not be readable or writable by other users:

   chown munge:munge /etc/munge/munge.key
   chmod 400 /etc/munge/munge.key
   

3. Distribute the key file to the machines on the cluster. It needs to be on the machines running slurmctld, slurmdbd, slurmd and any submit hosts you have configured. Use the distribution method of your choice.
4. The "munge" service should be running on any machines that need to use it for authentication. It should be enabled and started on all the machines you distributed a key to:

   systemctl enable munge
   systemctl start munge
   

5. Changing the authentication mechanism requires a restart of Slurm daemons. The daemons need to be stopped before updating the slurm.conf so that client commands do not use a mechanism other than what the running daemons are expecting.
6. Update your slurm.conf and slurmdbd.conf to use MUNGE authentication:

   AuthType = auth/munge
   CredType = cred/munge
   

7. Start the Slurm daemons back up with the appropriate method for your cluster.

Slurm

Beginning with version 23.11, Slurm has its own plugin that can create and validate credentials. It validates that the requests come from legitimate UIDs and GIDs on other hosts with matching users and groups. All hosts in the cluster must have a shared cryptographic key.

Setup

1. For the authentication to happen correctly you must have a shared key on the machine running slurmctld, slurmdbd and the nodes. You can create a key file by entering your own text or by generating random data:

   dd if=/dev/random of=/etc/slurm/slurm.key bs=1024 count=1
   

2. This key should be owned by SlurmUser and should not be readable or writable by other users. This example assumes the SlurmUser is 'slurm':

   chown slurm:slurm /etc/slurm/slurm.key
   chmod 600 /etc/slurm/slurm.key
   

3. Distribute the key file to the machines on the cluster. It needs to be on the machines running slurmctld, slurmdbd, slurmd and sackd.
4. Update your slurm.conf and slurmdbd.conf to use the Slurm authentication type:

   AuthType = auth/slurm
   CredType = cred/slurm
   

5. Start the daemons back up with the appropriate method for your cluster.

If the cluster does not have access to consistent user ids from LDAP or the operating system, you can use the use_client_ids option to allow it to use the Slurm authentication mechanism.

SACK

Slurm's internal authentication makes use of a subsystem — the Slurm Auth and Cred Kiosk (SACK) — that is responsible for handling requests from the auth/slurm and cred/slurm plugins. This subsystem is automatically started and managed internally by the slurmctld, slurmdbd, and slurmd daemons on each system with no need to run a separate daemon.

For login nodes not running one of these Slurm daemons, the sackd daemon should be run to allow the Slurm client commands to authenticate to the rest of the cluster. This daemon can also manage cached configuration files for configless environments.

JWT

Slurm can be configured to use JSON Web Tokens (JWT) for authentication purposes. This is configured with the AuthAltType parameter and is used only for client to server communication. You can read more about this authentication mechanism and how to install it here.

Last modified 12 March 2024