PBS to Slurm
Below is some information which will be useful in transitioning from the PBS style of batch jobs used on Fionn to the Slurm jobs used on Kay. While the concepts are quite similar in most common use cases, different commands and options are used.
Translation table for commands
| Function | PBS | Slurm |
|---|---|---|
| Submit job script | qsub | sbatch |
| Request interactive job | qsub -I | srun --pty bash |
| Query queue state | qstat (or showq) | squeue |
| Cancel job | qdel (or canceljob) | scancel |
Translation table for batch file options
Job specifications can be given on the commandline (sbatch and srun) or as part of a batch submission script. These options must be preceded by #PBS or #SBATCH for PBS and Slurm respectively.
| To specify: | qsub option | sbatch option | Comment |
|---|---|---|---|
| queue to run in | -q queuename | -p queuename | Slurm refers to queues as partitions |
| account to charge | -A accountname | -A accountname | |
| number of nodes | -l nodes=X:ppn=Y | -N X | On Kay, you just need to request the number of Nodes, X as all nodes have the same core count |
| walltime | -l walltime=HH:MM:SS | -t HH:MM:SS | Slurm also allows other time formats, see man page |
| stdout file | -o outfilename | -o outfilename | This will combine stdout/stderr on slurm if -e not given also |
| stderr file | -e errorfilename | -e errorfilename | This will combine stdout/stderr on slurm if -o not given also |
| Combining stdout/stderr | -j oe | see above | |
| email address | -M me@x.y | --mail-user=me@x.y | |
| email options | -mb | --mail-type=BEGIN | Send email when job starts |
| -me | --mail-type=END | Send email when job ends | |
| Job name | -N NAME | --job-name=NAME | |
| environment variables | $PBS_O_WORKDIR | $SLURM_SUBMIT_DIR | submit directory |
| $PBS_NODEFILE | $SLURM_JOB_NODELIST | list of nodes assigned to job | |
| $PBS_JOBID | $SLURM_JOBID | job ID | |
| $PBS_NUM_NODES | $SLURM_JOB_NUM_NODES | number of nodes allocated to job | |
| $PBS_NUM_PPN | $SLURM_CPUS_ON_NODE | number of cores per node |
#!/bin/sh #PBS -q ProdQ #PBS -l nodes=2:ppn=24 #PBS -l walltime=120:00:00 # Charge job to myaccount #PBS -A myaccount # Write stdout+stderr to file #PBS -o output.txt #PBS -j oe # Mail me on job start & end #PBS -M myemail@universityname.ie #PBS -m be cd $PBS_O_WORKDIR module load intel mpiexec -n 48 ./my_mpi_app |
#!/bin/sh #SBATCH -p ProdQ #SBATCH -N 2 #SBATCH -t 120:00:00 # Charge job to myaccount #SBATCH -A myaccount # Write stdout+stderr to file #SBATCH -o output.txt # Mail me on job start & end #SBATCH --mail-user=myemail@universityname.ie #SBATCH --mail-type=BEGIN,END cd $SLURM_SUBMIT_DIR module load intel srun ./my_mpi_app |
