5 UNICORE 5 and Cray XT
In this section the modifications needed to run DEISA jobs with UNICORE on a Cray system are described. Currently in DEISA, there is only one Cray system in production accessible via UNICORE, namely the Cray XT4 at CSC called Louhi (louhi.csc.fi). The information herein pertains mainly to this system. Louhi user guide describes CSC environment (and also the yod command described below):
On Cray XT systems, you must use the yod command to launch parallel executables. For example, to run the "MyModel_task" script shown in Figure 10 in ADD REFERENCE, you need yod in front of the $CPMD command:
yod $CPMD input.in $CPMD_PPLIBRARY > output.out
More information on yod can be found from "Cray XT Series Programming Environment User's Guide"
Normally, when using UNICORE, one reserves resources choosing in Application specific settings switches Threads and Tasks. When using Cray XT, currently only the switch Tasks has any effect, the switch Threads will be ignored. The switch Tasks will denote directly the number of reserved cores.
However, the compute nodes on Cray XT has currently dual core processors and later even quad cores, i.e. multi-cores. Hence, when one wishes to run
- a single core job: one sets Tasks to twice the required number of cores.
- In addition, one has to use the job starting command yod with the switch -SN to indicate that the job will use single cores in nodes.
- Basically this is practical only when the program needs more memory than normally available for each core. In this case one core can use all the memory available in a node. But the price is that then in each processor node only one core is doing calculations, and the other cores will be idle.
- a multi-core job (default; currently dual core job only): one sets Tasks to the number of required cores.
- In this case there is no need to give any special switches with the job starting command yod (since running multi-core jobs is the default).
Note that the Tasks parameter denotes the number of reserved cores, whether they are actually used in the calculations or not.
1. Single core job with 64 cores (reserves 64 dual core nodes, i.e. 128 = 2 x 64 cores):
Tasks = 128
yod -SN <rest of the yod command parameters>
2. Multi-core job with 64 cores (reserves 64/2 = 32 dual core nodes):
Tasks = 64
yod <rest of the yod command parameters>
In this example, the first line is the value used for the UNICORE parameter Tasks, and the second line is used in the job script for the parallel job launcher yod.