4 Example of a UNICORE 5 job
As an example of a job definition through UNICORE, we will consider a job consisting of two main tasks. The first task has to run the CPMD executable (using some input files), while the second one must run the MyPostp.x executable on the output of the first one.
In our example, all the work is done on your personal computer or workstation, on which the UNICORE client was installed and configured. The input files used in the run reside locally on this computer, so they will have to be transferred to the target host at the beginning: this is also done through UNICORE.
After having started the UNICORE client, you can create a new job named for example “MyModel_job”, as shown in Figure 8. The new job must be assigned to a specific host to be run. Suppose that your home site is CINECA; you will first have to choose the “DEISA_Cineca” so-called Usite (UNICORE Site), and then the “CINECA SP5” Vsite (Virtual Site) inside this UNICORE site (Usite) (SP5 is the name of the CINECA IBM cluster).
Having chosen the “CINECA SP5” site, all the tasks inside the “MyModel_job” job will be run on that platform if you do not select another platform inside the same USPACE, where USPACE is a new, empty scratch directory. You may define one or more subjobs within your UNICORE job, for which you can choose new platforms that you have been granted access to.
Now, a new task must be added to run the CPMD program. You can use a script task, i.e. a task which consists of a script to be run on the target site. This is shown in Figure 9. Other tasks are available, for example, the command task allows you to run a single command on the remote platform; the command can be chosen by browsing the remote file systems. The script task that we will use in this case allows you to write and execute a shell script (in general consisting of more than one command) on the remote platform. This is a convenient way to run existing jobs under UNICORE. Several tasks might be combined within one job. You can also install other plugins, which offer you the possibility to run tasks using an application specific GUI.
The script content can now be defined as shown in Figure 10.
Notice the use of the “module load deisa” command at the beginning of the script. This is mandatory to set up a correct environment. Loading the “cpmd” modulefile gives an access to the CPMD application in a portable way, without knowing the specificities of the installation of this software on the platform used. Since a local input file, stored in the disk of your own PC or workstation, is needed by this task, it has to be imported in the job’s USPACE; this can be done by using the “File imports” panel of the task, as shown in Figure 11, where the local file input.in is copied in the USPACE.
Now, we must assign to the task a set of resources available on CINECA SP5. In the resources panel of “MyModel_job” we will add a new resource set, named “MyModel_task_res”, which will be composed of 8 tasks with one thread each. This resource set must then be assigned to “MyModel_task” (see Figure 12 and Figure 13).
Users may also specify to the LoadLeveler scheduler other keywords that are not listed into the resource editor, in order to require further resources, stack_limit and cpu_limit keywords and the Feature == “DEISA” requirement. This can be done by using the environment line in the options panel, and writing “stack_limit=250MB cpu_limit=78800 feature=DEISA” and keeping in mind that no blanks are allowed within the keyword indication, but only to delimit one keyword from the other.
The definition of this task is now complete. Now we will have to add a new task for the postprocessing phase; in our example this phase is also run on the same Vsite (the CINECA SP5 platform inside the “DEISA_CINECA” Usite), However, this is done in another job because the postprocessing phase is serial, so it should not hold the same number of processors (8) reserved by the previous phase. Thus, we will have to add a new subjob referring to that platform. This new job is named “MyPostp_subjob”, see Figure 15.
The new task, named “MyPostp_task”, is also a script task. Its content is shown in Figure 16.
Again, we will have to define a set of resources on CINECA SP5 for this task. This is shown in Figure 17. We will need only one processor for this phase.
This resource set should then be assigned to the “MyPost_task”, as previously done for “MyModel_task”. The output of “MyPostp_task” must be transferred to your personal computer; this can be done by using the “File exports” panel of the task, as shown in Figure 18.
Now, you will have to link the output of “MyModel_task” with the input of “MyPostp_task”; this can be accomplished by defining a “Transfer” task inside “MyModel_job” (see Figure 19).
This task copies the two files MyModel.conf and MyModel.output from the USPACE of “MyModel_job” to the USPACE of “MyPostp_subjob”, as shown in Figure 20; these files will also have to be renamed MyPostp.conf and MyPostp.output respectively.
Now you will have three tasks:
You will need to define the workflow between them: Indeed, “Transfer” must be run after “MyModel_task” has been completed, and “MyPostp_task” has to run after “Transfer” has been completed. This can be done inside the dependencies panel of “MyModel_job”, as shown in Figure 21. The arrows are created by right-clicking on the icon of the predecessor and dropping it to the successor.
Now the job is completely defined, and can be submitted through the UNICORE client, as shown in Figure 22.
The job can then be monitored using the lower-left “job monitoring” panel. The abstract job can also be saved as a local file, for future submission.