4 Using the DESHL client
Typically, users will place executable script files onto their Execution Platform. These script files must be saved as ANSI Unix-compatible format before copying (staging) to the DEISA site. Note that neither Notepad nor Wordpad are suitable as these create files with carriage return and linefeed line delimiters that will not run correctly at the target site.
One possible editor is emacs, which is available at http://www.emacswiki.org/emacs-en/CategoryWThirtyTwo
Files are created as dos files but can be converted to unix format:
Press ESC-X then type set-buffer-file-coding-system<RET> then type: unix <RET>
Note that UNICORE 6, the middleware on which DESHL is built, does not allow unlimited access to the file space at a DEISA site: instead, a set of directories, or “storages”, are configured. Normally, access is provided to $ROOT and $HOME (local to each DEISA site), and $DEISA_DATA and $DEISA_HOME (shared across most DEISA sites). You can list your Execution Sites by running the 'sites' command. The storages at a DEISA site can be listed by running the 'list' command with the name of the site, for example:
$ deshl sites fe04-ext.fec.cineca.it_13303_CINECA-SP6 unicore6.hector.ac.uk_8080_EPCC-HECToR gg.rzg.mpg.de_8080_RZG-VIP uc6.lrz-muenchen.de_5555_lrz_hlrb2_deisa unicore.grid.sara.nl_4014_SARA-HUYGENS unicore.csc.fi_8080_CSC $ deshl ls unicore.grid.sara.nl_4014_SARA-HUYGENS home deisa_scratch deisa_data deisa_home
When you first run a DESHL command, there will be a short delay while a new local server is started and you will be asked to enter the passphrase for your keystore. (Once the server is running, it is not necessary to re-enter your passphrase during your session.)
When a job is executed, a temporary directory called USPACE is created. The application and data required by the application are staged into the USPACE before the job is run. After the application has completed, any files of interest must be copied out of USPACE before the job completes. (Note that if the executable is already in the path on the execution host, it is not necessary to stage the executable.)
This subsection describes how to upload, submit and manage jobs via simple examples.
Firstly, create a new directory on your local workstation and call it 'exercises'. Go into this directory, and verify that the DESHL executable has been correctly added to your path by running from the command prompt:
> deshl –h
As previously stated, if this is the first time that you run a DESHL command, note that there will be a short delay while a new local server is started, when you will be asked to enter the password for your keystore. (Once the server is running, it is not necessary to re-enter your password during this session.)
> deshl –h Starting new server ... Please enter password for keystore /home/malcolm/mykeystore.jks: [-h] command [command-options] Where: -h,--help Print this message. With commands: fetch cp isdir mv status jobs rm mkdir ls submit sites terminate isfile exists Run deshl <command> -h for individual command options.
A simple test to verify that the DESHL client is correctly installed and configured is to try to list the contents of the DEISA_HOME storage at the configured site. For example:
> deshl list unicore.grid.sara.nl_4014_SARA-HUYGENS
Note that a fragment of the site name can be used, provided that it cannot be confused with another site's name, for example:
> deshl list sara::deisa_home
For the site, the DESHL client should display a list of the files and directories in the top level of the DEISA_HOME storage.
In your exercises directory, create files called hello-submit.sh and hello.sh, and add the contents to the respective files as shown below:
hello-submit.sh #!/bin/bash # Test job script for DESHL using SAGA. # # SAGA JobDefinition based directives: #$ SAGA_FileTransfer = $DEISA_HOME/jobs/hello.sh > hello.sh #$ SAGA_HostList = unicore.grid.sara.nl_4014_SARA-HUYGENS #$ SAGA_JobCmd = hello.sh
hello.sh #!/bin/bash echo "Here I am on `hostname` in $PWD. Current date is `date`"
Please note that you must edit hello-submit.sh to specify the Execution Site, thus the SAGA_hostLIst line should be changed to reflect the target site.
The hello-submit.sh file contains a set of SAGA directives for submitting and managing the execution of a job, the job in this case being the executable script, hello.sh.
The rest of this exercise will show you how to place (or 'stage') the executable script onto a DEISA site, and then use the SAGA script with the DESHL command line tool to submit the job and retrieve the output from the job.
The following steps will be performed:
Verify if a remote directory exists
Create a remote directory for the executable script
Upload the executable script to the remote execution host
Edit the SAGA script
Submit the job
Get the job status
Clean up the job and retrieve the job output
1. Verify if a remote directory exists
First we want to determine if the remote directory called DEISA_HOME exists:
> deshl exists sara::deisa_home/jobs
If the remote directory exists, the following message should be displayed:
exists: target sara:::deisa_home/jobs exists
Otherwise, the following message is displayed:
exists: target sara::deisa_home/jobs does not exist
2. Create a remote directory for the script
If the remote directory does not exist, create it using the deshl mkdir command:
> deshl mkdir sara::deisa_home/jobs
Verify that the remote directory now exists by listing the contents of its parent directory. This should show the jobs directory in the listing.
> deshl list sara::deisa_home
drwx 32768 Feb 07 2010 01:22:01 jobs
3. Upload the executable script to the server
You now must upload the executable script, hello.sh, from your local machine to the DEISA site where the job will be run. To do this, use the deshl copy command:
> deshl cp hello.sh sara::deisa_home/jobs/hello.sh
If a file of the same name already exists in the target location, the following error will be displayed:
Error running command: cp: Copy failed.
Likely cause: Remote file exists, specify overwrite flag to overwrite.
(Check logs for further details.)
If this happens, then repeat the copy with –f to overwrite the existing file:
> deshl cp –f hello.sh sara::deisa_home/jobs/hello.sh
4. Submit the job
Now that your script is present at the remote site, you can submit a job to run it via the remote site's batch system.
> deshl submit hello-submit.sh
Your job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has been successfully submitted.
This returns a job identifier, which can be used to monitor the job’s status.
5. Get the job status
To get the status of the job, type:
> deshl status 0bea01c2-4a62-4be5-90f0-558886c8cb69
Job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has status: Running
where the example job identifier is replaced by the job identified returned by the deshl submit command.
You can obtain extended information about the job using the status command with the –f option:
> deshl status –f 0bea01c2-4a62-4be5-90f0-558886c8cb69
Job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has status: Running
Name: NAMD on 32 processors
Once the job has successfully completed, the status will be reported as Done.
NB 'Done' indicates the scheduler was able to queue and launch the job. To check the exit code for the application, use the 'exitcode' command: (0 is normally used to indicate successful completion)
> deshl exitcode 0bea01c2-4a62-4be5-90f0-558886c8cb69 Job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has exitcode: 0
6. Clean up the job and retrieve the job output
Once the job has completed, the output from the job can be retrieved, and any file resources consumed by the job released, using the 'fetch' command. The 'fetch' command allows you to retrieve the job output to your workstation. When a job is fetched, a directory is created in the DESHL output folder, where it name is based on the job's identifier. This is normally a subdirectory called 'output' in the DESHL installation directory.
$ deshl fetch -d output 0bea01c2-4a62-4be5-90f0-558886c8cb69
This will place two files in the job's sub-directory; one of these is the job’s output to standard output, and the other is the job’s output to standard error. Look at the files in the output directory to verify this.
In this part of the tutorial, you will stage in data required by a job, run the job and then retrieve data produced by the job. The example script, in this case, will perform a simple concatenate on two specified files and the file containing the joined files will be the final result.
In your exercises directory, create files called catfiles_submit.sh and concat.sh; add the contents as listed below.
catfiles-submit.sh #!/bin/bash # Test job script for DESHL using SAGA. # # SAGA JobDefinition based directives: #$ SAGA_FileTransfer = $DEISA_HOME > concat.sh #$ SAGA_FileTransfer = $DEISA_HOME > filea #$ SAGA_FileTransfer = $DEISA_HOME > fileb #$ SAGA_FileTransfer = $DEISA_HOME < filec #$ SAGA_HostList = epcc-hpcx #$ SAGA_JobCmd = concat.sh
concat.sh #!/bin/bash cat filea fileb > filec
Again, please note that you will have to change the value of SAGA_HostList to specify your execution platform.
Use the deshl cp command to import the executable script file, concat.sh, to the jobs directory on the remote site.
> deshl cp concat.sh sara::deisa_home/jobs/concat.sh
> deshl ls sara::deisa_home/jobs/concat.sh
Use the 'deshl mkdir' and 'deshl cp' commands to create a tutorial directory on the remote host, then copy two files from your local machine into the remote tutorial directory as 'filea' and 'fileb'. Confirm that the files have been copied using the 'deshl ls' command.
> deshl mkdir sara::deisa_home/tutorial
> deshl cp localFileA.dat sara::deisa_home/tutorial/filea
> deshl cp localFileB.dat sara::deisa_home/tutorial/fileb
> deshl ls sara::deisa_home/tutorial
You can now submit the job as before:
> deshl submit catfiles-submit.sh
Again, you can monitor the job status using the 'deshl status' command. When the job has completed, an output data file called 'filec' will have been produced in the remote tutorial directory. Confirm that this file exists by listing its properties:
> deshl ls sara::deisa_home/tutorial/filec
-rw- 63 Mar 01 2007 03:29:57 filec
Export this file to your local machine using the 'deshl cp' command:
> deshl cp sara::deisa_home/tutorial/filec filec.dat
Inspect filec.dat to confirm it contains the concatenated contents of the two original files. Remember to clean up the job using the 'deshl fetch' command.