Note that UNICORE 6, the middleware on which DESHL is built, does not allow unlimited access to the file space at a DEISA site: instead, a set of directories, or “storages”, are configured. Normally, access is provided to $ROOT and $HOME (local to each DEISA site), and $DEISA_DATA and $DEISA_HOME (shared across most DEISA sites).  You can list your Execution Sites by running the 'sites' command. The storages at a DEISA site can be listed by running the 'list' command with the name of the site, for example:

$ deshl sites
fe04-ext.fec.cineca.it_13303_CINECA-SP6
unicore6.hector.ac.uk_8080_EPCC-HECToR
gg.rzg.mpg.de_8080_RZG-VIP
uc6.lrz-muenchen.de_5555_lrz_hlrb2_deisa
unicore.grid.sara.nl_4014_SARA-HUYGENS
unicore.csc.fi_8080_CSC

$ deshl ls unicore.grid.sara.nl_4014_SARA-HUYGENS
home
deisa_scratch
deisa_data
deisa_home

When you first run a DESHL command, there will be a short delay while a new local server is started and you will be asked to enter the passphrase for your keystore. (Once the server is running, it is not necessary to re-enter your passphrase during your session.)

When a job is executed, a temporary directory called USPACE is created. The application and data required by the application are staged into the USPACE before the job is run. After the application has completed, any files of interest must be copied out of USPACE before the job completes. (Note that if the executable is already in the path on the execution host, it is not necessary to stage the executable.)

This subsection describes how to upload, submit and manage jobs via simple examples.

Firstly, create a new directory on your local workstation and call it 'exercises'. Go into this directory, and verify that the DESHL executable has been correctly added to your path by running  from the command prompt:

> deshl –h  

As previously stated, if this is the first time that you run a DESHL command, note that there will be a short delay while a new local server is started, when you will be asked to enter the password for your keystore. (Once the server is running, it is not necessary to re-enter your password during this session.)

 > deshl –h

Starting new server ...

Please enter password for keystore /home/malcolm/mykeystore.jks:

[-h] command [command-options]
Where:
  -h,--help   Print this message.

With commands:
  fetch
  cp
  isdir
  mv
  status
  jobs
  rm
  mkdir
  ls
  submit
  sites
  terminate
  isfile
  exists

Run deshl <command> -h for individual command options.

A simple test to verify that the DESHL client is correctly installed and configured is to try to list the contents of the DEISA_HOME storage at the configured site. For example:

> deshl list unicore.grid.sara.nl_4014_SARA-HUYGENS

Note that a fragment of the site name can be used, provided that it cannot be confused with another site's name, for example:

> deshl list sara::deisa_home 

For the site, the DESHL client should display a list of the files and directories in the top level of the DEISA_HOME storage.

In your exercises directory, create files called hello-submit.sh and hello.sh, and add the contents to the respective files as shown below:

hello-submit.sh

#!/bin/bash
# Test job script for DESHL using SAGA.
#
# SAGA JobDefinition based directives:
#$ SAGA_FileTransfer = $DEISA_HOME/jobs/hello.sh > hello.sh
#$ SAGA_HostList = unicore.grid.sara.nl_4014_SARA-HUYGENS
#$ SAGA_JobCmd = hello.sh

hello.sh
 
#!/bin/bash

echo "Here I am on `hostname` in $PWD. Current date is `date`"

Please note that you must edit hello-submit.sh to specify the Execution Site, thus the SAGA_hostLIst line should be changed to reflect the target site.

The hello-submit.sh file contains a set of SAGA directives for submitting and managing the execution of a job, the job in this case being the executable script, hello.sh.

The rest of this exercise will show you how to place (or 'stage') the executable script onto a DEISA site, and then use the SAGA script with the DESHL command line tool to submit the job and retrieve the output from the job.

The following steps will be performed:

1. Verify if a remote directory exists

2. Create a remote directory for the executable script

3. Upload the executable script to the remote execution host

4. Edit the SAGA script

5. Submit the job

6. Get the job status

7. Clean up the job and retrieve the job output

1. Verify if a remote directory exists

First we want to determine if the remote directory called DEISA_HOME exists:

> deshl exists sara::deisa_home/jobs 

If the remote directory exists, the following message should be displayed:

exists: target sara:::deisa_home/jobs exists

Otherwise, the following message is displayed:

exists: target sara::deisa_home/jobs does not exist

2. Create a remote directory for the script

If the remote directory does not exist, create it using the deshl mkdir command:

> deshl mkdir sara::deisa_home/jobs

Verify that the remote directory now exists by listing the contents of its parent directory. This should show the jobs directory in the listing.

> deshl list sara::deisa_home
…
drwx   32768   Feb 07 2010 01:22:01   jobs
… 

3. Upload the executable script to the server

You now must upload the executable script, hello.sh, from your local machine to the DEISA site where the job will be run. To do this, use the deshl copy command:

> deshl cp hello.sh sara::deisa_home/jobs/hello.sh 

If a file of the same name already exists in the target location, the following error will be displayed:

Error running command: cp: Copy failed.
Likely cause: Remote file exists, specify overwrite flag to overwrite.
(Check logs for further details.) 

If this happens, then repeat the copy with –f to overwrite the existing file:

> deshl cp –f hello.sh sara::deisa_home/jobs/hello.sh 

4. Submit the job

Now that your script is present at the remote site, you can submit a job to run it via the remote site's batch system.

> deshl submit hello-submit.sh

Your job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has been successfully submitted.

This returns a job identifier, which can be used to monitor the job’s status.

5. Get the job status

To get the status of the job, type:

> deshl status 0bea01c2-4a62-4be5-90f0-558886c8cb69
Job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has status: Running

where the example job identifier is replaced by the job identified returned by the deshl submit command.

You can obtain extended information about the job using the status command with the –f option:

> deshl status –f 0bea01c2-4a62-4be5-90f0-558886c8cb69

Job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has status: Running
     Site: unicore.grid.sara.nl_4014_SARA-HUYGENS
     Name: NAMD on 32 processors
    

Once the job has successfully completed, the status will be reported as Done.

NB 'Done' indicates the scheduler was able to queue and launch the job. To check the exit code for the application, use the 'exitcode' command: (0 is normally used to indicate successful completion)

> deshl exitcode 0bea01c2-4a62-4be5-90f0-558886c8cb69
Job: 0bea01c2-4a62-4be5-90f0-558886c8cb69, has exitcode: 0

6. Clean up the job and retrieve the job output

Once the job has completed, the output from the job can be retrieved, and any file resources consumed by the job released, using the 'fetch' command. The 'fetch' command allows you to retrieve the job output  to your workstation.  When a job is fetched, a directory  is created in the DESHL output folder, where it name is based on the job's identifier. This is normally a subdirectory called 'output' in the DESHL installation directory. 

$ deshl fetch -d output 0bea01c2-4a62-4be5-90f0-558886c8cb69

This will place two files in the job's sub-directory; one of these is the job’s output to standard output, and the other is the job’s output to standard error. Look at the files in the output directory to verify this.

In this part of the tutorial, you will stage in data required by a job, run the job and then retrieve data produced by the job. The example script, in this case, will perform a simple concatenate on two specified files and the file containing the joined files will be the final result.

In your exercises directory, create files called catfiles_submit.sh and concat.sh; add the contents as listed below.

catfiles-submit.sh

#!/bin/bash
# Test job script for DESHL using SAGA.
#
# SAGA JobDefinition based directives:
#$ SAGA_FileTransfer = $DEISA_HOME > concat.sh
#$ SAGA_FileTransfer = $DEISA_HOME > filea
#$ SAGA_FileTransfer = $DEISA_HOME > fileb

#$ SAGA_FileTransfer = $DEISA_HOME < filec
#$ SAGA_HostList = epcc-hpcx
#$ SAGA_JobCmd = concat.sh

concat.sh
#!/bin/bash

cat filea fileb > filec

Again, please note that you will have to change the value of SAGA_HostList to specify your execution platform.

Use the deshl cp command to import the executable script file, concat.sh, to the jobs directory on the remote site.

> deshl cp concat.sh sara::deisa_home/jobs/concat.sh
> deshl ls sara::deisa_home/jobs/concat.sh

Use the 'deshl mkdir' and 'deshl cp' commands to create a tutorial directory on the remote host, then copy two files from your local machine into the remote tutorial directory as 'filea' and 'fileb'. Confirm that the files have been copied using the 'deshl ls' command.

For example:

> deshl mkdir sara::deisa_home/tutorial
> deshl cp localFileA.dat sara::deisa_home/tutorial/filea
> deshl cp localFileB.dat sara::deisa_home/tutorial/fileb
> deshl ls sara::deisa_home/tutorial

You can now submit the job as before:

> deshl submit catfiles-submit.sh

Again, you can monitor the job status using the 'deshl status' command. When the job has completed, an output data file called 'filec' will have been produced in the remote tutorial directory. Confirm that this file exists by listing its properties:

> deshl ls sara::deisa_home/tutorial/filec

-rw-   63      Mar 01 2007 03:29:57   filec

Export this file to your local machine using the 'deshl cp' command:

> deshl cp sara::deisa_home/tutorial/filec filec.dat

Inspect filec.dat to confirm it contains the concatenated contents of the two original files. Remember to clean up the job using the 'deshl fetch' command.