lllll

Articles

Users can submit computionnaly intensive tasks to the cluster nodes using Sun Grid Engine (SGE).  SGE is a distributed resource management software and it allows the resources within the cluster (cpu time,software, licenses etc) to be utilized effectively. This section tells you how to get started  using SGE.

 

Please do not run ANY computationally intensive tasks on the master!!! If this is done, we will have to kill your jobs, because they will slow down all other users

 

 

 

Submitting Batch Jobs to SGE

 

Batch jobs are submitted to SGE via scripts.

 

Here is an example of a serial job script, date.sh1. It basically executes the sleep command.

 

[tranchant@master0 ~]$ cat date.sh

#!/bin/bash

#

#$ -cwd

#$ -j y

#$ -S /bin/bash

date 

 

Entries which start with #

$ will be treated as SGE options.

-cwd means to execute the job for the current working directory.

-j y means to merge the standard error stream into the standard output stream instead of having two separate error and output streams.

-S /bin/bash specifies the interpreting shell for this job to be the Bash shell.

 

 

To submit this serial job script, you should use the qsub command.

[tranchant@master0 ~]$ qsub date.sh

your job 417 ("date.sh") has been submitted

 

 

Although the example job scripts are bash scripts, SGE can also accept other types of shell script

 

 

 

 

SGE tools

 

 

qstat

 

This command allows you to get the status of SGE and the jobs that are running or are waiting to be run (queued). Let's explore what qstat can do for us. To monitor jobs under SGE, use the qstat command. When executed with no arguments, it will display a summarized list of jobs.

 

[tranchant@master0 Repbase]$ qstat

job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 

-----------------------------------------------------------------------------------------------------------------

    424 0.55500 job_blastn tranchant    r     02/27/2012 13:39:25 This email address is being protected from spambots. You need JavaScript enabled to view it.       12   


 

 

 

 

Use qstat -f to display a more detailed list of jobs within SGE.

[tranchant@master0 ~]$qstat -f queuename

 

queuename                      qtype resv/used/tot. load_avg arch          states

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.    BIP   0/0/12         0.04     lx26-amd64    

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.    BIP   0/12/12        0.00     lx26-amd64    

    427 0.55500 job_blastn tranchant    r     02/27/2012 13:41:40    12        

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.    BIP   0/0/12         0.03     lx26-amd64    

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.    BIP   0/12/12        0.00     lx26-amd64    

    426 0.55500 job_blastn tranchant    r     02/27/2012 13:41:40    12        

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.       BIP   0/0/12         0.04     lx26-amd64    

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.       BIP   0/0/12         0.00     lx26-amd64    

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.       BIP   0/0/12         0.03     lx26-amd64    

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.       BIP   0/0/12         0.00     lx26-amd64    

---------------------------------------------------------------------------------

This email address is being protected from spambots. You need JavaScript enabled to view it.       BIP   0/0/16         0.06     lx26-amd64    


 

 

 

 

You can also use qstat to query the status of a job, given it’s job id. For this, you would use the -j N option where N

would be the job id.

 

tranchant@master0 Repbase]$ qstat -j 427

==============================================================

job_number:                 427

exec_file:                  job_scripts/427

submission_time:            Mon Feb 27 13:41:37 2012

owner:                      tranchant

uid:                        35115

group:                      evodyn

gid:                        507

sge_o_home:                 /home/tranchant

sge_o_log_name:             tranchant

sge_o_path:                 /home/tranchant/perl5/bin:/usr/local/Structure/bin:/usr/local/Structure:/usr/local/ABCtoolbox/binaries/linux:/usr/local/blast-2.2.17/bin:/usr/local/BEASTv1.6.2/bin:/usr/local/trinityrnaseq_r2011-11-26:/usr/local/pft2.2:/usr/local/vcftools_0.1.8/bin:/usr/local/velvet_1.2.02/:/usr/local/soap2.21release/:/usr/local/sequin/:/usr/local/phylip-3.69/:/usr/local/MUMmer3.23/:/usr/local/mosaik-aligner/bin/:/usr/local/LTR_FINDER.x86_64-1.0.5/:/usr/local/cufflinks-1.3.0.Linux_x86_64/:/usr/local/CAP3/:/usr/local/bwa-0.6.1/:/usr/local/bowtie-0.12.7/:/usr/local/BEDTools-Version-2.15.0/bin:/usr/local/artemis:/opt/python3/bin:/usr/local/java/latest/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/c3-4/:/home/tranchant/bin

sge_o_shell:                /bin/bash

sge_o_workdir:              /home/tranchant/Analysis/REPET/Repbase

sge_o_host:                 master0

account:                    sge

cwd:                        /home/tranchant/Analysis/REPET/Repbase

merge:                      y

mail_options:               abe

mail_list:                   This email address is being protected from spambots. You need JavaScript enabled to view it.

notify:                     FALSE

job_name:                   job_blastn.qsub

jobshare:                   0

hard_queue_list:            bioinfo.q

shell_list:                 NONE:/bin/bash

env_list:                   

script_file:                job_blastn.qsub

parallel environment:  ompi range: 12

usage    1:                 cpu=00:00:15, mem=14.61082 GBs, io=0.05051, vmem=N/A, maxvmem=1.480G

scheduling info:            queue instance " This email address is being protected from spambots. You need JavaScript enabled to view it. " dropped because it is full

                            queue instance " This email address is being protected from spambots. You need JavaScript enabled to view it. " dropped because it is full


 

 

 

qdel 

 

This command allows you to delete a job from SGE. For example, if you have submitted a job that you didn't want to run or one that you want to stop while it's running. You find the Job ID from the qstat command.  

 

[tranchant@master0 Repbase]$ qstat

job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 

-----------------------------------------------------------------------------------------------------------------

    428 0.00000 job_blastn tranchant    qw    02/27/2012 13:44:03                                   12        

[tranchant@master0 Repbase]$ qdel 428

tranchant has deleted job 428

 



 

qhost

 

The qhost command allows you to get a status of the nodes that are being used by SGE. Here is an example for my cluster.

 

 

[tranchant@master0 Repbase]$ qhost

HOSTNAME                ARCH         NCPU  LOAD  MEMTOT  MEMUSE  SWAPTO  SWAPUS

-------------------------------------------------------------------------------

global                  -               -     -       -       -       -       -

node0                   lx26-amd64     12  0.00   47.1G    3.2G    8.0G     0.0

node1                   lx26-amd64     12  0.01   47.1G    3.3G    8.0G  347.4M

node2                   lx26-amd64     12  0.00   47.1G    3.9G    8.0G     0.0

node3                   lx26-amd64     12  0.01   94.5G    4.2G    8.0G     0.0

node4                   lx26-amd64     12  0.00   47.1G    3.4G    8.0G     0.0

node5                   lx26-amd64     16  0.07   31.4G    3.2G    8.0G     0.0

 

 

The output lists the number of CPUs per node (NPROC), the total memory available on the node (MEMTOT), the memory in use (MEMUSE), the swap space available (SWAPTO), and the used swap space (SWAPUS).

 

For more information in using SGE, please refer to the SGE documentation and the man pages.

 

 

Notes

 

Copyright © 2019 IRD Bioinformatics. All Rights Reserved.
Joomla! is Free Software released under the GNU General Public License.