Torque

High Performance Computing - Seminar Plan • The Message Passing Interface (MPI) is a large and complex system. • Core (and most useful functions) are pretty easy to grasp with a bit of experience. • MPI will be covered in main module lectures in greater depth. • Purpose of this seminar is to show you how to compile MPI programs and how to submit/manage jobs running on the HPSG cluster.

High Performance Computing MPI and C-Language Seminars 2009 Photo Credit: NOAA (IBM Hardware)

1

3

High Performance Computing - Seminar Plan • Seminar Plan for Weeks 1-5 • Week 1 - Introduction, Data Types, Control Flow, Pointers • Week 2 - Arrays, Structures, Enumerations, I/O, File I/O • Week 3 - Dynamic Memory, Preprocessing, Compile Time Options

Compiling MPI Applications

• Week 4 - MPI in C and Using the Cluster • Week 5 - “How to Build a Performance Model” • Week 6-9 - Coursework Troubleshooting (a number of seminar tutors will be available during these sessions - go to each tutor’s office)

2

4

Compiling MPI Applications

Compiling MPI Applications written in C

•

MPI Applications have their own set of compilers - this handles the include locations and libraries without any additional user interaction.

•

Applications compiled with the MPI compilers cannot be run from the command line because OpenMPI is not available.

•

This will also compile your code for running on Myrinet and under OpenPBS/Torque.

•

You might get errors or the code may partially run.

•

Compilers for C, C++ and Fortran are installed (only C is needed for the coursework unless you choose to completely rewrite the application!).

•

Either way - best to run your codes using the OpenPBS/Torque system in the next section.

5

7

Compiling MPI Applications written in C

OpenMPI Installation Information

•

To compile an application written in C replace gcc with mpicc

•

•

Example:

If you want to know any OpenMPI installation information for your coursework then use the following command: ompi_info

mpicc -o hw-mpi -O2 helloworld-mpi.c

• •

The compiled in modules and installation information will be shown.

All of the include paths and libraries get compiled in by default.

6

8

Logging In • Access to the server is by SSH only. • Your account is of the form: hpcXXXXXXX where XXXXXXX is your student University number (e.g. hpc0234567). • You can collect your password by coming to the HPSG lab (CS2.04) in week 5. We will make an announcement via the Teaching Announcements and in lectures.

How to use the HPSG Cluster

• The server is located at: deepthought.dcs.warwick.ac.uk • Use of the cluster is subject to University, departmental and ITS usage policies. Be careful what you execute, if you have questions ask. You WILL loose your ITS and DCS accounts if you break the rules.

9

11

High Performance Systems Group Cluster

Checking the Current Runtime Queue

•

During the coursework you will have access to the HPSG IBM Cluster, this is not a high performance system but it will be good for performance models.

• To check the current execution queue:

•

42 x dual processor (Pentium III, 1.4Ghz), 2GB system RAM per node, Myrinet fibre-optics interconnect.

• The current queue will be shown.

•

Various head nodes and system management machines.

•

The system uses an OpenPBS/Torque queue to manage and batch jobs.

•

Note: Best performance is achieved when you densely pack your jobs (i.e. use both processors on the same node).

qstat

Job id ------------------------5277.frankie 5278.frankie 5279.frankie 5280.frankie 5281.frankie 5282.frankie 5283.frankie 5284.frankie 5285.frankie 5286.frankie

10

Name ---------------BOINC BOINC BOINC BOINC BOINC BOINC BOINC BOINC BOINC BOINC

User --------------sdh sdh sdh sdh sdh sdh sdh sdh sdh sdh

Time Use -------612:08:2 600:48:5 553:19:1 496:34:2 584:17:1 505:19:2 501:20:0 523:02:3 507:42:1 489:33:5

S R R R R R R R R R R

Queue ----boinc boinc boinc boinc boinc boinc boinc boinc boinc boinc

12





qstat

qstat

• The current queue will be shown. Job id ------------------------5277.frankie 5278.frankie 5279.frankie 5280.frankie 5281.frankie 5282.frankie 5283.frankie 5284.frankie 5285.frankie 5286.frankie


• The current queue will be shown. User --------------sdh sdh sdh sdh sdh sdh sdh sdh sdh sdh

Time Use -------612:08:2 600:48:5 553:19:1 496:34:2 584:17:1 505:19:2 501:20:0 523:02:3 507:42:1 489:33:5






Time Use -------612:08:2 600:48:5 553:19:1 496:34:2 584:17:1 505:19:2 501:20:0 523:02:3 507:42:1 489:33:5



13

15





qstat

qstat

• The current queue will be shown. Job id ------------------------5277.frankie 5278.frankie 5279.frankie 5280.frankie 5281.frankie 5282.frankie 5283.frankie 5284.frankie 5285.frankie 5286.frankie


• The current queue will be shown. User --------------sdh sdh sdh sdh sdh sdh sdh sdh sdh sdh

Time Use -------612:08:2 600:48:5 553:19:1 496:34:2 584:17:1 505:19:2 501:20:0 523:02:3 507:42:1 489:33:5




14



Time Use -------612:08:2 600:48:5 553:19:1 496:34:2 584:17:1 505:19:2 501:20:0 523:02:3 507:42:1 489:33:5



16

Job Status Information

Getting Something Run on the Cluster

• Jobs can be in one of many states:

• The cluster queuing system batches up jobs and runs them so that the resource use is shared fairly between users.

• C - Job completed after having run (in tidy up and completed state) • You can run jobs on the cluster in two ways:

• E - Job is exiting after having run (“ending”)

• Via a submit script (a ‘batch’ job) • Interactively

• H - Job is held by user or system • Q - Job is being held in the queue for resources • R - Job is running

• Do not run jobs outside of the queue - this will give you incorrect results and cause unfair resource use. We will monitor this and disable accounts for users who do not use the queue correctly.

• T - Job is being moved to a new queue/server • W - Job is waiting for its execution to be reached

• Jobs are submitted using the qsub command.

• S - Job is suspended (not supported on our cluster)

17

19

Checking the Current Machine Status

Interactive Jobs


• Interactive jobs allocate one (or more) nodes for you to use interactively - i.e. you can run commands on the node in a similar way to using a shell.

pbsnodes -a • The current queue will be shown.

• Useful if you need to see the application executing (for debug etc). • Can be used for X-windows jobs (if you really need to).

vogon41.deepthought.hpsg.dcs.warwick.ac.uk state = free np = 2 ntype = cluster status = opsys=linux,uname=Linux vogon41 2.6.24-vogon-stripped0 #5 SMP Thu Sep 18 10:19:48 BST 2008 i686,sessions=881 1061,nsessions=2,nusers=2,idletime=928290,totmem=4174136kb,availmem=4096888kb,physmem=207 6496kb,ncpus=2,loadave=0.00,netload=696181053,state=free,jobs=,varattr=,rectime=123287857 0

• Your job submission will block until a node becomes free for you to use. • To request an interactive session on one node: qsub -V -I

18

20

Batch Jobs

Sample OpenPBS Submit File:

• Batch jobs allow you to submit an application to be executed into the queue and then leave it to be run (i.e. you don’t need to sit there typing commands in and watching the output).

#!/bin/bash #PBS -V

• Very efficient if you want to submit lots of jobs.

mpirun ./IMB-MPI1 -msglen imblengths

cd $PBS_O_WORKDIR

• Scheduler will run jobs so that good resource usage is achieved. • Requires you to write a submit script (to say what you want to be executed). • Example submit command (single processor job): qsub -V submit.pbs

21


23


#!/bin/bash #PBS -V

#!/bin/bash #PBS -V

cd $PBS_O_WORKDIR

cd $PBS_O_WORKDIR



22

24


Some more options...

#!/bin/bash #PBS -V

•

-N (sets the job name)

cd $PBS_O_WORKDIR

•

-X (forwards X11 information for interactive jobs)


•

-a (Mark job as available to be run after )

•

-h (Submit job into a hold state)

25

27

Output and Error Streams

Submitting Parallel Jobs

•

The output and error streams of your job will be written to a file after your job has completed.

•

Write a script as before.

•

This may take a minute or two after completion - be patient!

•

Need additional parameters to tell the scheduler how many processors to allocate to the job.

•

Default file names are .o and .e

•

MPI runtime environment automatically knows how many processors are allocated by looking this up in the PBS shell variables (and TM interface).

•

You cannot request more than 64 processors in the system (the queues will reject your jobs).

26

28

Submitting Parallel Jobs

Removing Jobs • To remove a job from the current execution queue:

qsub -l nodes=X:ppn=Y -V submit.pbs

•

-l nodes=X:ppn=Y is the “resource request string”.

•

nodes=X requests a number of machines

•

ppn=Y requests a number of processors per node (either 1 or 2)

•

You can add the resource string into the submit file at the PBS commands.

•

You can add the string for interactive jobs (the commands only run on one node unless you run MPI or PBSDSH).

qdel • Get the job number from the queue (qstat)




Time Use -------616:39:0 601:53:2 553:51:2 497:06:4 585:21:4 506:23:5 501:52:1 523:34:4 507:42:1 489:33:5



29

31

Resource Constraints

Getting Extended Job Information

•

You can limit the resource requests for your job on time, memory, processors and user specified attributes.

• To get extended information on jobs in the system:

•

The queues running on the cluster automatically apply defaults to your jobs during submission.

qstat -a

Job ID Time -------------------5277.frankie 352:3 5278.frankie 352:4 5279.frankie 352:4 5280.frankie 352:3 5281.frankie 352:4 5282.frankie 352:4

-l mem=512Mb -l walltime=10:00:00

30

Username Queue

Jobname

SessID NDS

TSK Memory Time

S

-------- -------- ---------------- ------ ----- --- ------ ----- sdh boinc BOINC 12776 1 --- 720:0 R sdh

boinc

BOINC

16561

1

--

--

720:0 R

sdh

boinc

BOINC

15150

1

--

--

720:0 R

sdh

boinc

BOINC

13542

1

--

--

720:0 R

sdh

boinc

BOINC

10436

1

--

--

720:0 R

sdh

boinc

BOINC

12805

1

--

--

720:0 R

32

Resources • Condor/HPSG Resources: http://www2.warwick.ac.uk/fac/sci/dcs/people/research/csrcbc/ hpcsystems/dthought/

End of Seminar

• High Performance Computing coursework + MPI resources (FAQs, Common Errors etc)

Thanks for coming, next week How to Build a Performance Model

http://www2.warwick.ac.uk/fac/sci/dcs/people/research/csrcbc/ teaching/hpcseminars/

33

Final Notes... • This is the first year students are running jobs on a cluster using OpenPBS - there probably will be some bugs and faults in our queue definitions. • Email us as soon as its goes wrong, we will try to fix it (crosses fingers)! • Do not try to run jobs directly, if we find them we will lock the account always use the queue, this ensures fair execution and more reliable results (which are crucial to a good performance model). • The job queue will get busy but your jobs will get run (in the end). Please be responsible, only submit what you need and delete jobs which you know will go wrong.

34

35

Torque

Torque

Suggest Documents

Torque

torque

Torque

Torque

torque

Torque

TORQUE

Casimir torque

Torque Arm

Heavy Torque

Torque specifications

Torque Tables

Torque Tables

Torque Specifcations

Torque Tables

Torque Table

Cogging Torque and Torque Ripple in a Direct-Drive Interior ...

pdf-11164\haunted-houses-torque-books-the-unexplained-torque ...

Intelligent Torque Sensing and Robust Torque Control of Harmonic

A Non-Unity Torque Sharing Function for Torque Ripple ... - MDPI

The Science of Making Torque from Wind (TORQUE

man-25\2001-mitsubishi-eclipse-torque-converter-torque-specs.pdf ...

torque returned goods authorization - instructions - Bottle Cap Torque ... [PDF]

volvo eco-torque & dual-torque ratings - Volvo Trucks