User Tools

Site Tools


archive:parallel_environment

This is an old revision of the document!


A PCRE internal error occured. This might be caused by a faulty plugin

====== Parallel Environments (PE) ====== Parallel environments (PE's) are environments used to execute shared and/or distributed memory applications. ===== Documentation ===== [[http://gridscheduler.sourceforge.net/htmlman/htmlman5/sge_pe.html|sge_pe man page]] ===== PE's and Campus Rocks ===== Useful commands: * **qconf -spl** List PE's installed on the queue. * **qconf -sp PE_NAME** List the attributes of a specific PE. * **qconf -Ap PE_FILE** Create a new PE (note that on Campus Rocks, an ITTicket is required to install a new PE). * **-pe PE_NAME** Using the PE option in a qsub job request. * **$-pe PE_NAME** Using the PE in an embedded script qsub job request. ==== PE Attributes ==== * pe_name * slots * user_lists * xuser_lists * start_proc_args * stop_proc_args * allocation_rule * control_slaves * job_is_first_task * urgency_slots * accounting_summary Key attributes to check when using a PE for a parallel application are the //allocation_rule// and //control_slaves// options. Depending on your specific application, you may need to ensure slots are allocated in specific way. === Allocation Types === The //allocation_rule// option of a PE can have several different values: * **$pe_slots**: All slots are allocated to a single node * **$fill_up**: Slots are allocated to fill up the most appropriate node before spilling into the next node. * **$round_robin**: Slots are allocated one at a time, each one to a different node * **[INT value]**: Each node can be allocated with a maximum of the specified number of slots. Choosing the best allocation scheme will of course depend on the structure of the cluster being used. For example, Campus Rocks is divided into two queues, small.q and all.q with 1 node and 6 nodes respectively. The largest nodes have 64 slots available and the smallest have 8. Therefore, depending on the application, more or less nodes would be necessary for a job and thus the allocation scheme would have to be considered. === Tight Integration === The //control_slaves//, when set to TRUE, indicates a parallel application with a tight interface. There are more succinct definitions of this property elsewhere, but certain applications are optimized to run in the cluster environment (in our case SGE) and therefore have the potential to allow for a certain degree of internal management in the execution of the application. One such example is openmpi. ==== Currently Installed PE's ==== <code> user$ qconf -spl mpi orte mpich denovo </code> ==== mpi ==== <code> user$ qconf -sp mpi pe_name mpi slots 9999 user_lists NONE xuser_lists NONE start_proc_args /opt/gridengine/mpi/startmpi.sh $pe_hostfile stop_proc_args /opt/gridengine/mpi/stopmpi.sh allocation_rule $fill_up control_slaves FALSE job_is_first_task TRUE urgency_slots min accounting_summary TRUE </code> ==== orte ==== <code> user$ qconf -sp orte pe_name orte slots 9999 user_lists NONE xuser_lists NONE start_proc_args /bin/true stop_proc_args /bin/true allocation_rule $pe_slots control_slaves TRUE job_is_first_task FALSE urgency_slots min accounting_summary TRUE </code> ==== mpich ==== <code> user$ qconf -sp mpich pe_name mpich slots 9999 user_lists NONE xuser_lists NONE start_proc_args /opt/gridengine/mpi/startmpi.sh -catch_rsh $pe_hostfile stop_proc_args /opt/gridengine/mpi/stopmpi.sh allocation_rule $fill_up control_slaves TRUE job_is_first_task FALSE urgency_slots min accounting_summary TRUE </code> ==== denovo ==== <code> user$ qconf -sp denovo pe_name denovo slots 9999 user_lists NONE xuser_lists NONE start_proc_args /opt/gridengine/mpi/startmpi.sh $pe_hostfile stop_proc_args /opt/gridengine/mpi/stopmpi.sh allocation_rule $round_robin control_slaves TRUE job_is_first_task FALSE urgency_slots min accounting_summary TRUE </code>

You could leave a comment if you were logged in.
archive/parallel_environment.1433143901.txt.gz · Last modified: 2015/06/01 07:31 by emfeal