Amber JobStep Example

NERSC maintains a maximum wall clock time limit on some production queues in order to make the system widely available to the user community. Some MD simulations require very long run times. This page shows and example of using job steps to accomplish long simulations in smaller blocks.

LoadLevler allows job steps, which are essentially distinct batch jobs which are guaranteed to execute in a specific order and may communicate with each other via job return values. Thus based on the outcome of previous job steps , the next job step can pick up where the last one left off.

If the input coordinates and velocities are stored in the file inpcrd and the input command file is set to run for 500 ps, then the following load leveler script will run the simulation for 1.5 ns. Be sure to set irest = 1 and ntwr small enough to capture data near the end of each step.

#!/bin/ksh 
#@job_name         = steps
#@job_type	   = parallel
#@output	   = $(job_name).$(schedd_host).$(jobid).$(stepid).out
#@error  	   = $(job_name).$(schedd_host).$(jobid).$(stepid).err
#@network.MPI	   = csss,shared,us
#@environment 	   = MP_LABEL_IO=yes; MP_INFOLEVEL=2; COPY_ALL
#@wall_clock_limit = 12:00:00,11:45:00
#@notification     = never
#@class            = regular
#@node             = 1
#@tasks_per_node   = 16
#@step_name        = step0
#@queue
#@dependency       =(step0 ==0)
#@step_name        = step1
#@queue
#@dependency       =(step1 ==0)
#@step_name        = step2
#@queue

export STEP=`echo $LOADL_STEP_NAME |sed -e 's/step//'`
export LAST_STEP=$(($STEP - 1))

echo "STEP$STEP START:: "`date` `perl -e 'print time();'`
case $STEP in
    0)
	sander -O -i mdin -p prmtop -c inpcrd -ref crd.ref \
	 -x mdcrd.$STEP -inf mdinfo.$STEP -r restrt.$STEP -o mdout.$STEP
	;;
    *)
	sander -O -i mdin -p prmtop -c restrt.$LSTEP -ref crd.ref \
	 -x mdcrd.$STEP -inf mdinfo.$STEP -r restrt.$STEP -o mdout.$STEP
	;;
esac
echo "STEP$STEP END:: "`date` `perl -e 'print time();'`
return 0

This script will execute the three steps sequentially producing output from each step in the files mdout.{0,1,2..} and mdcrd.{0,1,2..}. If you wish to run a longer simulation you may add more steps.