From: David Turner (dpturner_at_lbl_dot_gov)
Date: 03/01/2005
Greetings Seaborg User,
NERSC recently made some adjustments to the LoadLeveler
scheduling algorithm. This note provides some background
information on how LoadLeveler schedules jobs, and how
NERSC is tuning LoadLeveler in order to provide our users
with equitable access to computational resources while meeting
DOE objectives.
When a job is submitted to LoadLeveler, is passes through
several different states before it begins execution. When it
enters the "idle" state (status I in llqs), it is assigned a
priority (SYSPRIO). This value is only computed once, and is
then used to order the eventual execution.
The current SYSPRIO is calculated thusly:
SYSPRIO =
(ClassPrio * 86,400) + (UserPrio * 86,400) + (GroupPrio * 86,400) - (QDate)
In regular use, UserPrio and GroupPrio are both zero (they
can be made non-zero to boost an individual's or a group's
priority). QDate is basically a clock, measured in seconds.
For jobs within a particular class, newer jobs will have a
larger value of QDate than older jobs, and thus a lower priority.
The term 86,400 (the number of seconds in one day) was chosen to
allow control of the scheduling in units of one day.
In the absence of user or group "boosting", the priority is
dominated by the class priority, as shown in the following table:
Class ClassPrio Contribution to SYSPRIO
Interactive 246 21,254,400
Debug 216 18,662,400
Premium 186 16,070,400
32+ Nodes 184 15,897,600
1-31 Nodes 180 15,552,000
Low 0 0
It is possible to define a "scheduling distance" between two jobs
as the number of days the the job with a lower scheduling priority
would have to be queued in order to catch up to a job with higher
scheduling priority (each day providing 86,400 seconds via QDate).
The following table shows the inherent scheduling distance between
adjacent batch classes in this new priority scheme:
Lower class Higher class SYSPRIO diff Scheduling distance
Debug Interactive 2,592,000 30 days
Premium Debug 2,592,000 30 days
32+ Nodes Premium 172,800 2 days
1-31 Nodes 32+ Nodes 345,600 4 days
Low 1-31 Nodes 15,552,000 180 days
The values for the class priorities were chosen to yield the above
scheduling distances, based on the following prioritized list of
NERSC objectives:
1. Provide good turnaround to interactive/debug jobs.
2. Ensure that the OMB metric is met (this states that 40% of the
computational resources be delivered to jobs using one-eighth
or more of the processors)
3. Ensure that specific DOE-selected projects can use their
allocations and receive good turnaround.
4. Favor large node jobs over small node jobs.
5. Favor premium jobs over regular jobs.
6. Favor regular jobs over low jobs.
LoadLeveler job scheduling will be discussed (along with other
topics) during the NERSC User Group (NUG) teleconference this
Thursday, March 3, at 9:00 AM Pacific Time. Users are encouraged
to attend this meeting and share their thoughts. To attend:
Local Call: 510-486-5008
Call toll free: 1-877-252-5250
Internal Users: 5008
Then press 1 and enter 7193# and follow the instructions.
Meeting ID: 7193.
--
Best regards,
David Turner
User Services Group email: dpturner_at_lbl_dot_gov
NERSC Division phone: (510) 486-4027
Lawrence Berkeley Lab fax: (510) 486-4316