NERSC New User's Guide

Welcome to the National Energy Research Scientific Computing Center. These documents and links are intended to help new users get started using the NERSC facilities and services.

If you have questions about NERSC or need assistance, please get in touch with us

The How to Get Help section will tell you how you can get help with any specific question or problem.

The About NERSC web pages have an overview of the center.

How to ... Get Help

NERSC has an online help desk where you can search FAQs and submit questions and problem reports. It is monitored by the NERSC consulting staff during business hours, M-F, 8-5 Pacific Time. See Getting Help at NERSC for more information.

The NERSC consulting, operations, and account support staff can assist you with your questions and requests. You will receive the fastest service if you directly contact the staff who can best respond to your inquiries. Consult the table below before contacting NERSC.

Our phone number is

Consulting Staff

The consulting staff are members of the NERSC User Services Group (USG). This group provides a wide range of services to NERSC clients. Visit the help web page for more information.

Web Pages

We hope you can find almost all the information you'll need through the web pages.

Here you'll find links to our Help pages, software documentation page, computer description pages and online tutorials page.

How to ... Get an Account

Before you can use NERSC facilities, you need a user account. Each user account belongs to a group account, also known as a repository. See the Accounts web pages for more information.

Group Accounts (repos)

Group accounts, known as repositories, are allocated by the Department of Energy to support projects sponsored by the DOE Office of Science. Each repository has one or more users, each with their own user account.

Each project is given a repository of computer time and archival storage, which is managed by the project's Principal Investigator, or designated Account Manager(s).

Principal Investigators apply for repositories through a process known as ERCAP, which stands for Energy Research Computer Applications Process. Proposals are submitted through a World Wide Web interface. See the NERSC allocations web page for more information.

Startup awards are available for new projects that wish to investigate using NERSC resources, or who wish to port or develop new codes. The maximum Startup award is 20,000 SP3-equivalent hours and 5,000 Storage Resource Units. A request for a Startup repository can be made at any time during the year, and decisions for Startup requests are made by NERSC within one to three weeks after applying. See Applying for Your First NERSC Allocation.

User Accounts (user names)

Each person using NERSC must have their own user account; sharing of accounts is not allowed. All new users must sign the Computer Use Policies form.

A user name may be associated with multiple repositories. Each user is allotted some share of a given repository's resources by the PI or a designated account manager. If a user belongs to more than one repo, he or she can choose which repo to charge a job. See MPP Accounts and Charging.

The PI or repo manager must request user accounts for members of their repository. The PI or manager makes the request online via the NIM web accounting interface. NIM is also used for adding and deleting users from repositories. PIs and Project Managers should read the NIM Guide for PIs, PI Proxies and Project Managers.

How to ... Get a Password

Please thoroughly read Password Policy and Procedures as a first step for all password matters.

Different NERSC resources use different password methods.

• For HPSS, you must change your password at the server auth.nersc.gov: see HPSS Passwords.
• For PDSF, NIM, the NERSC website, Bassi, Jacquard, and DaVinci, you must use your NERSC LDAP (NIM) password. See Getting Started with NIM and Change NIM Password.

Once you have been informed that your NERSC account is active, and you have returned a signed Computer Use Policies Form, you can obtain a password for your account from the NERSC Account Support Group at 1-800-66-NERSC, menu option 2 (or 510-486-8612). If you forget your password, a new one may be obtained by calling the same number.

Logging In the First Time

Your initial computer password is only good for one login on each NERSC machine. This first login prompts you to change your password and then logs you off. You will be prompted to type the initial password and then to select a new password, which you will have to enter twice. Your new password must contain at least 8 characters, and Department of Energy Password Requirements are quite stringent. In addition to these requirements, your new password must differ from the initial password by at least 3 characters. As soon as you are logged off, you can login again with your new password, except on the NERSC IBM SP, where it will take up to one hour for password changes to be propagated to all the nodes.

Changing Your Password

Use the NIM web interface to change your password for Bassi, Jacquard, PDSF, DaVinci, the NERSC website, and the NERSC CVS server. and HPSS passwords are changed vis SSH to auth.nersc.gov. See HPSS Passwords.

How to ... Select a Computer

NERSC has IBM POWER3 and POWER5 systems and a Linux cluster that provide High Performance Computing (HPC) platforms, and an analytics server that can be used to support calculations performed on the HPC systems. NERSC also hosts the Parallel Distributed Systems Facility (PDSF).

See the NERSC Systems page for more information.

Massively Parallel Processor (MPP) Systems

Franklin - Cray XT4

The NERSC Cray XT4 named franklin (franklin.nersc.gov), is a distributed memory machine with 9,669 compute nodes, each containing 2 CPU cores. Each processor is a 2.6 GHz Opteron. Nodes have from 4 GB of memory.

Bassi - IBM p575 POWER 5

The NERSC IBM POWER 5 system named bassi.nersc.gov, is a distributed memory machine with 111 compute nodes, each containing 8 processors. Each processor has a peak performance of 7.6 Gflops. Nodes have 32 GB of memory.

Jacquard - Opteron Linux Cluster

Jacquard is an Opteron cluster with 320 dual-processor nodes available to run scientific applications. The nodes are connected by a high-speed InfiniBand network and all have access to a large global GPFS file system.

Parallel Distributed Systems Facility

PDSF - Linux Cluster

The PDSF is a networked distributed computing environment used to meet the detector simulation and data analysis requirements of large scale High Energy Physics (HEP) and Nuclear Science (NS) investigations.

Analytics Server

DaVinci - SGI Altix

The NERSC Analytics Server, DaVinci, allows users to perform visualizations from remote locations. The machine is a Silicon Graphics Altix 350 with 32 1.4 GHz Itanium-2 processors, 192 GB of shared memory, and approximately 24 TB of disk space.

Mass Storage System

The NERSC mass storage system is the High Performance Storage System (HPSS).

Global File System

How to ... Connect and Log In

Once you have a login and its associated password you can begin using a NERSC machine. You must have an active user account on each machine you wish to use.

In order to use NERSC, you must first have a local computer connected to the internet. Software that employs the ssh protocol must be used to connect to NERSC computers for interactive sessions. Data can be transferred from one machine to another with the scp or sftp commands. See Connecting to NERSC for further information.

If you have problems connecting, contact us to Get Help.

How to ... Work with the OS (Operating System)

Once you're logged in to a NERSC machine, you'll be presented with a UNIX shell prompt and you'll normally be in your home ($HOME) directory. If you are new to UNIX, you'll have to learn the basics before you can do anything useful at NERSC. Some helpful web pages are listed on our UNIX training resources page.

Shells and Initialization Files

You have a choice of login shells, including csh, tcsh, bash and ksh. Please see Shells and Scripting for more information on these shells. You change shells through the NIM web interface. See Change Default Login Shell.

In your home directory, you will find default shell initialization files: .login, .cshrc, .profile, .kshenv. These are symbolic links to system defined files. You cannot modify them and you should not overwrite them. Make all customizations in auxiliary files named .login.ext, .cshrc.ext, etc.. These files are sourced from the .login, .cshrc, etc. files.

A warning: Do not copy initialization files from other machines to NERSC machines, or from one NERSC machine to another. Such copied files are unlikely to work properly.

Modules

It used to be that every time you wanted to use a new software package or library, you had to edit your shell initialization files. Those days are almost gone thanks to an extension to the UNIX system known as modules. See Modules Approach to Environment Management.

How to ... Compile a Program

The main computer languages used at NERSC are Fortran 90, Standard C, and C++. Almost all parallel programming at NERSC uses MPI and/or OpenMP.

No changes to the default environment are needed in order to use compilers. Their behavior is similar to what you would expect on any UNIX machine. For example, this command line will use the C compiler to create an executable program from the source code file named hello.c.

% cc hello.c

By default, the executable is named a.out.

Each platform has its own specific compilers. See the web page specific to your machine of interest for more information.

Linking with software libraries is simple in the modules environment; in general, you may use predefined environment variables in the compile line (common libraries will be linked by default).

How to ... Store Files

NERSC provides a number of types of file storage space for users:

• A $HOME directory for permanent file storage for each user on each platform.
• A $SCRATCH dirctory for temporary file storage for each user on each platform.
• A "project" directory in the NERSC Global Filesytem, which is mounted from all platforms. Project directories are for permanently storing files to be shared by a group. These directories are available upon request.
• an HPSS mass storage system for long-term file storage of large datasets, available via HPSS softare utiltities from the compute platforms.

Disk Directories

When you log into a NERSC computer interactively, you are in your home directory by default ($HOME). Directories consist of disk space where you can keep files that you want to be able to retain permanently (however, data stored in your home directory is backed up for disaster recovery purposes only; users should back up their own files as needed). Quotas are in effect that limit the amount of data that can be saved there. The myquota command will show you your usage and limits.

There is also a large, temporary disk space that you can use to store files while you run programs from batch scripts. referenced by $SCRATCH.

Never explicitly use /tmp on any NERSC system; this filesystem is very small and reserved for system use.

Archival Storage

You may also store your files on the NERSC HPSS mass storage system. The mass storage (or archival storage) facilities are used to permanently save large amounts of data that exceed the quota on your home directory. It will take from seconds to hours to retrieve data from the mass storage systems, depending on a number of factors, including the file size and the amount of time it has been dormant in storage.

The primary user interface to transfer files to and from HPSS is known as hsi.

See HPSS for more information on how to use the archival storage facilities.

How to ... Run a Program

You may either run a program interactively from a shell window or by submitting a script to the batch system. For instructions on running on a given system, see the web page for that system.

Interactive Processing

Interactive jobs should mainly be used for program development and testing. Each machine has relatively strict limits on interactive memory usage and CPU time.

To run a serial program or command interactively, type the name of the program or command at the machine prompt (%). If the location of the program is in your command PATH, you can just type the filename, otherwise type the complete pathname. Since the current working directory (.) is not in the search path by default, the correct way to execute the program named my_program in the current directory is

 % ./my_program

Batch Processing

Most production runs should be executed as jobs under control of the batch system. Batch (non-interactive) jobs are run by submitting a text file (script) that contains a list of commands. The script must also specify various parameters such amount of CPU needs, memory requirements, etc.

There are different batch systems at NERSC. You should the "Running Jobs" page on the system you intend to use. See NERSC Systems.

How to ... Run Applications

A number of software applications and tools have been installed on NERSC computers. Most are freely available to all NERSC users.

You can find out about applications from the software applications web page.

A Tools page contains information about programming and optimization tools as well as UNIX and GNU utilities.

How to ... Determine Account Charges

Computational and storage resources are allocated to repositories by DOE. Repositories are charged for computer usage by their members on the NERSC MPP machines and the analytics server DaVinci. The basic unit of charge is the MPP hour, equivalent to one hour of computing on one IBM POWER 3+ processor. Repositories are also charged for HPSS usage by their members.

Parallel jobs are charged for all processors on each node it uses.

A higher charging rate is available for those who need a faster turnaround time (but should be used with caution becaus this can deplete your project's allocation rapidly!) and a lower rate is available for those who can wait longer.

See the Usage Charging web page for more information.

Monitoring Usage

NERSC users monitor their usage using the NERSC Information Management system (NIM).

How to ... Find Documentation

The primary source of information about NERSC is our Web site, located at the URL http://www.nersc.gov.

Access to some web pages is restricted to NERSC users. You can access these pages using your NERSC LDAP (NIM) login and password.

Resources on the NERSC web

Most information on the NERSC website is partitioned by compute platform. Start at the main page for the computers you are interested in.

Information on compilers, libraries, applications, operating systems, and all other software can be found under the Software heading.

Descriptions of compute platforms, storage devices, and their capabilities and configurations are available under Systems.

Announcements and up-to-date information about system status are posted in the Status area.

Our online tutorials contain lots of information, documentation, tricks, tips and "how-to"s.

Online IBM Vendor Documentation

IBM's online documentation is not well presented for the High Performance Computing community. NERSC has collected some relevant IBM documentation for the benefit of our user community.

Online man pages

Each computer has a set of man pages that document every command. To read about command, type

%  man command

man even has its own man page, type

%  man man

How to ... Manage email

When you get a NERSC user account, an email alias is created for you of the form

 username@nersc.gov

where username is your login name. This email address is not directed to a fixed machine - there is no computer named nersc.gov . Instead, username@nersc.gov is an email alias that forwards mail to your home email address as specified in your application for computer time.

The NIM accounting interface will display your delivery address under the Contact Info tab (select My Contact Info from the My Stuff pull-down in the NIM main menu). Please keep your contact information in NIM up-to-date.

E-mail on the compute platforms

Reading email on the compute platforms is not permitted at NERSC. Email sent to a machine, for example username@franklin.nersc.gov, is automatically forwarded to the alias username@nersc.gov, and from there on to your home email address.

NERSC E-mail lists

NERSC maintains a number of user email lists to which you may subscribe. See NERSC User Email Lists for more information.

You may wish to join the NERSC Users Group (NUG) mailing list. NUG advises NERSC about its present services and future development. To join the NUG mailing list, send email to majordomo@nersc.gov with the following in the body (not the subject line) of your message:

   subscribe nug

How to ... Keep Up to Date

You'll need to keep abreast of new software installations, configuration changes, etc. Major news announcements are posted on the HPC Users web page. Also check the Status and Statistics page where you will find an archive of all Announcements.

Each time you log into a NERSC machine, you are presented with the Message of the Day (motd). Be sure to scan those messages for important announcements. If you miss the motd at login time, you can read it any time with the command cat /etc/motd.

NERSC Users Group

The NERSC Users Group (NUG) holds monthly teleconferences to discuss NERSC services and future developments. Join the NUG mailing list for information about the schedule and agenda.

Page last modified: Tue, 18 May 2004 21:48:05 GMT Page URL: http://www.nersc.gov/nusers/help/new_user/new_user.php Web contact: webmaster@nersc.gov Computing questions: consult@nersc.gov Privacy and Security Notice