Tutorials
Tutorials HomeOpenMP
IndexIntroduction
PARALLEL
DO
PARALLEL DO
SECTIONS
SINGLE
MASTER
CRITICAL
BARRIER
FLUSH
ATOMIC
ORDERED
FullDocument
Related Info
OpenMP Home PageATOMIC Directive
!$OMP ATOMIC
The ATOMIC directive ensures that a specific memory location is to be updated atomically, rather than exposing it to the possibility of multiple, simultaneous writing threads.
Example
!Filename: density.f
!
PROGRAM DENSITY
IMPLICIT NONE
INTEGER, PARAMETER:: NBINS=10
INTEGER, PARAMETER:: NPARTICLES=100000
REAL:: XMIN, XMAX, MAXMASS, MINMASS
REAL, DIMENSION(NPARTICLES):: X_LOCATION, PARTICLE_MASS
INTEGER, DIMENSION(NPARTICLES):: BIN
REAL, DIMENSION(NBINS):: GRID_MASS, GRID_DENSITY
INTEGER, DIMENSION(NBINS):: GRID_N
REAL:: DX,DXINV,TOTAL_MASS,CHECK_MASS
INTEGER:: I, CHECK_N, XMAX_LOC(1)
GRID_MASS=0.0
TOTAL_MASS=0.0
GRID_N=0
CHECK_MASS=0.0
CHECK_N=0
! Initialize particle positions and masses
CALL RANDOM_NUMBER(PARTICLE_MASS)
CALL RANDOM_NUMBER(X_LOCATION)
MAXMASS = MAXVAL(PARTICLE_MASS)
MINMASS = MINVAL(PARTICLE_MASS)
XMAX = MAXVAL(X_LOCATION)
XMIN = MINVAL(X_LOCATION)
PRINT *, 'MINMASS =',MINMASS,' MAXMASS = ',MAXMASS
PRINT *, 'XMIN =',XMIN,' XMAX = ',XMAX
! Grid Spacing (and inverse)
DX = (XMAX-XMIN) / FLOAT(NBINS)
DXINV = 1/DX
!$OMP PARALLEL DEFAULT(SHARED) PRIVATE(I) REDUCTION(+:TOTAL_MASS)
!$OMP DO
DO I = 1, NPARTICLES
IF (I==XMAX_LOC(1)) THEN
BIN(I) = NBINS
ELSE
BIN(I) = 1 + ( (X_LOCATION(I)-XMIN) * DXINV )
END IF
IF(BIN(I) < 1 .OR. BIN(I) > NBINS) THEN
! Off Grid!
PRINT *, 'ERROR: BIN =',BIN(I),' X =',X_LOCATION(I)
ELSE
!$OMP ATOMIC
GRID_MASS(BIN(I)) = GRID_MASS(BIN(I)) &
+ PARTICLE_MASS(I)
!$OMP ATOMIC
GRID_N(BIN(I)) = GRID_N(BIN(I)) + 1
TOTAL_MASS = TOTAL_MASS + PARTICLE_MASS(I)
END IF
END DO
!$OMP END DO
!$OMP END PARALLEL
DO I=1, NBINS
GRID_DENSITY(I) = GRID_MASS(I) * DXINV
END DO
PRINT *, 'Total Particles =',NPARTICLES
PRINT *, 'Total Mass =',TOTAL_MASS
DO I=1,NBINS
PRINT *, 'DENSITY(',I,' ) =',GRID_DENSITY(I),' &
&MASS(',I,' ) =',GRID_MASS(I)
END DO
! Check for consistency
DO I=1,NBINS
CHECK_MASS = CHECK_MASS + GRID_MASS(I)
CHECK_N = CHECK_N + GRID_N(I)
END DO
PRINT *, 'Particles on Grid =', CHECK_N
PRINT *, 'Total Mass on Grid =', CHECK_MASS
END PROGRAM DENSITY
Without the ATOMIC directives, difference threads would try to update the grid mass bins at the same time, causing erroneous results. Compiling and running on franklin:
> cat density.pbs
#PBS -N density
#PBS -j oe
#PBS -o density.out
#PBS -q interactive
#PBS -S /bin/bash
#PBS -l mppwidth=1
#PBS -l mppnppn=1
#PBS -l mppdepth=2
#PBS -l walltime=00:05:00
#PBS -V
cd $PBS_O_WORKDIR
ftn -o density -mp=nonuma -Minfo=mp density.f90
export OMP_NUM_THREADS=2
aprun -n 1 -N 1 ./density
> qsub density.pbs
500899.nid00003
> cat density.out
/opt/xt-pe/2.0.44a2/bin/snos64/ftn: INFO: linux target is being used
density.f90:
density:
40, Parallel region activated
43, Parallel loop activated; static block iteration allocation
55, Begin critical section
End critical section
57, Begin critical section
End critical section
62, Barrier
64, Begin critical section
End critical section
Parallel region terminated
MINMASS = 4.8334509E-06 MAXMASS = 0.9999951
XMIN = 4.3212090E-06 XMAX = 0.9999959
Total Particles = 100000
Total Mass = 50035.98
DENSITY( 1 ) = 50715.23 MASS( 1 ) = 5071.481
DENSITY( 2 ) = 50151.22 MASS( 2 ) = 5015.081
DENSITY( 3 ) = 50154.58 MASS( 3 ) = 5015.417
DENSITY( 4 ) = 49183.93 MASS( 4 ) = 4918.352
DENSITY( 5 ) = 49083.88 MASS( 5 ) = 4908.347
DENSITY( 6 ) = 50813.93 MASS( 6 ) = 5081.351
DENSITY( 7 ) = 51367.36 MASS( 7 ) = 5136.694
DENSITY( 8 ) = 49535.90 MASS( 8 ) = 4953.549
DENSITY( 9 ) = 50296.39 MASS( 9 ) = 5029.598
DENSITY( 10 ) = 49062.40 MASS( 10 ) = 4906.199
Particles on Grid = 100000
Total Mass on Grid = 50036.07
Application 4736271 resources: utime 0, stime 0
Running a second time reveals that the additions of numbers ranging over 5 orders of magnitude (MINMASS to MAXMASS) are not entirely associative; note the values of MASS and Total Mass. You will not get precisely the same results each time as you would if had used a single thread.
MINMASS = 4.8334509E-06 MAXMASS = 0.9999951 XMIN = 4.3212090E-06 XMAX = 0.9999959 Total Particles = 100000 Total Mass = 50035.98 DENSITY( 1 ) = 50715.10 MASS( 1 ) = 5071.468 DENSITY( 2 ) = 50151.42 MASS( 2 ) = 5015.101 DENSITY( 3 ) = 50154.55 MASS( 3 ) = 5015.414 DENSITY( 4 ) = 49184.07 MASS( 4 ) = 4918.366 DENSITY( 5 ) = 49083.94 MASS( 5 ) = 4908.353 DENSITY( 6 ) = 50813.96 MASS( 6 ) = 5081.354 DENSITY( 7 ) = 51367.18 MASS( 7 ) = 5136.676 DENSITY( 8 ) = 49535.82 MASS( 8 ) = 4953.542 DENSITY( 9 ) = 50296.48 MASS( 9 ) = 5029.606 DENSITY( 10 ) = 49062.43 MASS( 10 ) = 4906.203 Particles on Grid = 100000 Total Mass on Grid = 50036.08
As an illustration of how things would go wrong, here's sample output with the ATOMIC directive removed from the code. Note the particle conservation check fails:
MINMASS = 4.8334509E-06 MAXMASS = 0.9999951 XMIN = 4.3212090E-06 XMAX = 0.9999959 Total Particles = 100000 Total Mass = 50035.98 DENSITY( 1 ) = 45400.19 MASS( 1 ) = 4539.981 DENSITY( 2 ) = 44862.25 MASS( 2 ) = 4486.188 DENSITY( 3 ) = 45124.16 MASS( 3 ) = 4512.378 DENSITY( 4 ) = 44338.48 MASS( 4 ) = 4433.811 DENSITY( 5 ) = 44486.02 MASS( 5 ) = 4448.565 DENSITY( 6 ) = 45709.66 MASS( 6 ) = 4570.928 DENSITY( 7 ) = 46367.48 MASS( 7 ) = 4636.710 DENSITY( 8 ) = 44487.38 MASS( 8 ) = 4448.701 DENSITY( 9 ) = 45177.90 MASS( 9 ) = 4517.752 DENSITY( 10 ) = 44100.41 MASS( 10 ) = 4410.005 Particles on Grid = 90036 Total Mass on Grid = 45005.02 Application 4736632 resources: utime 0, stime 0
