Author Topic: SMP aoforce single core only  (Read 7838 times)

matteo.tiberti

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
SMP aoforce single core only
« on: August 03, 2011, 12:17:41 PM »
Hi all,
I'm trying to run a parallel aoforce calculation but it runs single core only and I can't understand why.

Here's my .bashrc:

Code: [Select]
export LD_LIBRARY_PATH=/usr/local/TURBOMOLE/libso/x86_64-unknown-linux-gnu_ga
export TURBOARCH=x86_64-unknown-linux-gnu
export TURBOMOLE_SYSNAME=x86_64-unknown-linux-gnu
export PARA_ARCH=SMP
export TURBODIR=/usr/local/TURBOMOLE/
export PARNODES=12
PATH=$PATH:$TURBODIR/scripts
PATH=$PATH:$TURBODIR/bin/`sysname`


Code: [Select]
teo@machine:~$ sysname
x86_64-unknown-linux-gnu_smp
teo@machine:~$ sysname -s
x86_64-unknown-linux-gnu
teo@machine:~$ which aoforce
/usr/local/TURBOMOLE/bin/x86_64-unknown-linux-gnu_smp/aoforce

This is the aoforce script I have in the path:

Code: [Select]
#!/bin/sh
# pared down and modified from mpirun_scripts
if [ -z "${PARA_ARCH}" ]; then
   export PARA_ARCH=MPI
   echo "A PARALLEL `basename $0` calculation will be started!"
   echo "A PARALLEL `basename $0` calculation will be started!" >&2
fi

DEBUG=""
# process the options
while getopts l:n:g: MPIOPT; do
   case $MPIOPT in
      "l") LOADOPT="$OPTARG" ;;
      "n") NPOPT="$OPTARG"
          # check if NPOPT is a number
         if [ -n "`echo "$NPOPT" | grep "[^0-9]"`" ]; then
            echo "The argument to option -n has to be a NUMBER!"
            exit 1
         fi
         ;;
      "g") DEBUG="$OPTARG" ;;
   esac
done

# Now to the default-number of NODES:
# if the number is given as an option to this script ("script" -n 8),
# KOTEN is $NPOPT, if $NPOPT is not set but $PARNODES,
# take this as default, else set KNOTEN to 2
# If OMP_NUM_THREADS or SMPCPUS are set, they will also be used.
# The hierarchy is:
#                  NPOPT > PARNODES > OMP_NUM_THREADS > SMPCPUS
KNOTEN=${SMPCPUS:-2}
KNOTEN=${OMP_NUM_THREADS:-$KNOTEN}
KNOTEN=${PARNODES:-$KNOTEN}
KNOTEN=${NPOPT:-$KNOTEN}

unset OMP_DYNAMIC
unset OMP_NUM_THREADS
export SMPCPUS=$KNOTEN
$TURBODIR/bin/`sysname -s`/aoforce

I do not understand why it always runs the aoforce executable from the serial version ("sysname -s").
I also tried setting SMPCPUS, KNOTEN, NPOPT OMP_NUM_THREADS before launching the script AND also by setting them inside the script with no change.
Launching "aoforce -n 12" doesn't work either. All I get is single-core aoforce.

Thank you for your help

uwe

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 558
  • Karma: +0/-0
Re: SMP aoforce single core only
« Reply #1 on: August 03, 2011, 02:38:56 PM »
Hi Matteo,

the parallelization of aoforce is fully included in the serial binary. So serial and SMP versions are identical, just in parallel runs the aoforce script in $TURBODIR/bin/[...]_smp sets the number of CPUs and then starts the binary from the serial directory. Same is true for escf and egrad.

The implementation is based on fork() and Unix sockets (see http://onlinelibrary.wiley.com/doi/10.1002/jcc.21692/abstract), so during the aoforce run you will see just one aoforce process during the parts of the code that run sequentially, and multiple aoforce processes during those parts which are parallelized. This is different to the MPI or GA versions of the other modules, where the different processes are started right from the start, and also different to OpenMP parallel versions of dscf and ricc2 which are called dscf_smp and ricc2_smp.

There are three possible reasons why you do not see 12 running aoforce processes:

  • your example is so small, that the parallel parts are more or less finished as soon as they are started
  • your example is so big, that the serial part of the parallelized code at the beginning of the job takes several minutes before the 'real' work starts in parallel
  • there is something wrong with your Turbomole version,  in that case just contact the Turbomole Support by email: turbomole@cosmologic.de and you will for sure get help!

Regards,

Uwe