mo 169 could not be found !

[jbaltrus]

It can't find some orbitals although they are not supposed to be there. There is only 168 orbitals I generate with TmolexClient 3.0 but for some reason it's looking for 169-760 and then fails

JOnas

*---------------------------------------------------------------------*
|               simplified C1 algorithm will be applied               |
*---------------------------------------------------------------------*
  MOs are in ASCII format !


reading orbital data $scfmo  from file mos .

orbital characterization : expanded

  mo         169  could not be found !
  mo         170  could not be found !
.............................

[jbaltrus]

I take it back, I did not realize that I still have to run dscf before ricc2

Bigger problem I am getting is:

-dscf runs only on one processor (is there a way to run it in parallel)?
-ricc2 runs only on one processor! This is a problem! I would like some help with that

Jonas

hostname is         compute-6-158.local
this is node-proc. number 0 running on node compute-6-158.local                                                             
the total number of node-proc. spawned is   1

ricc2(compute-6-158.local) : TURBOMOLE V6.2 22 Apr 2010 at 17:18:47
Copyright (C) 2010 TURBOMOLE GmbH, Karlsruhe


    2010-12-18 21:21:16.541



                              R I C C 2 - PROGRAM

                          the quantum chemistry groups
                             at the universities in
                           Karlsruhe, Bochum & Mainz
                                   Germany



   *-----------------------------------------------------------------------*
   |                     program will use  1 thread(s)                     |
   *-----------------------------------------------------------------------*

My script is:
#!/bin/bash
#$ -S /bin/csh

# Use current working directory
#$ -cwd
#
# Join stdout and stderr
#$ -j y
# Preserve current environmental variables
#$ -V
# Assign NPROC
#$ -pe orte 16

setenv TURBODIR /opt/turbomole62/TURBOMOLE
setenv TURBOTMPDIR /scratch

##### Parallel job
# Set environment variables for a MPI job

setenv PARA_ARCH MPI
setenv PARNODES 16
limit stacksize unlimited

setenv NCPUS 16

dscf>dscf.out
ricc2>ricc2.out
#jobex -c 500 -energy 7 -gcart 4 -statpt -ri
#/Users/jbaltrus/scripts/aoforce_smp>aoforce.out
#tmole

echo "Job finished at: `date`"

[Arnim]

Hi Jonas,

your PATH points only to the serial version.
In the chapters 3.2.1 and 3.2.2 in the manual
some information on how to set the PATH can be set.
Here is a brief summary:

############ MPI version ############
export TURBODIR=/<somewhere>/TURBOMOLE
export PATH=$TURBODIR/scripts:$PATH

export PARA_ARCH=MPI
export PATH=$TURBODIR/bin/`sysname`:$PATH
export PARNODES=n

dscf > dscf.out
ricc2 > ricc2.out
############ MPI version ############

############ SMP version ############
export TURBODIR=/<somewhere>/TURBOMOLE
export PATH=$TURBODIR/scripts:$PATH

export PATH=$TURBODIR/bin/`sysname`:$PATH
export OMP_NUM_THREADS=n

dscf_smp > dscf.out
ricc2_smp > ricc2.out
############ SMP version ############

[jbaltrus]

Arnim,

I see no difference between your and mine scripts. I added to additional paths but it still starts on one processor:

#!/bin/bash
#$ -S /bin/csh

# Use current working directory
#$ -cwd
#
# Join stdout and stderr
#$ -j y
# Preserve current environmental variables
#$ -V
# Assign NPROC
#$ -pe orte 16

setenv TURBODIR /opt/turbomole62/TURBOMOLE
setenv PATH $TURBODIR/scripts:$PATH
setenv PATH $TURBODIR/bin/`sysname`:$PATH
setenv TURBOTMPDIR /scratch

##### Parallel job
# Set environment variables for a MPI job

setenv PARA_ARCH MPI
setenv PARNODES 16
limit stacksize unlimited

setenv NCPUS 16

dscf>dscf.out
ricc2>ricc2.out
#jobex -c 500 -energy 7 -gcart 4 -statpt -ri
#/Users/jbaltrus/scripts/aoforce_smp>aoforce.out
#tmole

echo "Job finished at: `date`"

[olehtone]

Hi,

setenv PARA_ARCH MPI

should probably be set before PATH definition.


Olli

[jbaltrus]

Arnim suggested the same. that just brings about whole lot of problems. COuld anybody help me single out step by step what's going on so I can;t run it?

Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
which: no prsh in (/opt/turbomole62/TURBOMOLE/bin/em64t-unknown-linux-gnu_mpi:/opt/turbomole62/TURBOMOLE/scripts:/opt/gridengine/bin/lx26-amd64:/usr/java/latest/bin:/opt/fips/bin:/tmp/16581.1.all.q:/opt/turbomole62/TURBOMOLE/bin/em64t-unknown-linux-gnu:/opt/turbomole62/TURBOMOLE/scripts:/opt/gridengine/bin/lx26-amd64:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/cce/10.0/bin:/opt/fsl/bin:/opt/fips:/opt/brains2/bin:/usr/local/bin:/bin:/usr/bin:/opt/freesurfer/bin:/opt/freesurfer/bin/Linux:/opt/freesurfer/fsfast/bin:/opt/freesurfer/fsfast/bin/Linux:/opt/freesurfer/afni/Linux:/opt/freesurfer/bin/noarch:/opt/freesurfer/local/bin/Linux:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/kent:/opt/ncbi/build:/opt/rocks/bin:/opt/rocks/sbin:/Users/jbaltrus/bin:./:/opt/crystal09/bin/Linux-ifort_11.1_openmpi_emt64/v1_0_1:/Users/jbaltrus:/opt/crystal09/crgra2006:/usr/X11R6/bin:/opt/bio/ncbi/bin:/opt/bio/mpiblast/bin/:/opt/bio/EMBOSS/bin:/opt/bio/clustalw/bin:/opt/bio/t_coffee/bin:/opt/bio/phylip/exe:/opt/bio/mrbayes:/opt/bio/fasta:/opt/bio/glimmer/bin:/opt/bio/glimmer/scripts:/opt/bio/gmap/bin:/opt/bio/gromacs/bin:/opt/bio/autodocksuite/bin:/opt/freesurfer/bin:/opt/freesurfer/bin/Linux:/opt/freesurfer/fsfast/bin:/opt/freesurfer/fsfast/bin/Linux:/opt/freesurfer/afni/Linux:/opt/freesurfer/bin/noarch:/opt/freesurfer/local/bin/Linux:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/rocks/bin:/opt/rocks/sbin)
dscf ended normally
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
dscf_mpi: Rank 0:3: MPI_Init: ibv_create_qp(left ring) failed
dscf_mpi: Rank 0:3: MPI_Init: probably you need to increase pinnable memory in /etc/security/limits.conf
dscf_mpi: Rank 0:3: MPI_Init: ibv_ring_createqp() failed
dscf_mpi: Rank 0:3: MPI_Init: Can't initialize RDMA device
dscf_mpi: Rank 0:3: MPI_Init: MPI BUG: Cannot initialize RDMA protocol
MPI Application rank 3 exited before MPI_Init() with status 1
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
mpirun: Broken pipe
which: no prsh in (/opt/turbomole62/TURBOMOLE/bin/em64t-unknown-linux-gnu_mpi:/opt/turbomole62/TURBOMOLE/scripts:/opt/gridengine/bin/lx26-amd64:/usr/java/latest/bin:/opt/fips/bin:/tmp/16581.1.all.q:/opt/turbomole62/TURBOMOLE/bin/em64t-unknown-linux-gnu:/opt/turbomole62/TURBOMOLE/scripts:/opt/gridengine/bin/lx26-amd64:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/cce/10.0/bin:/opt/fsl/bin:/opt/fips:/opt/brains2/bin:/usr/local/bin:/bin:/usr/bin:/opt/freesurfer/bin:/opt/freesurfer/bin/Linux:/opt/freesurfer/fsfast/bin:/opt/freesurfer/fsfast/bin/Linux:/opt/freesurfer/afni/Linux:/opt/freesurfer/bin/noarch:/opt/freesurfer/local/bin/Linux:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/kent:/opt/ncbi/build:/opt/rocks/bin:/opt/rocks/sbin:/Users/jbaltrus/bin:./:/opt/crystal09/bin/Linux-ifort_11.1_openmpi_emt64/v1_0_1:/Users/jbaltrus:/opt/crystal09/crgra2006:/usr/X11R6/bin:/opt/bio/ncbi/bin:/opt/bio/mpiblast/bin/:/opt/bio/EMBOSS/bin:/opt/bio/clustalw/bin:/opt/bio/t_coffee/bin:/opt/bio/phylip/exe:/opt/bio/mrbayes:/opt/bio/fasta:/opt/bio/glimmer/bin:/opt/bio/glimmer/scripts:/opt/bio/gmap/bin:/opt/bio/gromacs/bin:/opt/bio/autodocksuite/bin:/opt/freesurfer/bin:/opt/freesurfer/bin/Linux:/opt/freesurfer/fsfast/bin:/opt/freesurfer/fsfast/bin/Linux:/opt/freesurfer/afni/Linux:/opt/freesurfer/bin/noarch:/opt/freesurfer/local/bin/Linux:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/rocks/bin:/opt/rocks/sbin)
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
ricc2_mpi: Rank 0:1: MPI_Init: ibv_create_qp(left ring) failed
ricc2_mpi: Rank 0:1: MPI_Init: probably you need to increase pinnable memory in /etc/security/limits.conf
ricc2_mpi: Rank 0:1: MPI_Init: ibv_ring_createqp() failed
ricc2_mpi: Rank 0:1: MPI_Init: Can't initialize RDMA device
ricc2_mpi: Rank 0:1: MPI_Init: MPI BUG: Cannot initialize RDMA protocol
MPI Application rank 1 exited before MPI_Init() with status 1
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
mpirun: Broken pipe

[jbaltrus]

Actuially, simple dscf fails after the statistics run and I am getting the same errors. The trick here that ulimit -a is fine on the computational nodes so not sure what to do:

[jbaltrus@compute-4-111:~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 204800
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
which: no prsh in (/opt/turbomole62/TURBOMOLE/bin/em64t-unknown-linux-gnu_mpi:/opt/turbomole62/TURBOMOLE/scripts:/opt/gridengine/bin/lx26-amd64:/usr/java/latest/bin:/opt/fips/bin:/tmp/16648.1.UI:/opt/turbomole62/TURBOMOLE/bin/em64t-unknown-linux-gnu:/opt/turbomole62/TURBOMOLE/scripts:/opt/gridengine/bin/lx26-amd64:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/cce/10.0/bin:/opt/fsl/bin:/opt/fips:/opt/brains2/bin:/usr/local/bin:/bin:/usr/bin:/opt/freesurfer/bin:/opt/freesurfer/bin/Linux:/opt/freesurfer/fsfast/bin:/opt/freesurfer/fsfast/bin/Linux:/opt/freesurfer/afni/Linux:/opt/freesurfer/bin/noarch:/opt/freesurfer/local/bin/Linux:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/kent:/opt/ncbi/build:/opt/rocks/bin:/opt/rocks/sbin:/Users/jbaltrus/bin:./:/opt/crystal09/bin/Linux-ifort_11.1_openmpi_emt64/v1_0_1:/Users/jbaltrus:/opt/crystal09/crgra2006:/usr/X11R6/bin:/opt/bio/ncbi/bin:/opt/bio/mpiblast/bin/:/opt/bio/EMBOSS/bin:/opt/bio/clustalw/bin:/opt/bio/t_coffee/bin:/opt/bio/phylip/exe:/opt/bio/mrbayes:/opt/bio/fasta:/opt/bio/glimmer/bin:/opt/bio/glimmer/scripts:/opt/bio/gmap/bin:/opt/bio/gromacs/bin:/opt/bio/autodocksuite/bin:/opt/freesurfer/bin:/opt/freesurfer/bin/Linux:/opt/freesurfer/fsfast/bin:/opt/freesurfer/fsfast/bin/Linux:/opt/freesurfer/afni/Linux:/opt/freesurfer/bin/noarch:/opt/freesurfer/local/bin/Linux:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/rocks/bin:/opt/rocks/sbin)
dscf ended normally
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
    This will severely limit memory registrations.
dscf_mpi: Rank 0:2: MPI_Init: ibv_create_cq() failed
dscf_mpi: Rank 0:2: MPI_Init: Can't initialize RDMA device
dscf_mpi: Rank 0:2: MPI_Init: MPI BUG: Cannot initialize RDMA protocol
MPI Application rank 2 exited before MPI_Init() with status 1
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)
forrtl: error (78): process killed (SIGTERM)