parallel NumForce calculation using RI-MP2 in the CC2 module

[lyzhu]

Hello,

I have got the optimzied structure using parallel RI-MP2 in the CC2 nodule. Now I want to calculate the freqency. I just added the keyword in the control file:
$vibrational normal modes  file=gsmodes.out
$vibrational spectrum  file=gsvibspectrum.out
$hessian  file=gshessian.out
$dipole

and then run "NumForce -level cc2 -ri -central >gsf.out" calculation.
My script for submitted job like this:

****************************
#!/bin/bash
#BSUB -J F4
#BSUB -n 8
#BSUB -W 999:00
#BSUB -R "rusage[Memory=1000]"

export TURBODIR=/export/apps/TURBOMOLE60

export PARA_ARCH=MPI

source ${TURBODIR}/Config_turbo_env

nprocs=0
for host in ${LSB_HOSTS}; do
   ((nprocs= ${nprocs} + 1))
   echo ${host} >> ${LSB_JOBID}_hosts
done

export PARNODES=${nprocs}
export HOSTS_FILE=${LSB_JOBID}_hosts

echo PARNODES=${PARNODES}
cat ${LSB_JOBID}_hosts

**********************************

However, the job didn't do the parallel calculation, just used one node.  What's the problem?  Any comments or suggestions are welcome!

regards!

Ling

[Arnim]

Hello,

it would be totally inefficient to run NumForce with parallel binaries.
Since NumForce starts independent gradient calculations for the displacements,
a so-called "trivial parallelization" can be used with the option NumForce -mfile.

Cheers,
Arnim

[lyzhu]

Hi, Arnim

Thanks for your reply.

Now I am trying to do " NumForce trivial parallelization " with NumForce -mfile. I removed the "export PARA_ARCH=MPI" part in my parallel binary. The submitted script is as following:

********************************
#!/bin/bash
#BSUB -J 4T-F4-1-mp2
#BSUB -n 8
#BSUB -W 999:00
#BSUB -R "rusage[Memory=400] "

export TURBODIR=/export/apps/TURBOMOLE60

source ${TURBODIR}/Config_turbo_env

nprocs=0
for host in ${LSB_HOSTS}; do
   ((nprocs= ${nprocs} + 1))
   echo ${host} >> ${LSB_JOBID}_hosts
done

export PARNODES=${nprocs}
export HOSTS_FILE=${LSB_JOBID}_hosts

echo PARNODES=${PARNODES}
cat ${LSB_JOBID}_hosts

*************************************************************
Then I submitted the job with "NumForce -ri -level cc2 -central -mfile ${LSB_JOBID}_hosts" >gsf.out

However, there are 8 error files in the KraftWerk subdirectory.

In the *.err file, the included informaion is:

****************************************
OpenSSH_3.9p1, OpenSSL 0.9.7a Feb 19 2003
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: /etc/ssh/ssh_config line 7: Deprecated option "FallBackToRsh"
debug1: Connecting to compute-1-25 [10.255.255.213] port 22.
debug1: Connection established.
debug1: identity file /home/lyzhu/.ssh/id_rsa type 1
debug1: identity file /home/lyzhu/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_3.9p1
debug1: match: OpenSSH_3.9p1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.9p1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'compute-1-25' is known and matches the RSA host key.
debug1: Found key in /home/lyzhu/.ssh/known_hosts:32
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: publickey
debug1: Offering public key: /home/lyzhu/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 149
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending command: cd /home/lyzhu/PROG/TURBOMOLE/DFT-D/4T-F4TCNQ-1-rimp2-aug-ccpvdz/rimp2-freq/numforce/KraftWerk ;
               export PATH=$PATH:. ;                          . ENVIRONMENT.12276 ;                           /export/apps/TURBOMOLE
60/scripts/runsingle xm01 compute-1-25:1
Bad : modifier in $ (.).
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
debug1: fd 2 clearing O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.1 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 1
****************************************

Any suggestions are welcome!

Thanks in advance!

[mpjohans]

Hello!

The "bad modifier in $" error comes from incorrect parsing of the "export PATH=$PATH:." command. Use "export PATH=${PATH}:." instead (the shell is stupid and doesn't know when the variable name ends otherwise).

But I think that's a (t)csh specific error message, while NumForce assumes (ba)sh. It will crash even if you fix the above, unless you transform the ssh command line to not assume any special shell. Some years ago I made the following fix which still seems to work. Edit the lines in NumForce that sends out the jobs to the nodes to something like this:

Code: [Select]

               $remoteshell $loginname "/bin/bash -c '\
                 cd $WORKDIR ; . ./$ENVFILE ; export PATH=\${PATH}:. ;\
                 $SCRIPTS/runsingle $dir $machine $scrpath ' "\
               2> $dir.$machine.err &

Have a nice day,
    Mikael J.
    http://www.iki.fi/~mpjohans