structure optimisation with newish g-xtb executable

[acd81]

Dear all,

I have tried using the newish g-xtb executable (that with analytical gradients) in structure optimisation through jobex , but to no avail. Replacing the original xtb by the newish g-xtb executable is somewhat straightforward and also the implicitly included xtb wrapper also seems to work alright in principle. The original wrapper is capable of producing appropriate energy/gradient files (in Turbomole style) from the energy/engrad data provided by g-xtb (xtb coord –gxtb –grad). However, the original wrapper produces a (Turbomole style) gradient file with a single "cycle = 0" entry, that is not amended by additional data as the optimisation via jobex proceeds. Whilst the need for some adaptations to the original xtb wrapper for the newish g-xtb executable comes at no particular surprise, one is unable to solve the problem by oneself, as there is no access to the xtb wrapper implemented in the Turbomole suite (version 7.8 is currently used).

Could I cordially inquire if there is any advice and/or a practical solution to this seemingly minor problem – perhaps an improved xtb wrapper that could be shared with us?

Please refrain from advertising the use of the original xtb environment and/or ORCA for structure optimisations with g-xtb.

Thanks a lot in advance,
Paul

[uwe]

Hello,

using jobex -level xtb does not call the original xtb binary from github, but the Turbomole-own xtb version that was implemented by the Grimme group some (long) time ago. So there is no wrapper around xtb. 
The Turbomole-own binary implementation is called tb while the xtb binary that is part of usual Turbomole distributions is a newer one built from the xtb github sources - but this version is not used by jobex.

I have not tried the new g-xtb executable yet, but if it writes out energy and gradients, it could be used by jobex with some minor modifications.

Did someone else use g-xtb with jobex already? If not, I'll have a look at it.

Do you run Turbomole under Linux or MacOS? Or from Windows? If I post a patch for the jobex shell script, it would run under sh/bash only. Alternatively one could replace the tb binary by a script that calls g-xtb instead - that would work under Windows too.

Best Regards, Uwe

[acd81]

Thanks Uwe, for your response.

First, let me apologise for the rather misleading content of the original post. The newish g-xtb executable has, like all the previous xtb executables, the necessary wrapper already included, such that energy and gradient values of g-xtb (via xtb coord –gxtb --grad) are already provided in Turbomole style energy and gradient files.

I have played a bit around with the original jobex script (in a UNIX environment) und made some obvious adaptations to get the newish g-xtb executable running in the context of Turbomole. The adaptations to be made in the script referenced as xtbprog in jobex are also pretty straightforward. However, a minor issue arises from the fact that previous xtb executables wrote "cycle =      1 ..." in the gradient file, whereas the g-xtb executable now writes "cycle =      0 ...", which lets statpt run into some difficulties. Just a minor hiccup, which a suitable "sed " shell command can't solve. Despite several attempts I could not get the modified jobex to run properly. On the one hand, the older xtb implementation seems to use different executables, one set for generating energy and gradient values in Turbomole style, whilst a second set seems to deal with parsing the control file for addt options and to ensure that energy and gradient files include all sequential structures. The other reason is rooted in my obvious inabilities.

So, I resorted to creating a simple, quick-and-dirty jobex-simulating shell script myself. That's not a difficult task, despite obvious inabilities in this department, with the advent of AI. Whoever your preferred AI bot is, he is your hero by making this task rather straightforward. Based on this quick-and-dirty solution, the newish g-xtb executable can well be used together with statpt to perform structure optimisation in Turbomole style.

I suppose the general audience would be more than happy, if a more general solution (f.i. parsing the control file for addt options) could be provided for a UNIX environment.
Many thanks in advance, Uwe, for all your efforts.

[uwe]

Hi,

glad to read that you (and whoever your virtual friend was) solved the issue. The cycle = 0 output of g-xtb is strange as the 'old' xtb, which is available as source code, prints out a hard-coded 1. Nevertheless, for a reasonable geometry optimization it is necessary to keep all gradients of former steps for GDIIS, so simply overwriting the gradient file is not useful.

But as g-xtb might be interesting for a larger number of people, I think I will post a modified jobex script that takes care of those issues somewhere here to the users forum. I have introduced a new level (named it gxtb) and added the possibility to append xtb command line options to the jobex call, such that jobex can be used together with most of the available xtb options. I'm not really sure if that always makes sense, but in the past I've simply been so often completely surprised by the ideas people have come up with regarding how and for what purposes Turbomole can be used. The ability to mix everything with everything has always been a popular playground, and it has actually resulted in some new useful methods...

Good luck with your calculations!

[uwe]

Hi,

I have uploaded my jobex script which works with the new g-xtb binary. Please see this post:

https://forum.turbomole.org/index.php/topic,1465.0.html

You could simply copy the xtb binary from https://github.com/grimme-lab/g-xtb and replace the xtb in TURBOMOLE (it is called TURBOMOLE/bin/em64t-unknown-linux-gnu_smp/xtb_omp there). Or provide the path to xtb and use the -l option when calling jobex (as described in the other post mentioned above).

Best Regards, Uwe

[acd81]

Thanks Uwe for all your efforts

I hate to appear as a somewhat ungrateful sod, but there seem to be some minor issues with the newish jobex script. Cutting the first line in energy/gradient files delivered by the gxtb executable for each structure is perhaps not advisable, especially for the first geometry.

Thanks again
Paul

[uwe]

Hello,

if it is not working for you then we need to find out why. You are not ungrateful, you probably just found a bug :-)

So what exactly is not working for you? Please try the following:

• create a new empty directory
• add a coord file with a bad version of water:
  $coord
     0 0 0 o
     0 0 1.1 h
     0 0.9 0 h
  $end
  
• run the new jobex and use g-xtb:
  ./jobex -level gxtb -gxtbopt "--gxtb" -l /opt/g-xtb/
  

It should converge after 13 cycles. The format of the first line of energy is different than the others, because in the first iteration the energy file is just the one that comes out from xtb. The next iterations the jobex script replaces the numbering and adds the energy to the list instead of overwriting the file.

Here are my energy and gradient files:

$energy
     1   -75.23325235144   -75.23325235144   -75.23325235144
2 -75.66593433263 -75.66593433263 -75.66593433263
3 -75.99730204657 -75.99730204657 -75.99730204657
4 -76.22329392624 -76.22329392624 -76.22329392624
5 -76.34119446249 -76.34119446249 -76.34119446249
6 -76.40004722231 -76.40004722231 -76.40004722231
7 -76.42682269276 -76.42682269276 -76.42682269276
8 -76.43633980622 -76.43633980622 -76.43633980622
9 -76.43758038512 -76.43758038512 -76.43758038512
10 -76.43763301930 -76.43763301930 -76.43763301930
11 -76.43764128486 -76.43764128486 -76.43764128486
12 -76.43764148550 -76.43764148550 -76.43764148550
13 -76.43764151471 -76.43764151471 -76.43764151471
$end

$grad
  cycle =      1    SCF energy =   -75.23325235144   |dE/dxyz| =  5.556497
    0.00000000000000      0.00000000000000      0.00000000000000      o
    0.00000000000000      0.00000000000000      1.10000000000000      h
    0.00000000000000      0.90000000000000      0.00000000000000      h
  -2.0461731509913E-16   3.5237385695318E+00   1.4068150535933E+00
   2.2280144153358E-16   1.8274760112836E-01  -1.6301741545061E+00
  -1.8184126434449E-17  -3.7064861706601E+00   2.2335910091283E-01
  cycle =      2    SCF energy =   -75.66593433263   |dE/dxyz| =  3.350495
    0.00000000000000     -0.07858558774573     -0.03991867793921      o
    0.00000000000000      0.02145544451535      1.11346876538194      h
    0.00000000000000      0.95713014323038      0.02644991255727      h
  -5.0069761198835E-17   1.9440895371739E+00   1.2269791737467E+00
  -9.7118128696191E-17   9.4029440959888E-02  -1.3295019842453E+00
   1.4718788989503E-16  -2.0381189781338E+00   1.0252281049859E-01
  cycle =      3    SCF energy =   -75.99730204657   |dE/dxyz| =  1.991257
    0.00000000000000     -0.13465525426622     -0.12847465184543      o
    0.00000000000000      0.02819765844538      1.19980882856020      h
    0.00000000000000      1.00645759582083      0.02866582328523      h
   7.6320232225114E-17   1.1823113012197E+00   6.6389152058688E-01
  -4.6697582343410E-17   9.4388917836092E-02  -6.9748931654029E-01
  -2.9622649881704E-17  -1.2767002190557E+00   3.3597795953410E-02
  cycle =      4    SCF energy =   -76.22329392624   |dE/dxyz| =  1.049930
    0.00000000000000     -0.19162644043428     -0.15973148700441      o
    0.00000000000000     -0.02501126149334      1.29079412665841      h
    0.00000000000000      1.11663770192761     -0.03106263965399      h
   1.9344154798775E-16   5.9500878478953E-01   3.3347248779876E-01
  -1.2425141495107E-16   8.1269971032125E-02  -4.0917644332189E-01
  -6.9190133036678E-17  -6.7627875582165E-01   7.5703955523125E-02
  cycle =      5    SCF energy =   -76.34119446249   |dE/dxyz| =  0.543904
    0.00000000000000     -0.23683555999509     -0.18472120076716      o
    0.00000000000000     -0.08616738013912      1.38511608501192      h
    0.00000000000000      1.22300294013419     -0.10039488424475      h
   8.4229923440197E-17   2.9352245080086E-01   1.5137241309517E-01
  -4.8178607073076E-17   6.4181705359630E-02  -2.2271307378276E-01
  -3.6051316367122E-17  -3.5770415616048E-01   7.1340660687591E-02
  cycle =      6    SCF energy =   -76.40004722231   |dE/dxyz| =  0.260061
    0.00000000000000     -0.26719773920514     -0.19480074467953      o
    0.00000000000000     -0.16014336536825      1.47828952658074      h
    0.00000000000000      1.32734110457337     -0.18348878190120      h
   3.7523750188200E-17   1.2932939905598E-01   5.5621712522371E-02
  -9.2124459164383E-18   4.5861997263104E-02  -1.0987703785910E-01
  -2.8311304271762E-17  -1.7519139631908E-01   5.4255325336725E-02
  cycle =      7    SCF energy =   -76.42682269276   |dE/dxyz| =  0.105543
    0.00000000000000     -0.27491894382930     -0.18415998118590      o
    0.00000000000000     -0.25113687676234      1.56839620467105      h
    0.00000000000000      1.42605582059162     -0.28423622348515      h
  -3.5213217936345E-17   4.4563075057829E-02   1.0675627291078E-02
   2.0937235301532E-17   2.7675445577763E-02  -4.4056428192751E-02
   1.4275982634814E-17  -7.2238520635592E-02   3.3380800901674E-02
  cycle =      8    SCF energy =   -76.43633980622   |dE/dxyz| =  0.026925
    0.00000000000000     -0.25946203646841     -0.15476380348449      o
    0.00000000000000     -0.34942386590239      1.64614528952160      h
    0.00000000000000      1.50888590237078     -0.39138148603712      h
  -1.7234030724725E-17   7.0009616996206E-03  -3.7902502051992E-03
   4.9028665831785E-17   1.0589034787377E-02  -8.8948473483115E-03
  -3.1794635107060E-17  -1.7589996486997E-02   1.2685097553511E-02
  cycle =      9    SCF energy =   -76.43758038512   |dE/dxyz| =  0.005224
    0.00000000000000     -0.24288144448340     -0.13274817893793      o
    0.00000000000000     -0.40075999909210      1.67936091843045      h
    0.00000000000000      1.54364144357549     -0.44661273949253      h
  -1.2130984280538E-16  -9.3179504721277E-04  -3.4618180096819E-03
   7.0120236151876E-17   2.2653625158260E-03   8.5422504561657E-04
   5.1189606653505E-17  -1.3335674686131E-03   2.6075929640653E-03
  cycle =     10    SCF energy =   -76.43763301930   |dE/dxyz| =  0.003117
    0.00000000000000     -0.23375154452123     -0.12209673683244      o
    0.00000000000000     -0.41937777537035      1.68845639253280      h
    0.00000000000000      1.55312931989158     -0.46635965570038      h
  -4.9625232352233E-17  -1.0524466986104E-03  -1.1181674317567E-03
   3.1442156551813E-17  -6.2912507435198E-04   1.8838843623823E-03
   1.8183075800420E-17   1.6815717729623E-03  -7.6571693062560E-04
  cycle =     11    SCF energy =   -76.43764128486   |dE/dxyz| =  0.000750
    0.00000000000000     -0.23404963615214     -0.12273818113333      o
    0.00000000000000     -0.41603485467885      1.68543760990797      h
    0.00000000000000      1.55008449083099     -0.46269942877466      h
  -2.3069946409549E-18   5.4153431131609E-05  -5.0611967725882E-04
   8.0560537969641E-18  -8.4040324539837E-05   5.4170928899070E-04
  -5.7490591560092E-18   2.9886893408245E-05  -3.5589611731815E-05
  cycle =     12    SCF energy =   -76.43764148550   |dE/dxyz| =  0.000499
    0.00000000000000     -0.23388750688097     -0.12254327425585      o
    0.00000000000000     -0.41581352835099      1.68499089391104      h
    0.00000000000000      1.54970103523196     -0.46244761965521      h
  -4.8521457798800E-17   2.6901614890052E-04  -2.5892930706005E-04
   4.3834270317003E-17  -4.3659374293263E-05   2.3651212981162E-04
   4.6871874817967E-18  -2.2535677460736E-04   2.2417177248395E-05
  cycle =     13    SCF energy =   -76.43764151471   |dE/dxyz| =  0.000434
    0.00000000000000     -0.23392298889678     -0.12251469891373      o
    0.00000000000000     -0.41576425783174      1.68491726862495      h
    0.00000000000000      1.54968724672851     -0.46240256971124      h
   1.1213882268587E-17   2.4466047684702E-04  -2.1456841664086E-04
  -6.3030276948246E-18  -2.8642319166333E-05   1.8422444438995E-04
  -4.9108545737628E-18  -2.1601815768071E-04   3.0343972250893E-05
$end

[acd81]

jobex terminates at the 1st step as the first line in energy are gradient files or missing

energy file:
1 -75.23325235144 -75.23325235144 -75.23325235144
$end

gradient file:
  cycle =      1    SCF energy =   -75.23325235144   |dE/dxyz| =  5.556497
    0.00000000000000      0.00000000000000      0.00000000000000      o
    0.00000000000000      0.00000000000000      1.10000000000000      h
    0.00000000000000      0.90000000000000      0.00000000000000      h
  -2.9504095650600E-17   3.5237385695318E+00   1.4068150535933E+00
   4.8794670977309E-17   1.8274760112836E-01  -1.6301741545061E+00
  -1.9290575326709E-17  -3.7064861706601E+00   2.2335910091283E-01
$end

just the $energy and $grad flags are missing, unfortunately

Cheers, Paul

[acd81]

suffice to say I'm using Turbomole version 7.8
perhaps that explains the somewhat odd behaviour

[acd81]

the script runs into problems at line 861
  if [[ $first_cycle -eq 1 ]]; then


outout to stderr
/home/pitti/opt/Tmol-7.8/scripts/jobexN: 861: [[: not found
grep: energy.old: No such file or directory
grep: gradient.old: No such file or directory
awk: cannot open energy.old (No such file or directory)
------------------

[uwe]

Hi,

could you change the first line from

#!/bin/sh

to

#!/bin/bash

and retry?

[acd81]

Halleluhja
/usr/bin/bash (in my particular UNIX installation) it is - all seems to be working fine

always trust the more sophisticated bash, especially if there is no softlink that refers 'sh' back to 'bash'.

What a relief!
Thanks a lot again.

Cheers, Paul