ccsd(t) progress and restart

[Turboooo]

Hi,
I do ccsd (t) calculation. Unfortunately, the walltime limit on my server is very low  . Therefore, I have some questions:

1. Can be determined how much percent of the triples have been calculated or how far the calculation is?

2. Can a canceled (due to walltime limit) ccsd(t) be restarted or is there some or is there some kind of workaround for this problem?

Greetings
Turboooo

[antti_karttunen]

Hi,

I definitely agree that some indicator on the progress of CCSD(T) triples would be very helpful!

As for the restart, I think there is some capability built in. CCSD works nicely, at least. I haven't tried CCSD(T), but it told me:

  hard_restart flag is disabled because the check sum for the
  R0 amplitudes "$chkr0_CCSD(T) " differs from the reference value:

(I had not saved the scratch files)

Antti

[Turboooo]

The ccsd restart works nicely. One can also do first a ccsd calculation and use in a second step the result for a ccsd(t) calculation (This is good due to the walltime limit of the server). Unfortunately I don't no how to restart the ccsd(t) calculation. The "statistics.ricc2" file is the only file in my scratch diretory that change during my calculation. I don't belive that this is enough input to restart the calcualtion.

[Arnim]

Hi,

the perturbative triples correction can not be restarted (similar to MP2 energies).

The statistics.ricc2 file can be used to monitor the progress of the triples calculation. If the calculation does not fit completely in memory, batches will be used. The number of batches will be printed in that file and starting time of each batch.

Cheers,

Arnim

[Turboooo]

Thanks for the replies.

How can the statistics.ricc2 file used to monitor the progress? I see in the file statistics.ricc2 how many triplet calculations were already started.  Is there a way to estimate how many triplets have yet to be calculated?


Greetings
Turboooo

[Arnim]

If the 'num. of batches for virtuals:' is n, you will get n * (n+1)/2 * (n+2)/3 batches.

[Hauke]

The statistics.ricc2 file can be used to monitor the progress of the triples calculation. If the calculation does not fit completely in memory, batches will be used. The number of batches will be printed in that file and starting time of each batch.

I'm using TM 6.3.1 and my statistics.ricc2 contains for example 100 lines and looks like

Code: [Select]

start with occ. triple  1-  1-  2 at 2011-11-24 12:20:27.229
start with occ. triple  3- 13- 17 at 2011-11-24 12:20:33.482
start with occ. triple  6- 21- 27 at 2011-11-24 12:20:39.724
start with occ. triple 11- 16- 27 at 2011-11-24 12:20:45.928
start with occ. triple 23- 27- 28 at 2011-11-24 12:20:52.059
start with occ. triple  1-  1-  2 at 2011-11-24 12:20:55.735
start with occ. triple  3- 13- 17 at 2011-11-24 12:21:15.323
start with occ. triple  6- 21- 27 at 2011-11-24 12:21:35.173
start with occ. triple 11- 16- 27 at 2011-11-24 12:21:55.015
start with occ. triple 23- 27- 28 at 2011-11-24 12:22:14.883
start with occ. triple  1-  1-  2 at 2011-11-24 12:22:19.074
start with occ. triple  3- 13- 17 at 2011-11-24 12:22:38.594
start with occ. triple  6- 21- 27 at 2011-11-24 12:22:58.313
start with occ. triple 11- 16- 27 at 2011-11-24 12:23:18.064
start with occ. triple 23- 27- 28 at 2011-11-24 12:23:37.738
start with occ. triple  1-  1-  2 at 2011-11-24 12:23:41.482
start with occ. triple  3- 13- 17 at 2011-11-24 12:24:01.214
start with occ. triple  6- 21- 27 at 2011-11-24 12:24:20.946
start with occ. triple 11- 16- 27 at 2011-11-24 12:24:40.704
start with occ. triple 23- 27- 28 at 2011-11-24 12:25:00.381
start with occ. triple  1-  1-  2 at 2011-11-24 12:25:03.965
start with occ. triple  3- 13- 17 at 2011-11-24 12:25:23.678
start with occ. triple  6- 21- 27 at 2011-11-24 12:25:43.385
start with occ. triple 11- 16- 27 at 2011-11-24 12:26:03.115
start with occ. triple 23- 27- 28 at 2011-11-24 12:26:22.766
start with occ. triple  1-  1-  2 at 2011-11-24 12:26:29.696
start with occ. triple  3- 13- 17 at 2011-11-24 12:27:10.478
start with occ. triple  6- 21- 27 at 2011-11-24 12:27:51.329
start with occ. triple 11- 16- 27 at 2011-11-24 12:28:32.217
start with occ. triple 23- 27- 28 at 2011-11-24 12:29:13.149
start with occ. triple  1-  1-  2 at 2011-11-24 12:29:20.326
start with occ. triple  3- 13- 17 at 2011-11-24 12:30:01.102
start with occ. triple  6- 21- 27 at 2011-11-24 12:30:41.932
start with occ. triple 11- 16- 27 at 2011-11-24 12:31:22.849
start with occ. triple 23- 27- 28 at 2011-11-24 12:32:03.755
start with occ. triple  1-  1-  2 at 2011-11-24 12:32:08.018
start with occ. triple  3- 13- 17 at 2011-11-24 12:32:27.643
start with occ. triple  6- 21- 27 at 2011-11-24 12:32:47.262
start with occ. triple 11- 16- 27 at 2011-11-24 12:33:06.874
start with occ. triple 23- 27- 28 at 2011-11-24 12:33:26.697
start with occ. triple  1-  1-  2 at 2011-11-24 12:33:33.249
start with occ. triple  3- 13- 17 at 2011-11-24 12:34:13.940
start with occ. triple  6- 21- 27 at 2011-11-24 12:34:54.681
start with occ. triple 11- 16- 27 at 2011-11-24 12:35:38.171
start with occ. triple 23- 27- 28 at 2011-11-24 12:36:23.710
start with occ. triple  1-  1-  2 at 2011-11-24 12:36:28.540
start with occ. triple  3- 13- 17 at 2011-11-24 12:36:51.403
start with occ. triple  6- 21- 27 at 2011-11-24 12:37:12.412
start with occ. triple 11- 16- 27 at 2011-11-24 12:37:34.778
start with occ. triple 23- 27- 28 at 2011-11-24 12:37:57.462
start with occ. triple  1-  1-  2 at 2011-11-24 12:37:59.296
start with occ. triple  3- 13- 17 at 2011-11-24 12:38:06.674
start with occ. triple  6- 21- 27 at 2011-11-24 12:38:14.019
start with occ. triple 11- 16- 27 at 2011-11-24 12:38:21.326
start with occ. triple 23- 27- 28 at 2011-11-24 12:38:28.527
start with occ. triple  1-  1-  2 at 2011-11-24 12:38:31.972
start with occ. triple  3- 13- 17 at 2011-11-24 12:38:53.730
start with occ. triple  6- 21- 27 at 2011-11-24 12:39:14.854
start with occ. triple 11- 16- 27 at 2011-11-24 12:39:34.605
start with occ. triple 23- 27- 28 at 2011-11-24 12:39:55.328
start with occ. triple  1-  1-  2 at 2011-11-24 12:39:59.261
start with occ. triple  3- 13- 17 at 2011-11-24 12:40:22.918
start with occ. triple  6- 21- 27 at 2011-11-24 12:40:46.733
start with occ. triple 11- 16- 27 at 2011-11-24 12:41:10.600
start with occ. triple 23- 27- 28 at 2011-11-24 12:41:32.200
start with occ. triple  1-  1-  2 at 2011-11-24 12:41:36.151
start with occ. triple  3- 13- 17 at 2011-11-24 12:41:58.510
start with occ. triple  6- 21- 27 at 2011-11-24 12:42:20.821
start with occ. triple 11- 16- 27 at 2011-11-24 12:42:43.708
start with occ. triple 23- 27- 28 at 2011-11-24 12:43:05.406
start with occ. triple  1-  1-  2 at 2011-11-24 12:43:12.539
start with occ. triple  3- 13- 17 at 2011-11-24 12:43:58.755
start with occ. triple  6- 21- 27 at 2011-11-24 12:44:44.379
start with occ. triple 11- 16- 27 at 2011-11-24 12:45:28.817
start with occ. triple 23- 27- 28 at 2011-11-24 12:46:14.206
start with occ. triple  1-  1-  2 at 2011-11-24 12:46:18.984
start with occ. triple  3- 13- 17 at 2011-11-24 12:46:41.590
start with occ. triple  6- 21- 27 at 2011-11-24 12:47:03.566
start with occ. triple 11- 16- 27 at 2011-11-24 12:47:25.176
start with occ. triple 23- 27- 28 at 2011-11-24 12:47:47.700
start with occ. triple  1-  1-  2 at 2011-11-24 12:47:49.402
start with occ. triple  3- 13- 17 at 2011-11-24 12:47:55.332
start with occ. triple  6- 21- 27 at 2011-11-24 12:48:01.265
start with occ. triple 11- 16- 27 at 2011-11-24 12:48:07.183
start with occ. triple 23- 27- 28 at 2011-11-24 12:48:13.031
start with occ. triple  1-  1-  2 at 2011-11-24 12:48:16.159
start with occ. triple  3- 13- 17 at 2011-11-24 12:48:35.450
start with occ. triple  6- 21- 27 at 2011-11-24 12:48:54.702
start with occ. triple 11- 16- 27 at 2011-11-24 12:49:13.930
start with occ. triple 23- 27- 28 at 2011-11-24 12:49:33.099
start with occ. triple  1-  1-  2 at 2011-11-24 12:49:36.654
start with occ. triple  3- 13- 17 at 2011-11-24 12:49:55.908
start with occ. triple  6- 21- 27 at 2011-11-24 12:50:16.864
start with occ. triple 11- 16- 27 at 2011-11-24 12:50:37.675
start with occ. triple 23- 27- 28 at 2011-11-24 12:50:56.834
start with occ. triple  1-  1-  2 at 2011-11-24 12:50:58.432
start with occ. triple  3- 13- 17 at 2011-11-24 12:51:04.377
start with occ. triple  6- 21- 27 at 2011-11-24 12:51:10.332
start with occ. triple 11- 16- 27 at 2011-11-24 12:51:16.255
start with occ. triple 23- 27- 28 at 2011-11-24 12:51:22.103

I don't see any number of batches. Is this feature only in TM 6.4 or does it mean that in my case the calculation was still completely in memory ($maxcor was just 1 GB and even for bigger calculations I don't see any difference structure in statistics.ricc2 ). Monitoring the progress of triples would be helpful.

[Arnim]

Oh, I forgot... In versions < 6.5 you have to add $profile to control. And then 'grep "start with batch" statistics.ricc2'.

Arnim

[Turboooo]

Thanks, now I get it.

It would be awesome, if Turbomole gets some kind of soft stop function.
For example on the HLRN it is only allowed to calculate 12 hours. Therefore it would be nice if one can tell Turbomole to calculate only 11 hours and than save all you doing and all in the memory to scratch. So you can restart your ccsd(t) calculation afterwards.