RE: Re: Gadget stalling without apparent reason

From: Raymond Carlberg <raymond.carlberg_at_utoronto.ca>
Date: Mon, 31 Oct 2016 23:42:38 +0000

HI - I am running Gadget2 on a 32 node machine. Works well for more than 10,000 steps, then dt goes to zero with no complaints from the program so the run continues for some time with no progress. DONOTSTOP is not implemented. Softenings are good. TreeAllocFactor boosted, etc, but no go. The gravity calculation has extreme density ranges.
--Ray

Restarting shows a possible problem (Nf=0 below):
Step= 9 t= 1.55115 dt= 0.000128937
Nf= 0000064647 total-Nf= 0001622692 ex-frac= 21.9985 iter= 1
work-load balance: 18.532 max=4.10968 avg=0.221762 PE0=1.98117
particle-load balance: 4.77931
max. nodes: 37131, filled: 0.0066013
part/sec=4554.93 | 245.788 ia/part=5650.51 (0)

Step= 10 t= 1.55115 dt= 0
Nf= 0000000000 total-Nf= 0001622692 ex-frac= -nan iter= 0
work-load balance: -nan max=0 avg=0 PE0=0
particle-load balance: 4.77931
max. nodes: 37131, filled: 0.0066013
part/sec=0 | -nan ia/part=-nan (-nan)

^^^^^^^^^^^^^^^^^^^and the output at the time of the problem.

Begin Step 13838, Time: 1.55121, Systemstep: 0.000152588
domain decomposition...
NTopleaves= 4124
work-load balance=36.5349 memory-balance=4.45177
exchange of 0000288372 particles
domain decomposition done.
begin Peano-Hilbert order...
Peano-Hilbert done.
Start force computation...
Tree construction.
Tree construction done.
Begin tree force.
tree is done.
force computation done.

Begin Step 13839, Time: 1.55121, Systemstep: 0
Start force computation...
Begin tree force.
tree is done.
force computation done.
Received on 2016-11-01 00:42:42