forcing domain decomposition after deleting particles

From: Mark Baumann <markb_at_physics.utexas.edu>
Date: Fri, 24 Jun 2011 18:46:58 -0500 (CDT)

Hello,

I am deleting SPH particles when they collide with a sink particle.
After doing so I force a domain decomposition using:

All.NumForcesSinceLastDomainDecomp = 1 + All.TreeDomainUpdateFrequency *
All.TotNumPart;

This line only updates the value of NumForcesSinceLastDomainDecomp on the
local processor, but I've noticed that this usually triggers domain
decomposition on all processes since they will sometimes exchange
particles. Therefore, is it sufficient to just set the local value, or
should I export the new value to all processes?

Currently I am experiencing an unexpected crash on one of the processes
during the "tree force" step. I've noticed that domain decomposition is
only occurring for the process on which the particle was deleted and I am
wondering if that could be part of the reason for the crash.

I have included the following code in the routine gravity_tree() but
otherwise I have not modified the tree force or domain decomposition code.

for(i=0, NumForceUpdate = 0; i<NumPart; i++)
   {
     if (P[i].Ti_endstep == All.Ti_Current)
#ifdef SELECTIVE_NO_GRAVITY
       if (!((1 << P[i].Type) & (SELECTIVE_NO_GRAVITY)))
#endif
         NumForceUpdate++;
   }

I am updating the local lists of particles (P and SphP) as well as the
values of NumPart and N_gas whenever a particle is deleted. After that I
synchronize the new particle numbers on the rest of the processors using
Allgather, which appears to be properly updating the value of
All.TotNumPart (and All.TotN_gas) an all processes.

Another question I have is, when should I modify the value of
NumForcesSinceLastDomainDecomp? Presumably after modifying the local
particle counts, but does it matter whether I do it before or after
synchronizing All.TotNumPart (and All.TotN_gas)?

Thank you for any feedback!

Mark
Received on 2011-06-25 02:05:03