Domain decomposition takes too much time

From: Johan Maes <>
Date: Fri, 29 Feb 2008 18:21:26 +0100

Dear Gadget list,

We are currently testing the scalability of our new computer cluster
(with dual-core AMD Opteron processors) by simulating the adiabatic
collapse of a 1/r gas sphere. The same simulation was repeated for
1,8,16,24,32 and 64 CPU's. However, as from 24 processors, the
computation time starts to increase. Looking at the cpu.txt files, it
seems that the domain decomposition is taking much more time than it's
supposed to.

#CPU's DD time fraction (%)
1 2.5
8 5
16 21
24 27
32 50

For 64 CPU's, it even takes up ~90% of the computation time:

Step 1010, Time: 0.809326, CPUs: 64
  12829.22 359.31 546.15 11790.72

What could be causing this problem?

Also, I used 3e5 particles for these (dimensionless) simulations.
Strangely enough, when I increase the number to 3e6, the DD time
fraction for 64 CPU's decreases to 63%. Or is this supposed to happen? I
don't know a whole lot about the domain decomposition so this could just
be me doing something wrong...

Thanks for any suggestions/help!

Best regards,

Received on 2008-02-29 18:50:02

This archive was generated by hypermail 2.3.0 : 2022-09-01 14:03:42 CEST