Dear gadget-list members,
In the course of furthering my understanding of the issue I performed a
health-test using the available config option provided with G4.
Attached is the corresponding log output for a run, on a representative
subset of the available nodes of the cluster I am working on.
Looking at these results it seems very surprising for me, that the data
transmission rate for the intranode cube is so much lower than the one
for the internode cube. This makes me wonder if maybe in the code the
words "inter" and "intra" were swapped (I would expect "intra" to mean
within a node and "inter" between separate nodes). The resulting full
hypercube communication seems very slow too, so overall, I think there
might be a problem with the machines anyways, but it would be helpful if
someone could clear up this issue, so I can provide accurate information
to the system administration.
Best regards,
Leonard Romano
On 09.06.21 17:09, Leonard Romano wrote:
> Dear gadget-list members,
>
> When I am running cosmological simulations with Gadget-4 I notice that
> the domain decomposition becomes a dominant part of the computational
> cost.
>
> I am running a simulation with 256³ gas and DM particles on 44 Intel
> Xeon (e5-2699a v4) nodes, and the domain decomposition keeps getting
> overwhelmingly expensive (20%-50% of the total CPU time). Curiously
> with the same settings but only 2x128³ particles it only takes about 10%.
> From a quick glance at the code-paper I would have expected the
> opposite behavior.
>
> Attached are the config-options I compiled with and my parameters.
> I would be very grateful if someone has any suggestions or comments
> about how to improve or understand this behaviour.
>
> Best,
> Leonard
>
--
===================================================
Leonard Romano, B.Sc.（レオナルド・ロマノ）
Physics Department
Technical University of Munich (TUM), Germany
Theoretical Astrophysics Group
Department of Earth and Space Science
Graduate School of Science, Osaka University, Japan
he / him / his
===================================================
Received on 2021-06-13 22:02:19