Hangup when generating initial conditions

From: Ken Osato <ken.osato_at_yukawa.kyoto-u.ac.jp>
Date: Thu, 27 Jan 2022 13:38:03 +0100

Dear Gadget users,

Actually, I had a similar problem raised by Julianne, related to the
routine shared_memory_handler(), when running gravity-only simulations
with Gadget-4.
The error seems to occur for MPICH-based libraries since I'm also using
Intel MPI (v. 2020.4.304) on our cluster.

Volker has already fixed this issue and I've run the simulation in order
to test the code in my environment.
First, I've run the simulation with 1024^3 particles and the run is
successful without errors.
However, when I increase the number of particles to 2048^3, it hangs up
in generating initial conditions.
This error occurs for both of analytic calculations
(PowerSpectrumType=1) and loading table (PowerSpectrumType=2).
I attach the log files for this run.
When I ran the simulation with older version of Gadget-4 (Git commit
b4bb065ce3dec478d2a2d7101cefc5f5faade084, Wed Dec 23 17:05:02 2020
+0100), there was no error for initial conditions.
I think the current error again might be related to the different
implementation between OpenMPI and MPICH.

There is also a quite minor error about finalization. I always find the
following error message every time the job ends.
> Abort(806969615) on node 185 (rank 185 in comm 0): Fatal error in
> PMPI_Finalize: Other MPI error, error stack:
> PMPI_Finalize(214)...............: MPI_Finalize failed
> PMPI_Finalize(159)...............:
> MPID_Finalize(1288)..............:
> MPIDI_OFI_mpi_finalize_hook(1892): OFI domain close failed
> (ofi_init.c:1892:MPIDI_OFI_mpi_finalize_hook:Device or resource busy)
But it seems that the job successfully finishes, the log file (stdout)
ends with "endrun called, calling MPI_Finalize() bye!".
And I found no errors in output snapshots. Probably, it might be also
due to MPI libraries.

Best regards,

Ken Osato
Yukawa Institute for Theoretical Physics, Kyoto University
Kitashirakawa Oiwakecho, Sakyo-ku, Kyoto 606-8502, Japan
Tel: +81-75-753-7000
E-mail: ken.osato_at_yukawa.kyoto-u.ac.jp

Received on 2022-01-27 13:38:25

This archive was generated by hypermail 2.3.0 : 2023-01-10 10:01:33 CET