Error from function deal_with_sph_node_request

From: Goddard, Julianne <Julianne.Goddard_at_uky.edu>
Date: Fri, 22 Oct 2021 20:14:43 +0000

Hello Everyone,

I am running a zoom-in cosmological simulation with periodic boundary conditions in Gadget4. I am using grackle for cooling and star formation is enabled. The zoom region in the simulation is about 1.5 Mpc in radius, and the effective resolution here is 1024^3. I have found that the code runs to completion if I run on only one node, however if I increase to two or more nodes I start to get one of the following errors:

"Code termination on task=91, function deal_with_sph_node_request(), file src/mpi_utils/shared_mem_handler.cc, line 272: p=1564695652 MaxPart=5869 MaxNodes=13117"

or

"Fatal error in PMPI_Recv: Unknown error class, error stack:
PMPI_Recv(171)........................: MPI_Recv(buf=0x7f63546475c0, count=8, MPI_BYTE, src=31, tag=10, MPI_COMM_WORLD, status=0x1) failed
MPIDU_Complete_posted_with_error(1137): Process failed"

I have once had the code complete running in parallel without experiencing these errors, but since I have not been able to replicate. Has anyone else experienced this type of error or have advice on how to fix the problem?

Thank You,

Julianne
Received on 2021-10-22 22:15:00