Not enough memory

From: Hassan Fathivavsari <h.fathie_at_gmail.com>
Date: Thu, 11 Feb 2021 19:58:32 +0100

Dear all,

I am a new user of GADGET.
I have created an initial condition file using MUSIC. But when I execute
the GADGET
code (mpirun -np 6 ./Gadget4 param.txt), I got the following error:

------------------------
Code termination on task=5, function mymalloc_movable_fullinfo(), file
src/data/mymalloc.cc, line 326:
Not enough memory in mymalloc_fullinfo() to allocate 685.336 MB for
variable 'rhogrid' at pmforce_zoom_optimized_prepare_density()/src/pm/
pm_periodic.cc/line 378 (FreeBytes=97.1207 MB).
-------------------------

But I have enough memory on my Laptop. I have also pasted below the full
message
printed out in the Terminal:
I would be very grateful if some one could help me in this issue.

With best regards,
Hassan

===================== Full Terminal Message =========================

pc_at_ubuntu:~/Desktop/SPH/gadget4-master/IC_from_MUSIC$ mpirun -np 6
./Gadget4 param.txt
Shared memory islands host a minimum of 6 and a maximum of 6 MPI ranks.

  ___ __ ____ ___ ____ ____ __
 / __) /__\ ( _ \ / __)( ___)(_ _)___ /. |
( (_-. /(__)\ )(_) )( (_-. )__) )( (___)(_ _)
 \___/(__)(__)(____/ \___/(____) (__) (_)

This is Gadget, version 4.0.
Git commit unknown, unknown

Code was compiled with the following compiler and flags:
mpicxx -std=c++11 -O3 -Wall -I/hdf5/build/include -I/usr/local/include
-I/usr/local/include -I./IC_from_MUSIC//build -Isrc


Code was compiled with the following settings:
    EVALPOTENTIAL
    FOF
    FOF_GROUP_MIN_LEN=32
    FOF_PRIMARY_LINK_TYPES=2
    GADGET2_HEADER
    INDIVIDUAL_GRAVITY_SOFTENING=4+8+16+32
    NSOFTCLASSES=4
    PERIODIC
    PMGRID=1024
    PM_ZOOM_OPTIMIZED
    POWERSPEC_ON_OUTPUT
    SELFGRAVITY
    SUBFIND


Running on 6 MPI tasks.


BEGRUN: Size of particle structure 84 [bytes]
BEGRUN: Size of sph particle structure 96 [bytes]
BEGRUN: Size of gravity tree node 72 [bytes]
BEGRUN: Size of neighbour tree node 112 [bytes]
BEGRUN: Size of subfind auxiliary data 80 [bytes]


-------------------------------------------------------------------------------------------------------------------------
AvailMem: Largest = 2795.93 Mb (on task= 0), Smallest = 2795.93 Mb
(on task= 0), Average = 2795.93 Mb
Total Mem: Largest = 6267.95 Mb (on task= 0), Smallest = 6267.95 Mb
(on task= 0), Average = 6267.95 Mb
Committed_AS: Largest = 3472.02 Mb (on task= 0), Smallest = 3472.02
Mb (on task= 0), Average = 3472.02 Mb
SwapTotal: Largest = 2048.00 Mb (on task= 0), Smallest = 2048.00 Mb
(on task= 0), Average = 2048.00 Mb
SwapFree: Largest = 2048.00 Mb (on task= 0), Smallest = 2048.00 Mb
(on task= 0), Average = 2048.00 Mb
AllocMem: Largest = 3472.02 Mb (on task= 0), Smallest = 3472.02 Mb
(on task= 0), Average = 3472.02 Mb
avail /dev/shm: Largest = 3132.36 Mb (on task= 0), Smallest =
 3132.36 Mb (on task= 0), Average = 3132.36 Mb
-------------------------------------------------------------------------------------------------------------------------
Task=0 has the maximum commited memory and is host: ubuntu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Obtaining parameters from file 'param.txt':

        InitCondFile ./ics_gadget.dat
        OutputDir ./output
        SnapshotFileBase snapshot
        OutputListFilename ./ExpansionList_16
        ICFormat 1
        SnapFormat 3
        TimeLimitCPU 86400
        CpuTimeBetRestartFile 7200
        MaxMemSize 100
        TimeBegin 0.0078125
        TimeMax 1
        ComovingIntegrationOn 1
        Omega0 2.20836e-06
        OmegaLambda 0.75
        OmegaBaryon 0.04
        HubbleParam 0.73
        BoxSize 100
        Hubble 100
        OutputListOn 1
        TimeBetSnapshot 0
        TimeOfFirstSnapshot 0
        TimeBetStatistics 0.01
        NumFilesPerSnapshot 1
        MaxFilesWithConcurrentIO 1
        ErrTolIntAccuracy 0.012
        CourantFac 0.3
        MaxSizeTimestep 0.005
        MinSizeTimestep 0
        TypeOfOpeningCriterion 1
        ErrTolTheta 0.7
        ErrTolThetaMax 1
        ErrTolForceAcc 0.0025
        TopNodeFactor 5
        ActivePartFracForNewDomainDecomp 0.01
        DesNumNgb 64
        MaxNumNgbDeviation 1
        UnitLength_in_cm 3.08568e+24
        UnitMass_in_g 1.989e+43
        UnitVelocity_in_cm_per_s 100000
        GravityConstantInternal 0
        SofteningComovingClass0 0.001
        SofteningComovingClass1 0.007
        SofteningComovingClass2 0.025
        SofteningComovingClass3 0.081
        SofteningMaxPhysClass0 0.0005
        SofteningMaxPhysClass1 0.007
        SofteningMaxPhysClass2 0.025
        SofteningMaxPhysClass3 0.081
        SofteningClassOfPartType0 0
        SofteningClassOfPartType1 0
        SofteningClassOfPartType2 1
        SofteningClassOfPartType3 2
        SofteningClassOfPartType4 3
        SofteningClassOfPartType5 3
        DesLinkNgb 20
        ArtBulkViscConst 1
        MinEgySpec 0
        InitGasTemp 0

MALLOC: Allocation of shared memory took 0.000141637 sec

found 16 times in output-list.
BEGRUN: Hubble (internal units) = 100
BEGRUN: h = 0.73
BEGRUN: G (internal units) = 43.0187
BEGRUN: UnitMass_in_g = 1.989e+43
BEGRUN: UnitLenth_in_cm = 3.08568e+24
BEGRUN: UnitTime_in_s = 3.08568e+19
BEGRUN: UnitVelocity_in_cm_per_s = 100000
BEGRUN: UnitDensity_in_cgs = 6.76991e-31
BEGRUN: UnitEnergy_in_cgs = 1.989e+53


READIC: filenr=0, './ics_gadget.dat' contains:
READIC: Type 0 (gas): 0 (tot= 0) masstab= 0
READIC: Type 1: 17576 (tot= 17576) masstab= 3.65236e-09
READIC: Type 2: 0 (tot= 0) masstab= 0
READIC: Type 3: 0 (tot= 0) masstab= 0
READIC: Type 4: 0 (tot= 0) masstab= 0
READIC: Type 5: 54196 (tot= 54196) masstab= 0

READIC: Reading file `./ics_gadget.dat' on task=0 and distribute it to 0 to
5.
READIC: reading block 0 (Coordinates)...
READIC: reading block 1 (Velocities)...
READIC: reading block 2 (ParticleIDs)...
READIC: reading block 3 (Masses)...
READIC: reading done. Took 0.00612891 sec, total size 2.12379 MB,
corresponds to effective I/O rate of 346.521 MB/sec

READIC: Total number of particles : 71772

INIT: Testing ID uniqueness...
INIT: success. took=0.00309694 sec

DOMAIN: Begin domain decomposition (sync-point 0).
DOMAIN: Sum=2 TotalCost=2 NumTimeBinsToBeBalanced=1 MultipleDomains=2
DOMAIN: Increasing TopNodeAllocFactor=0.08 new value=0.104
DOMAIN: Increasing TopNodeAllocFactor=0.104 new value=0.1352
DOMAIN: Increasing TopNodeAllocFactor=0.1352 new value=0.17576
DOMAIN: Increasing TopNodeAllocFactor=0.17576 new value=0.228488
DOMAIN: Increasing TopNodeAllocFactor=0.228488 new value=0.297034
DOMAIN: Increasing TopNodeAllocFactor=0.297034 new value=0.386145
DOMAIN: Increasing TopNodeAllocFactor=0.386145 new value=0.501988
DOMAIN: Increasing TopNodeAllocFactor=0.501988 new value=0.652585
DOMAIN: Increasing TopNodeAllocFactor=0.652585 new value=0.84836
DOMAIN: Increasing TopNodeAllocFactor=0.84836 new value=1.10287
DOMAIN: Increasing TopNodeAllocFactor=1.10287 new value=1.43373
DOMAIN: Increasing TopNodeAllocFactor=1.43373 new value=1.86385
DOMAIN: Increasing TopNodeAllocFactor=1.86385 new value=2.423
DOMAIN: Increasing TopNodeAllocFactor=2.423 new value=3.1499
DOMAIN: NTopleaves=246, determination of top-level tree involved 14
iterations and took 0.00647044 sec
DOMAIN: we are going to try at most 149 different settings for combining
the domains on tasks=6, nnodes=1
DOMAIN: total_cost=2 total_load=1
DOMAIN: best solution found after 1 iterations by task=5 for nextra=0,
reaching maximum imbalance of 1.02449|1.03052
DOMAIN: combining multiple-domains took 0.000829803 sec
DOMAIN: exchange of 71772 particles
DOMAIN: particle exchange done. (took 0.00680363 sec)
DOMAIN: domain decomposition done. (took in total 0.0150899 sec)
PEANO: Begin Peano-Hilbert order...
PEANO: done, took 0.00189733 sec.
INIT: AvgType1Mass = 3.65236e-09 (min=3.65236e-09 max=3.65236e-09)
Ndm1tot=17576
INIT: For this AvgType1Mass, the mean particle spacing is -nan and the
assigned softening is 0.001

SNAPSHOT: Setting next time for snapshot file to Time_next= 0.0288901
 (DumpFlag=1)



Sync-Point 0, Time: 0.0078125, Redshift: 127, Systemstep: 0, Dloga: 0,
Nsync-grv: 71772, Nsync-hyd: 0
DOMAIN: Begin domain decomposition (sync-point 0).
DOMAIN: Sum=2 TotalCost=2 NumTimeBinsToBeBalanced=1 MultipleDomains=2
DOMAIN: NTopleaves=246, determination of top-level tree involved 14
iterations and took 0.00247401 sec
DOMAIN: we are going to try at most 149 different settings for combining
the domains on tasks=6, nnodes=1
DOMAIN: total_cost=2 total_load=1
DOMAIN: best solution found after 1 iterations by task=5 for nextra=0,
reaching maximum imbalance of 1.02449|1.03052
DOMAIN: combining multiple-domains took 0.00157532 sec
DOMAIN: exchange of 71772 particles
DOMAIN: particle exchange done. (took 0.00363552 sec)
DOMAIN: domain decomposition done. (took in total 0.00819422 sec)
PEANO: Begin Peano-Hilbert order...
PEANO: done, took 0.000835626 sec.
ACCEL: Start tree gravity force computation... (71772 particles)
PM-PERIODIC: Starting periodic PM calculation. (Rcut=0.854492) presently
allocated=1.83447 MB
-------------------------- Allocated Memory Blocks---- ( Step 0
)------------------
Task Nr F Variable MBytes Cumulative
 Function|File|Linenumber
------------------------------------------------------------------------------------------
   4 0 0 GetGhostRankForSimulCommRank 0.0001
0.0001 mymalloc_init()|src/data/mymalloc.cc|137
   4 1 0 GetShmRankForSimulCommRank 0.0001
0.0001 mymalloc_init()|src/data/mymalloc.cc|138
   4 2 0 GetNodeIDForSimulCommRank 0.0001
0.0002 mymalloc_init()|src/data/mymalloc.cc|139
   4 3 0 SharedMemBaseAddr 0.0001
0.0002 mymalloc_init()|src/data/mymalloc.cc|153
   4 4 1 slab_to_task 0.0039
0.0042 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|45
   4 5 1 slabs_x_per_task 0.0001
0.0042 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|60
   4 6 1 first_slab_x_of_task 0.0001
0.0043 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|63
   4 7 1 slabs_y_per_task 0.0001
0.0043 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|66
   4 8 1 first_slab_y_of_task 0.0001
0.0044 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|69
   4 9 1 P 1.5972
1.6016 allocate_memory()|src/io/../fof/../data/simparticles.h|273
   4 10 1 SphP 0.0001
1.6017 allocate_memory()|src/io/../fof/../data/simparticles.h|274
   4 11 1 NextActiveParticleHydro 0.0001
1.6017 timebins_allocate()|src/time_integration/timestep.cc|463
   4 12 1 NextInTimeBinHydro 0.0001
1.6018 timebins_allocate()|src/time_integration/timestep.cc|466
   4 13 1 PrevInTimeBinHydro 0.0001
1.6019 timebins_allocate()|src/time_integration/timestep.cc|469
   4 14 1 NextActiveParticleGravity 0.0761
1.6780 timebins_allocate()|src/time_integration/timestep.cc|463
   4 15 1 NextInTimeBinGravity 0.0761
1.7541 timebins_allocate()|src/time_integration/timestep.cc|466
   4 16 1 PrevInTimeBinGravity 0.0761
1.8302 timebins_allocate()|src/time_integration/timestep.cc|469
   4 17 1 FirstTopleafOfTask 0.0001
1.8303 domain_allocate()|src/domain/domain.cc|163
   4 18 1 NumTopleafOfTask 0.0001
1.8303 domain_allocate()|src/domain/domain.cc|164
   4 19 1 TopNodes 0.0022
1.8325 domain_allocate()|src/domain/domain.cc|165
   4 20 1 TaskOfLeaf 0.0010
1.8335 domain_allocate()|src/domain/domain.cc|166
   4 21 1 ListOfTopleaves 0.0010
1.8345 domain_decomposition()|src/domain/domain.cc|118
   4 22 0 part 1.0726
2.9070 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|184
   4 23 1 localfield_globalindex 0.0001
2.9071 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|272
   4 24 1 localfield_data 0.0001
2.9072 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|274
   4 25 1 localfield_first 0.0001
2.9072 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|275
   4 26 1 localfield_sendcount 0.0001
2.9073 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|276
   4 27 1 localfield_offset 0.0001
2.9073 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|277
   4 28 1 localfield_recvcount 0.0001
2.9074 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|278
------------------------------------------------------------------------------------------
Code termination on task=4, function mymalloc_movable_fullinfo(), file
src/data/mymalloc.cc, line 326:
Not enough memory in mymalloc_fullinfo() to allocate 685.336 MB for
variable 'rhogrid' at pmforce_zoom_optimized_prepare_density()/src/pm/
pm_periodic.cc/line 378 (FreeBytes=97.0926 MB).

application called MPI_Abort(MPI_COMM_WORLD, 1) - process 4
pc_at_ubuntu:~/Desktop/SPH/gadget4-master/IC_from_MUSIC$
Received on 2021-02-11 19:58:51

This archive was generated by hypermail 2.3.0 : 2022-09-01 14:03:43 CEST