Hi Hassan,
Further to Tiago's message, your setting of
MaxMemSize 100
in the parameterfile allows the code to use at most 100 MB. Without increasing this sufficiently, the code cannot allocate ~685 MB... (independent of what physical memory your machine has)
Regards,
Volker
> On 12. Feb 2021, at 10:07, Tiago Castro <tiagobscastro_at_gmail.com> wrote:
>
> Dear Hassan,
>
> I believe you face the same problem that I did some months ago, your laptop shmem is apparently set to half the total memory of your machine. Check if this thread might help you: https://wwwmpa.mpa-garching.mpg.de/gadget/gadget-list/0804.html
>
> Regards,
> Tiago Castro Post Doc, Department of Physics / UNITS / OATS
> Phone: (+39 040 3199 120)
> Mobile: (+39 388 794 1562)
> Email: tiagobscastro_at_gmail.com
> Website: tiagobscastro.com
> Skype: tiagobscastro
> Address: Osservatorio Astronomico di Trieste / Villa Bazzoni
> Via Bazzoni, 2, 34143 Trieste TS
>
>
>
>
> Em qui., 11 de fev. de 2021 às 20:00, Hassan Fathivavsari <h.fathie_at_gmail.com> escreveu:
> Dear all,
>
> I am a new user of GADGET.
> I have created an initial condition file using MUSIC. But when I execute the GADGET
> code (mpirun -np 6 ./Gadget4 param.txt), I got the following error:
>
> ------------------------
> Code termination on task=5, function mymalloc_movable_fullinfo(), file src/data/mymalloc.cc, line 326:
> Not enough memory in mymalloc_fullinfo() to allocate 685.336 MB for variable 'rhogrid' at pmforce_zoom_optimized_prepare_density()/src/pm/pm_periodic.cc/line 378 (FreeBytes=97.1207 MB).
> -------------------------
>
> But I have enough memory on my Laptop. I have also pasted below the full message
> printed out in the Terminal:
> I would be very grateful if some one could help me in this issue.
>
> With best regards,
> Hassan
>
> ===================== Full Terminal Message =========================
>
> pc_at_ubuntu:~/Desktop/SPH/gadget4-master/IC_from_MUSIC$ mpirun -np 6 ./Gadget4 param.txt
> Shared memory islands host a minimum of 6 and a maximum of 6 MPI ranks.
>
> ___ __ ____ ___ ____ ____ __
> / __) /__\ ( _ \ / __)( ___)(_ _)___ /. |
> ( (_-. /(__)\ )(_) )( (_-. )__) )( (___)(_ _)
> \___/(__)(__)(____/ \___/(____) (__) (_)
>
> This is Gadget, version 4.0.
> Git commit unknown, unknown
>
> Code was compiled with the following compiler and flags:
> mpicxx -std=c++11 -O3 -Wall -I/hdf5/build/include -I/usr/local/include -I/usr/local/include -I./IC_from_MUSIC//build -Isrc
>
>
> Code was compiled with the following settings:
> EVALPOTENTIAL
> FOF
> FOF_GROUP_MIN_LEN=32
> FOF_PRIMARY_LINK_TYPES=2
> GADGET2_HEADER
> INDIVIDUAL_GRAVITY_SOFTENING=4+8+16+32
> NSOFTCLASSES=4
> PERIODIC
> PMGRID=1024
> PM_ZOOM_OPTIMIZED
> POWERSPEC_ON_OUTPUT
> SELFGRAVITY
> SUBFIND
>
>
> Running on 6 MPI tasks.
>
>
> BEGRUN: Size of particle structure 84 [bytes]
> BEGRUN: Size of sph particle structure 96 [bytes]
> BEGRUN: Size of gravity tree node 72 [bytes]
> BEGRUN: Size of neighbour tree node 112 [bytes]
> BEGRUN: Size of subfind auxiliary data 80 [bytes]
>
>
> -------------------------------------------------------------------------------------------------------------------------
> AvailMem: Largest = 2795.93 Mb (on task= 0), Smallest = 2795.93 Mb (on task= 0), Average = 2795.93 Mb
> Total Mem: Largest = 6267.95 Mb (on task= 0), Smallest = 6267.95 Mb (on task= 0), Average = 6267.95 Mb
> Committed_AS: Largest = 3472.02 Mb (on task= 0), Smallest = 3472.02 Mb (on task= 0), Average = 3472.02 Mb
> SwapTotal: Largest = 2048.00 Mb (on task= 0), Smallest = 2048.00 Mb (on task= 0), Average = 2048.00 Mb
> SwapFree: Largest = 2048.00 Mb (on task= 0), Smallest = 2048.00 Mb (on task= 0), Average = 2048.00 Mb
> AllocMem: Largest = 3472.02 Mb (on task= 0), Smallest = 3472.02 Mb (on task= 0), Average = 3472.02 Mb
> avail /dev/shm: Largest = 3132.36 Mb (on task= 0), Smallest = 3132.36 Mb (on task= 0), Average = 3132.36 Mb
> -------------------------------------------------------------------------------------------------------------------------
> Task=0 has the maximum commited memory and is host: ubuntu
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Obtaining parameters from file 'param.txt':
>
> InitCondFile ./ics_gadget.dat
> OutputDir ./output
> SnapshotFileBase snapshot
> OutputListFilename ./ExpansionList_16
> ICFormat 1
> SnapFormat 3
> TimeLimitCPU 86400
> CpuTimeBetRestartFile 7200
> MaxMemSize 100
> TimeBegin 0.0078125
> TimeMax 1
> ComovingIntegrationOn 1
> Omega0 2.20836e-06
> OmegaLambda 0.75
> OmegaBaryon 0.04
> HubbleParam 0.73
> BoxSize 100
> Hubble 100
> OutputListOn 1
> TimeBetSnapshot 0
> TimeOfFirstSnapshot 0
> TimeBetStatistics 0.01
> NumFilesPerSnapshot 1
> MaxFilesWithConcurrentIO 1
> ErrTolIntAccuracy 0.012
> CourantFac 0.3
> MaxSizeTimestep 0.005
> MinSizeTimestep 0
> TypeOfOpeningCriterion 1
> ErrTolTheta 0.7
> ErrTolThetaMax 1
> ErrTolForceAcc 0.0025
> TopNodeFactor 5
> ActivePartFracForNewDomainDecomp 0.01
> DesNumNgb 64
> MaxNumNgbDeviation 1
> UnitLength_in_cm 3.08568e+24
> UnitMass_in_g 1.989e+43
> UnitVelocity_in_cm_per_s 100000
> GravityConstantInternal 0
> SofteningComovingClass0 0.001
> SofteningComovingClass1 0.007
> SofteningComovingClass2 0.025
> SofteningComovingClass3 0.081
> SofteningMaxPhysClass0 0.0005
> SofteningMaxPhysClass1 0.007
> SofteningMaxPhysClass2 0.025
> SofteningMaxPhysClass3 0.081
> SofteningClassOfPartType0 0
> SofteningClassOfPartType1 0
> SofteningClassOfPartType2 1
> SofteningClassOfPartType3 2
> SofteningClassOfPartType4 3
> SofteningClassOfPartType5 3
> DesLinkNgb 20
> ArtBulkViscConst 1
> MinEgySpec 0
> InitGasTemp 0
>
> MALLOC: Allocation of shared memory took 0.000141637 sec
>
> found 16 times in output-list.
> BEGRUN: Hubble (internal units) = 100
> BEGRUN: h = 0.73
> BEGRUN: G (internal units) = 43.0187
> BEGRUN: UnitMass_in_g = 1.989e+43
> BEGRUN: UnitLenth_in_cm = 3.08568e+24
> BEGRUN: UnitTime_in_s = 3.08568e+19
> BEGRUN: UnitVelocity_in_cm_per_s = 100000
> BEGRUN: UnitDensity_in_cgs = 6.76991e-31
> BEGRUN: UnitEnergy_in_cgs = 1.989e+53
>
>
> READIC: filenr=0, './ics_gadget.dat' contains:
> READIC: Type 0 (gas): 0 (tot= 0) masstab= 0
> READIC: Type 1: 17576 (tot= 17576) masstab= 3.65236e-09
> READIC: Type 2: 0 (tot= 0) masstab= 0
> READIC: Type 3: 0 (tot= 0) masstab= 0
> READIC: Type 4: 0 (tot= 0) masstab= 0
> READIC: Type 5: 54196 (tot= 54196) masstab= 0
>
> READIC: Reading file `./ics_gadget.dat' on task=0 and distribute it to 0 to 5.
> READIC: reading block 0 (Coordinates)...
> READIC: reading block 1 (Velocities)...
> READIC: reading block 2 (ParticleIDs)...
> READIC: reading block 3 (Masses)...
> READIC: reading done. Took 0.00612891 sec, total size 2.12379 MB, corresponds to effective I/O rate of 346.521 MB/sec
>
> READIC: Total number of particles : 71772
>
> INIT: Testing ID uniqueness...
> INIT: success. took=0.00309694 sec
>
> DOMAIN: Begin domain decomposition (sync-point 0).
> DOMAIN: Sum=2 TotalCost=2 NumTimeBinsToBeBalanced=1 MultipleDomains=2
> DOMAIN: Increasing TopNodeAllocFactor=0.08 new value=0.104
> DOMAIN: Increasing TopNodeAllocFactor=0.104 new value=0.1352
> DOMAIN: Increasing TopNodeAllocFactor=0.1352 new value=0.17576
> DOMAIN: Increasing TopNodeAllocFactor=0.17576 new value=0.228488
> DOMAIN: Increasing TopNodeAllocFactor=0.228488 new value=0.297034
> DOMAIN: Increasing TopNodeAllocFactor=0.297034 new value=0.386145
> DOMAIN: Increasing TopNodeAllocFactor=0.386145 new value=0.501988
> DOMAIN: Increasing TopNodeAllocFactor=0.501988 new value=0.652585
> DOMAIN: Increasing TopNodeAllocFactor=0.652585 new value=0.84836
> DOMAIN: Increasing TopNodeAllocFactor=0.84836 new value=1.10287
> DOMAIN: Increasing TopNodeAllocFactor=1.10287 new value=1.43373
> DOMAIN: Increasing TopNodeAllocFactor=1.43373 new value=1.86385
> DOMAIN: Increasing TopNodeAllocFactor=1.86385 new value=2.423
> DOMAIN: Increasing TopNodeAllocFactor=2.423 new value=3.1499
> DOMAIN: NTopleaves=246, determination of top-level tree involved 14 iterations and took 0.00647044 sec
> DOMAIN: we are going to try at most 149 different settings for combining the domains on tasks=6, nnodes=1
> DOMAIN: total_cost=2 total_load=1
> DOMAIN: best solution found after 1 iterations by task=5 for nextra=0, reaching maximum imbalance of 1.02449|1.03052
> DOMAIN: combining multiple-domains took 0.000829803 sec
> DOMAIN: exchange of 71772 particles
> DOMAIN: particle exchange done. (took 0.00680363 sec)
> DOMAIN: domain decomposition done. (took in total 0.0150899 sec)
> PEANO: Begin Peano-Hilbert order...
> PEANO: done, took 0.00189733 sec.
> INIT: AvgType1Mass = 3.65236e-09 (min=3.65236e-09 max=3.65236e-09) Ndm1tot=17576
> INIT: For this AvgType1Mass, the mean particle spacing is -nan and the assigned softening is 0.001
>
> SNAPSHOT: Setting next time for snapshot file to Time_next= 0.0288901 (DumpFlag=1)
>
>
>
> Sync-Point 0, Time: 0.0078125, Redshift: 127, Systemstep: 0, Dloga: 0, Nsync-grv: 71772, Nsync-hyd: 0
> DOMAIN: Begin domain decomposition (sync-point 0).
> DOMAIN: Sum=2 TotalCost=2 NumTimeBinsToBeBalanced=1 MultipleDomains=2
> DOMAIN: NTopleaves=246, determination of top-level tree involved 14 iterations and took 0.00247401 sec
> DOMAIN: we are going to try at most 149 different settings for combining the domains on tasks=6, nnodes=1
> DOMAIN: total_cost=2 total_load=1
> DOMAIN: best solution found after 1 iterations by task=5 for nextra=0, reaching maximum imbalance of 1.02449|1.03052
> DOMAIN: combining multiple-domains took 0.00157532 sec
> DOMAIN: exchange of 71772 particles
> DOMAIN: particle exchange done. (took 0.00363552 sec)
> DOMAIN: domain decomposition done. (took in total 0.00819422 sec)
> PEANO: Begin Peano-Hilbert order...
> PEANO: done, took 0.000835626 sec.
> ACCEL: Start tree gravity force computation... (71772 particles)
> PM-PERIODIC: Starting periodic PM calculation. (Rcut=0.854492) presently allocated=1.83447 MB
> -------------------------- Allocated Memory Blocks---- ( Step 0 )------------------
> Task Nr F Variable MBytes Cumulative Function|File|Linenumber
> ------------------------------------------------------------------------------------------
> 4 0 0 GetGhostRankForSimulCommRank 0.0001 0.0001 mymalloc_init()|src/data/mymalloc.cc|137
> 4 1 0 GetShmRankForSimulCommRank 0.0001 0.0001 mymalloc_init()|src/data/mymalloc.cc|138
> 4 2 0 GetNodeIDForSimulCommRank 0.0001 0.0002 mymalloc_init()|src/data/mymalloc.cc|139
> 4 3 0 SharedMemBaseAddr 0.0001 0.0002 mymalloc_init()|src/data/mymalloc.cc|153
> 4 4 1 slab_to_task 0.0039 0.0042 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|45
> 4 5 1 slabs_x_per_task 0.0001 0.0042 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|60
> 4 6 1 first_slab_x_of_task 0.0001 0.0043 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|63
> 4 7 1 slabs_y_per_task 0.0001 0.0043 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|66
> 4 8 1 first_slab_y_of_task 0.0001 0.0044 my_slab_based_fft_init()|src/pm/pm_mpi_fft.cc|69
> 4 9 1 P 1.5972 1.6016 allocate_memory()|src/io/../fof/../data/simparticles.h|273
> 4 10 1 SphP 0.0001 1.6017 allocate_memory()|src/io/../fof/../data/simparticles.h|274
> 4 11 1 NextActiveParticleHydro 0.0001 1.6017 timebins_allocate()|src/time_integration/timestep.cc|463
> 4 12 1 NextInTimeBinHydro 0.0001 1.6018 timebins_allocate()|src/time_integration/timestep.cc|466
> 4 13 1 PrevInTimeBinHydro 0.0001 1.6019 timebins_allocate()|src/time_integration/timestep.cc|469
> 4 14 1 NextActiveParticleGravity 0.0761 1.6780 timebins_allocate()|src/time_integration/timestep.cc|463
> 4 15 1 NextInTimeBinGravity 0.0761 1.7541 timebins_allocate()|src/time_integration/timestep.cc|466
> 4 16 1 PrevInTimeBinGravity 0.0761 1.8302 timebins_allocate()|src/time_integration/timestep.cc|469
> 4 17 1 FirstTopleafOfTask 0.0001 1.8303 domain_allocate()|src/domain/domain.cc|163
> 4 18 1 NumTopleafOfTask 0.0001 1.8303 domain_allocate()|src/domain/domain.cc|164
> 4 19 1 TopNodes 0.0022 1.8325 domain_allocate()|src/domain/domain.cc|165
> 4 20 1 TaskOfLeaf 0.0010 1.8335 domain_allocate()|src/domain/domain.cc|166
> 4 21 1 ListOfTopleaves 0.0010 1.8345 domain_decomposition()|src/domain/domain.cc|118
> 4 22 0 part 1.0726 2.9070 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|184
> 4 23 1 localfield_globalindex 0.0001 2.9071 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|272
> 4 24 1 localfield_data 0.0001 2.9072 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|274
> 4 25 1 localfield_first 0.0001 2.9072 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|275
> 4 26 1 localfield_sendcount 0.0001 2.9073 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|276
> 4 27 1 localfield_offset 0.0001 2.9073 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|277
> 4 28 1 localfield_recvcount 0.0001 2.9074 pmforce_zoom_optimized_prepare_density()|src/pm/pm_periodic.cc|278
> ------------------------------------------------------------------------------------------
> Code termination on task=4, function mymalloc_movable_fullinfo(), file src/data/mymalloc.cc, line 326:
> Not enough memory in mymalloc_fullinfo() to allocate 685.336 MB for variable 'rhogrid' at pmforce_zoom_optimized_prepare_density()/src/pm/pm_periodic.cc/line 378 (FreeBytes=97.0926 MB).
>
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 4
> pc_at_ubuntu:~/Desktop/SPH/gadget4-master/IC_from_MUSIC$
>
>
> -----------------------------------------------------------
>
> If you wish to unsubscribe from this mailing, send mail to
> minimalist_at_MPA-Garching.MPG.de with a subject of: unsubscribe gadget-list
> A web-archive of this mailing list is available here:
> http://www.mpa-garching.mpg.de/gadget/gadget-list
Received on 2021-02-12 10:29:30