Clipping particel outside of high-resolution region for zoomed-in runs

From: Weiguang Cui <cuiweiguang_at_gmail.com>
Date: Mon, 21 Feb 2022 19:25:26 +0000

Dear all,

Thanks to Volker for pointing out the bug in intel-2018 compiler. Now the
zoomed-in DMO IC runs towards z=0 without a problem. Then, I am trying a
test with the hydro run (only cooling is opened, but not star formation
etc.). It ran fine at the beginning, then, I encounter a problem of
reaching the min time-step:

> Code termination on task=67, function
> timebins_get_bin_and_do_validity_checks(), file
> src/time_integration/timestep.cc, line 509: time-step of integer size 1 not
> allowed

This is overcome by increasing the `MinEgySpec` to 5000. Note that too high
temperature will result another problem of `predicting into the past'.

Anyway, the code runs smoothly down to almost 0, then I encountered another
problem:

> Sync-Point 61113, Time: 0.870968, Redshift: 0.148148, Systemstep:
> 9.17242e-06, Dloga: 1.05314e-05, Nsync-grv: 0, Nsync-hyd:
> 11508
> DOMAIN: Begin domain decomposition (sync-point 61113).
> DOMAIN: Shrink-wrap region size for RANDOMIZE_DOMAINCENTER_TYPES is 25000.1
> Code termination on task=0, function domain_find_type_extension(), file
> src/domain/domain_box.cc, line 286: inappropriately big region selection
> for RANDOMIZE_DOMAINCENTER_TYPES


I think I can still play with the `MinEgySpec` parameter to see if the code
runs down to 0. But I think the major issue in this is these high-res gas
particles wonder into a vast empty space. It was be able to be clipped
before with a configure option in Gadget3, but I can not find it in the
current document. Any suggestions or comments on this?

Here is the latest Config options with parameters for your reference.
Please let me know if you think there are something wrong in the setting of
this simulation. Thanks.

This is Gadget, version 4.0.
> Git commit 678a54ee473ba558afb43705cd0674cf527881af, Mon Nov 29 09:23:55
> 2021 +0000


> Code was compiled with the following compiler and flags:
> mpicxx -std=c++11 -O3 -g -I/hdf5/build/include -I/gsl/build/include
> -I/fftw3/build/include -Ibuild -Isrc


> Code are compiled with the following settings for your reference:

    ALLOW_HDF5_COMPRESSION
> ASMTH=1.2
> COOLING
> DOUBLEPRECISION=1
> DOUBLEPRECISION_FFTW
> FMM
> FOF
> FOF_GROUP_MIN_LEN=32
> FOF_LINKLENGTH=0.2
> FOF_PRIMARY_LINK_TYPES=2
> FOF_SECONDARY_LINK_TYPES=1+4+16+32
> GADGET2_HEADER
> IDS_32BIT
> IMPROVED_VELOCITY_GRADIENTS
> MERGERTREE
> MULTIPOLE_ORDER=3
> NTAB=256
> NTYPES=6
> PERIODIC
> PLACEHIGHRESREGION=1+2
> PMGRID=512
> PM_ZOOM_OPTIMIZED
> POSITIONS_IN_64BIT
> PRESSURE_ENTROPY_SPH
> RANDOMIZE_DOMAINCENTER
> RCUT=4.5
> SELFGRAVITY
> SUBFIND
> SUBFIND_HBT
> SUBFIND_ORPHAN_TREATMENT
> TREE_NUM_BEFORE_NODESPLIT=2
> WENDLAND_BIAS_CORRECTION
> WENDLAND_C6_KERNEL


> Running on 120 MPI tasks.


> BEGRUN: Size of particle structure 144 [bytes]
> BEGRUN: Size of sph particle structure 352 [bytes]
> BEGRUN: Size of gravity tree node 152 [bytes]
> BEGRUN: Size of neighbour tree node 192 [bytes]
> BEGRUN: Size of subfind auxiliary data 80 [bytes]


>
>
> -------------------------------------------------------------------------------------------------------------------------
> AvailMem: Largest = 126747.35 Mb (on task= 15), Smallest =
> 126699.70 Mb (on task= 0), Average = 126724.46 Mb
> Total Mem: Largest = 128743.44 Mb (on task= 15), Smallest =
> 128743.12 Mb (on task= 75), Average = 128743.24 Mb
> Committed_AS: Largest = 2043.43 Mb (on task= 0), Smallest =
> 1996.09 Mb (on task= 15), Average = 2018.78 Mb
> SwapTotal: Largest = 8096.00 Mb (on task= 0), Smallest =
> 8096.00 Mb (on task= 0), Average = 8096.00 Mb
> SwapFree: Largest = 8096.00 Mb (on task= 0), Smallest =
> 8096.00 Mb (on task= 0), Average = 8096.00 Mb
> AllocMem: Largest = 2043.43 Mb (on task= 0), Smallest =
> 1996.09 Mb (on task= 15), Average = 2018.78 Mb
> avail /dev/shm: Largest = 64305.91 Mb (on task= 15), Smallest =
> 64305.75 Mb (on task= 75), Average = 64305.81 Mb
>
> -------------------------------------------------------------------------------------------------------------------------
> Task=0 has the maximum commited memory and is host:
> m6187.pri.cosma7.alces.network
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Obtaining parameters from file 'parameter.txt':

        InitCondFile halo_6312
> OutputDir ./snaps/
> SnapshotFileBase
> snap_g4_halo_6312-hydro
> OutputListFilename outputlist.txt
> ICFormat 1
> SnapFormat 3
> TimeLimitCPU 864000
> CpuTimeBetRestartFile 7100
> MaxMemSize 2500
> TimeBegin 0.004
> TimeMax 1
> ComovingIntegrationOn 1
> Omega0 0.3
> OmegaLambda 0.7
> OmegaBaryon 0.048
> HubbleParam 0.68
> Hubble 0.1
> BoxSize 200000
> OutputListOn 1
> TimeBetSnapshot 0
> TimeOfFirstSnapshot 0
> TimeBetStatistics 0.01
> NumFilesPerSnapshot 1
> MaxFilesWithConcurrentIO 64
> ErrTolIntAccuracy 0.01
> CourantFac 0.2
> MaxSizeTimestep 0.001
> MinSizeTimestep 0
> TypeOfOpeningCriterion 1
> ErrTolTheta 0.75
> ErrTolThetaMax 1
> ErrTolForceAcc 0.002
> TopNodeFactor 3
> ActivePartFracForNewDomainDecomp 0.01
> ActivePartFracForPMinsteadOfEwald 0.05
> DesNumNgb 64
> MaxNumNgbDeviation 4
> UnitLength_in_cm 3.08568e+21
> UnitMass_in_g 1.989e+43
> UnitVelocity_in_cm_per_s 100000
> GravityConstantInternal 0
> SofteningComovingClass0 2
> SofteningMaxPhysClass0 2
> SofteningComovingClass1 2
> SofteningMaxPhysClass1 2
> SofteningComovingClass2 15

     SofteningMaxPhysClass2 15
> SofteningComovingClass3 0
> SofteningMaxPhysClass3 0
> SofteningComovingClass4 0
> SofteningMaxPhysClass5 0
> SofteningComovingClass5 0
> SofteningMaxPhysClass4 0
> SofteningClassOfPartType0 0
> SofteningClassOfPartType1 1
> SofteningClassOfPartType2 2
> SofteningClassOfPartType3 0
> SofteningClassOfPartType4 0
> SofteningClassOfPartType5 0
> ArtBulkViscConst 1
> MinEgySpec 5000
> InitGasTemp 682.5
> TreecoolFile TREECOOL
> DesLinkNgb 32


MALLOC: Allocation of shared memory took 0.000548531 sec


found 152 times in output-list.
> BEGRUN: Hubble (internal units) = 0.1
> BEGRUN: h = 0.68
> BEGRUN: G (internal units) = 43018.7
> BEGRUN: UnitMass_in_g = 1.989e+43
> BEGRUN: UnitLenth_in_cm = 3.08568e+21
> BEGRUN: UnitTime_in_s = 3.08568e+16
> BEGRUN: UnitVelocity_in_cm_per_s = 100000
> BEGRUN: UnitDensity_in_cgs = 6.76991e-22
> BEGRUN: UnitEnergy_in_cgs = 1.989e+53


COOLING: read ionization table with 171 entries in file `TREECOOL'.
> COOLING: using 170 ionization table entries from file `TREECOOL'.



Best,
Weiguang

---------------------------------------------
https://weiguangcui. github.io/ <https://weiguangcui.github.io/>
Received on 2022-02-21 20:26:18

This archive was generated by hypermail 2.3.0 : 2022-09-01 14:03:43 CEST