Dear all,
I understand that this simulation (2048^3 particles) is large to run, is it
normal that the SUBFIND takes about 4ks to complete? It seems that there
are only two tasks to run subfind. Are there some options in SUBFIND that I
did set not correctly?
```
SUBFIND: We now execute a parallel version of SUBFIND.
SUBFIND: Previous subhalo catalogue had approximately a size 2.76992e+09,
and the summed squared subhalo size was 1.16758e+14
SUBFIND: Number of FOF halos treated with collective SubFind algorithm = 2
SUBFIND: Number of processors used in different partitions for the
collective SubFind code = 4
SUBFIND: (The adopted size-limit for the collective algorithm was 5965232
particles, for threshold size factor 0.6)
SUBFIND: The other 8913146 FOF halos are treated in parallel with serial
code
SUBFIND: subfind_distribute_groups() took 0.044987 sec
SUBFIND: particle balance=1.23335
SUBFIND: subfind_exchange() took 20.5998 sec
SUBFIND: particle balance for processing=1
SUBFIND: root-task=2: Collectively doing halo 1 of length 6001658 on 2
processors.
SUBFIND: root-task=0: Collectively doing halo 0 of length 6318694 on 2
processors.
SUBFIND: subdomain decomposition took 5.28067 sec
SUBFIND: serial subfind subdomain decomposition took 4.50761 sec
SUBFIND: root-task=2: total number of subhalo coll_candidates=2858
SUBFIND: root-task=2: number of subhalo candidates small enough to be done
with one cpu: 2858. (Largest size 234093)
SUBFIND: root-task=0: total number of subhalo coll_candidates=3148
SUBFIND: root-task=0: number of subhalo candidates small enough to be done
with one cpu: 3148. (Largest size 1116065)
SUBFIND: root-task=2: bringing the independent ones back took 1.43275 sec
SUBFIND: root-task=2: the collective unbinding of remaining halos took *728.107
sec*
SUBFIND: root-task=2: found 2312 bound substructures in FoF group of length
6001658
SUBFIND: root-task=2: Collective processing of halo 1 took 1212.43
SUBFIND: root-task=0: bringing the independent ones back took 1.69057 sec
SUBFIND: root-task=0: the collective unbinding of remaining halos took *893.829
sec*
SUBFIND: root-task=0: found 2642 bound substructures in FoF group of length
6318694
SUBFIND: subfind_hbt_single_group() processing for Ngroups=1 took *1459.14
sec*
SUBFIND: root-task=0: Collective processing of halo 0 took 1473.2
SUBFIND: Processing overall took (total time=1482.26 sec)
ALLOCATE: Changing to MaxPart = 17425843
ALLOCATE: Changing to MaxPartSph = 0
SUBFIND: subfind_exchange() (for return to original CPU) took 26.0112 sec
TREE: Full tree construction for all particles. (presently allocated=3948.2
MB)
GRAVTREE: Tree construction done. took 71.5354 sec <numnodes>=1.90281e+06
NTopnodes=70545 NTopleaves=61727 tree-build-scalability=0.962969
SUBFIND: SO iteration 1: need to repeat for 7848033 halo centers.
(took 0.309375 sec)
SUBFIND: SO iteration 2: need to repeat for 7848033 halo centers.
(took 0.2519 sec)
SUBFIND: SO iteration 3: need to repeat for 7848033 halo centers.
(took 0.268218 sec)
SUBFIND: SO iteration 4: need to repeat for 7848033 halo centers.
(took 0.266999 sec)
SUBFIND: SO iteration 5: need to repeat for 7848033 halo centers.
(took 0.288053 sec)
SUBFIND: SO iteration 6: need to repeat for 7848033 halo centers.
(took 0.266823 sec)
SUBFIND: SO iteration 7: need to repeat for 7848033 halo centers.
(took 0.264547 sec)
SUBFIND: SO iteration 8: need to repeat for 7848033 halo centers.
(took 0.263144 sec)
SUBFIND: SO iteration 9: need to repeat for 7848033 halo centers.
(took 0.256196 sec)
SUBFIND: SO iteration 10: need to repeat for 7848033 halo centers.
(took 0.264536 sec)
SUBFIND: SO iteration 11: need to repeat for 7848033 halo centers.
(took 0.257483 sec)
SUBFIND: SO iteration 12: need to repeat for 7848033 halo centers.
(took 0.25124 sec)
SUBFIND: SO iteration 13: need to repeat for 7848033 halo centers.
(took 0.261861 sec)
SUBFIND: SO iteration 14: need to repeat for 7848033 halo centers.
(took 0.257919 sec)
SUBFIND: SO iteration 15: need to repeat for 7848030 halo centers.
(took 0.268111 sec)
SUBFIND: SO iteration 16: need to repeat for 38913 halo centers.
(took 0.259465 sec)
SUBFIND: SO iteration 17: need to repeat for 37 halo centers.
(took 0.0242081 sec)
SUBFIND: SO iteration 1: need to repeat for 7848033 halo centers.
(took 0.264944 sec)
SUBFIND: SO iteration 2: need to repeat for 7848033 halo centers.
(took 0.237437 sec)
SUBFIND: SO iteration 3: need to repeat for 7848033 halo centers.
(took 0.257997 sec)
SUBFIND: SO iteration 4: need to repeat for 7848033 halo centers.
(took 0.263542 sec)
SUBFIND: SO iteration 5: need to repeat for 7848033 halo centers.
(took 0.268112 sec)
SUBFIND: SO iteration 6: need to repeat for 7848033 halo centers.
(took 0.254724 sec)
SUBFIND: SO iteration 7: need to repeat for 7848033 halo centers.
(took 0.268252 sec)
SUBFIND: SO iteration 8: need to repeat for 7848033 halo centers.
(took 0.25666 sec)
SUBFIND: SO iteration 9: need to repeat for 7848033 halo centers.
(took 0.272972 sec)
SUBFIND: SO iteration 10: need to repeat for 7848033 halo centers.
(took 0.251183 sec)
SUBFIND: SO iteration 11: need to repeat for 7848033 halo centers.
(took 0.265267 sec)
SUBFIND: SO iteration 12: need to repeat for 7848033 halo centers.
(took 0.257801 sec)
SUBFIND: SO iteration 13: need to repeat for 7848033 halo centers.
(took 0.269823 sec)
SUBFIND: SO iteration 14: need to repeat for 7848033 halo centers.
(took 0.265494 sec)
SUBFIND: SO iteration 15: need to repeat for 7848030 halo centers.
(took 0.247782 sec)
SUBFIND: SO iteration 16: need to repeat for 36428 halo centers.
(took 0.259644 sec)
SUBFIND: SO iteration 17: need to repeat for 35 halo centers.
(took 0.019228 sec)
SUBFIND: SO iteration 1: need to repeat for 7848033 halo centers.
(took 0.265704 sec)
SUBFIND: SO iteration 2: need to repeat for 7848033 halo centers.
(took 0.229607 sec)
SUBFIND: SO iteration 3: need to repeat for 7848033 halo centers.
(took 0.254473 sec)
SUBFIND: SO iteration 4: need to repeat for 7848033 halo centers.
(took 0.25696 sec)
SUBFIND: SO iteration 5: need to repeat for 7848033 halo centers.
(took 0.26164 sec)
SUBFIND: SO iteration 6: need to repeat for 7848033 halo centers.
(took 0.25506 sec)
SUBFIND: SO iteration 7: need to repeat for 7848033 halo centers.
(took 0.240955 sec)
SUBFIND: SO iteration 8: need to repeat for 7848033 halo centers.
(took 0.260188 sec)
SUBFIND: SO iteration 9: need to repeat for 7848033 halo centers.
(took 0.248877 sec)
SUBFIND: SO iteration 10: need to repeat for 7848033 halo centers.
(took 0.256545 sec)
SUBFIND: SO iteration 11: need to repeat for 7848033 halo centers.
(took 0.25118 sec)
SUBFIND: SO iteration 12: need to repeat for 7848033 halo centers.
(took 0.25854 sec)
SUBFIND: SO iteration 13: need to repeat for 7848033 halo centers.
(took 0.246834 sec)
SUBFIND: SO iteration 14: need to repeat for 7848033 halo centers.
(took 0.27007 sec)
SUBFIND: SO iteration 15: need to repeat for 7848033 halo centers.
(took 0.258597 sec)
SUBFIND: SO iteration 16: need to repeat for 122350 halo centers.
(took 0.248988 sec)
SUBFIND: SO iteration 17: need to repeat for 181 halo centers.
(took 0.0358 sec)
SUBFIND: SO iteration 1: need to repeat for 7848033 halo centers.
(took 0.269485 sec)
SUBFIND: SO iteration 2: need to repeat for 7848033 halo centers.
(took 0.228188 sec)
SUBFIND: SO iteration 3: need to repeat for 7848033 halo centers.
(took 0.234185 sec)
SUBFIND: SO iteration 4: need to repeat for 7848033 halo centers.
(took 0.229443 sec)
SUBFIND: SO iteration 5: need to repeat for 7848033 halo centers.
(took 0.224041 sec)
SUBFIND: SO iteration 6: need to repeat for 7848033 halo centers.
(took 0.232641 sec)
SUBFIND: SO iteration 7: need to repeat for 7848033 halo centers.
(took 0.230443 sec)
SUBFIND: SO iteration 8: need to repeat for 7848033 halo centers.
(took 0.219413 sec)
SUBFIND: SO iteration 9: need to repeat for 7848033 halo centers.
(took 0.220803 sec)
SUBFIND: SO iteration 10: need to repeat for 7848033 halo centers.
(took 0.237108 sec)
SUBFIND: SO iteration 11: need to repeat for 7848033 halo centers.
(took 0.232947 sec)
SUBFIND: SO iteration 12: need to repeat for 7848033 halo centers.
(took 0.240728 sec)
SUBFIND: SO iteration 13: need to repeat for 7848033 halo centers.
(took 0.228163 sec)
SUBFIND: SO iteration 14: need to repeat for 7848033 halo centers.
(took 0.221182 sec)
SUBFIND: SO iteration 15: need to repeat for 7848033 halo centers.
(took 0.224345 sec)
SUBFIND: SO iteration 16: need to repeat for 6272558 halo centers.
(took 0.226659 sec)
SUBFIND: SO iteration 17: need to repeat for 46051 halo centers.
(took 0.188434 sec)
SUBFIND: SO iteration 18: need to repeat for 52 halo centers.
(took 0.01652 sec)
SUBFIND: determining spherical overdensity masses took 16.475 sec
SUBFIND: assembled and ordered groups and subhalos (took 5.52871 sec)
FOF/SUBFIND: writing group catalogue file:
'./snaps//groups_026/fof_subhalo_tab_026.0' (file 1 of 32)
FOF/SUBFIND: writing group catalogue block 0 (GroupLen)...
FOF/SUBFIND: writing group catalogue block 1 (GroupMass)...
FOF/SUBFIND: writing group catalogue block 2 (GroupPos)...
FOF/SUBFIND: writing group catalogue block 3 (GroupVel)...
FOF/SUBFIND: writing group catalogue block 4 (GroupLenType)...
FOF/SUBFIND: writing group catalogue block 5 (GroupOffsetType)...
FOF/SUBFIND: writing group catalogue block 6 (GroupMassType)...
FOF/SUBFIND: writing group catalogue block 7 (GroupAscale)...
FOF/SUBFIND: writing group catalogue block 9 (Group_M_Mean200)...
FOF/SUBFIND: writing group catalogue block 10 (Group_R_Mean200)...
FOF/SUBFIND: writing group catalogue block 11 (Group_M_Crit200)...
FOF/SUBFIND: writing group catalogue block 12 (Group_R_Crit200)...
FOF/SUBFIND: writing group catalogue block 13 (Group_M_Crit500)...
FOF/SUBFIND: writing group catalogue block 14 (Group_R_Crit500)...
FOF/SUBFIND: writing group catalogue block 15 (Group_M_TopHat200)...
FOF/SUBFIND: writing group catalogue block 16 (Group_R_TopHat200)...
FOF/SUBFIND: writing group catalogue block 17 (GroupNsubs)...
FOF/SUBFIND: writing group catalogue block 18 (GroupFirstSub)...
FOF/SUBFIND: writing group catalogue block 19 (SubhaloGroupNr)...
FOF/SUBFIND: writing group catalogue block 20 (SubhaloRankInGr)...
FOF/SUBFIND: writing group catalogue block 21 (SubhaloLen)...
FOF/SUBFIND: writing group catalogue block 22 (SubhaloMass)...
FOF/SUBFIND: writing group catalogue block 23 (SubhaloPos)...
FOF/SUBFIND: writing group catalogue block 24 (SubhaloVel)...
FOF/SUBFIND: writing group catalogue block 25 (SubhaloLenType)...
FOF/SUBFIND: writing group catalogue block 26 (SubhaloOffsetType)...
FOF/SUBFIND: writing group catalogue block 27 (SubhaloMassType)...
FOF/SUBFIND: writing group catalogue block 28 (SubhaloCM)...
FOF/SUBFIND: writing group catalogue block 29 (SubhaloSpin)...
FOF/SUBFIND: writing group catalogue block 30 (SubhaloVelDisp)...
FOF/SUBFIND: writing group catalogue block 31 (SubhaloVmax)...
FOF/SUBFIND: writing group catalogue block 32 (SubhaloVmaxRad)...
FOF/SUBFIND: writing group catalogue block 33 (SubhaloHalfmassRad)...
FOF/SUBFIND: writing group catalogue block 34 (SubhaloHalfmassRadType)...
FOF/SUBFIND: writing group catalogue block 35 (SubhaloIDMostbound)...
FOF/SUBFIND: writing group catalogue block 36 (SubhaloParentRank)...
FOF/SUBFIND: Group catalogues saved. took = 8.89541 sec, total size 3525.03
MB, corresponds to effective I/O rate of 396.275 MB/sec
SUBFIND: Subgroup catalogues saved. took = 9.7367 sec
SUBFIND: Finished with SUBFIND. (*total time=4092.76 sec*)
SUBFIND: Total number of subhalos with at least 32 particles: 9849779
SUBFIND: Largest subhalo has 4419279 particles/cells.
SUBFIND: Total number of particles/cells in subhalos: 2897277494
```
Many thanks for any suggestions. Please let me know if you need any more
information.
Best,
Weiguang
-------------------------------------------
https://weiguangcui.github.io/
Received on 2021-08-07 21:42:59