Segmentation fault using 2LPTic

From: Tom Kirsh <tk27182_at_gmail.com>
Date: Fri, 11 May 2018 11:45:00 -0500

 Hello,

I'm currently trying to create the initial conditions file for a 1024^3
particle simulation. I'm using the NCSA XE nodes to run this simulation,
with 4096 nodes and 16 ppn. Previously, I had no trouble running the IC
file for a 512^3 simulation and 256^3 simulation, however, when I try to
create the IC file for the 1024^3 particle simulation, I get the following
output:

----------------------------------------
Begin Torque Prologue on nid25338
at Thu May 10 21:17:35 CDT 2018
Job Id: 8599200.bw
Username: kirsh
Group: ILL_baoy
Job name: ics1024
Requested resources: neednodes=4096:ppn=16:xe,
nodes=4096:ppn=16:xe,walltime=0
                                                                    0:30:00
Queue: normal
Account: baoy
End Torque Prologue: 0.132 elapsed
----------------------------------------




Task=0 Local_nx=1024

allocated 8208 Mbyte on Task 0

reading Lagrangian glass file...
reading 'dummy_glass.dat' with 4096 particles

Nglass= 4096

1073741824 particles on task=0 (slabs=1024)

Total number of particles = 1073741824


start computing displacement fields...
vel_prefac= 2.13829, vel_prefac2= 4.27658, hubble_a=13.6921 fom=0.999977

allocated 72.0469 Mbyte on Task 0

starting axes=3...
Application 66871546 exit codes: 139
Application 66871546 resources: utime ~34s, stime ~26s, Rss ~33714504,
inblocks
                                             ~8849, outblocks ~21293

------------------------------------------------------------
------------------

The ics.param file is the same as the 512^3 IC file, where the only
difference is the number of particles and boxsize. I've included the
parameterfile below

Nmesh 1024 % This is the size of the FFT grid used to
                           % compute the displacement field. One
                           % should have Nmesh >= Nsample.

Nsample 1024 % sets the maximum k that the code uses,
                           % i.e. this effectively determines the
                           % Nyquist frequency that the code assumes,
                           % k_Nyquist = 2*PI/Box * Nsample/2
                           % Normally, one chooses Nsample such that
                           % Ntot = Nsample^3, where Ntot is the
                           % total number of particles


Box 1024000.0 % Periodic box size of simulation

FileBase ics1024 % Base-filename of output files
OutputDir ./ICs/ % Directory for output

GlassFile dummy_glass.dat % File with unperturbed glass or
                                  % Cartesian grid

GlassTileFac 64 % Number of times the glass file is
                                  % tiled in each dimension (must be
                                  % an integer)


Omega 0.272 % Total matter density (at z=0)
OmegaLambda 0.728 % Cosmological constant (at z=0)
OmegaBaryon 0.045 % Baryon density (at z=0)
OmegaDM_2ndSpecies 0.0 % Omega for a second dark matter species (at
z=0)
HubbleParam 0.7 % Hubble paramater (may be used for power spec
parameterization)

Redshift 40.0 % Starting redshift

Sigma8 0.81 % power spectrum normalization

SphereMode 1 % if "1" only modes with |k| < k_Nyquist are
                           % used (i.e. a sphere in k-space), otherwise
modes with
                           % |k_x|,|k_y|,|k_z| < k_Nyquist are used
                           % (i.e. a cube in k-space)


WhichSpectrum 0 % "1" selects Eisenstein & Hu spectrum,
                           % "2" selects a tabulated power spectrum in
                           % the file 'FileWithInputSpectrum'
                           % otherwise, Efstathiou parametrization is used


FileWithInputSpectrum /u/sciteam/kirsh/2LPTic/matter2_psz40.txt %
filename of tabulated input
                                            % spectrum (if used)
InputSpectrum_UnitLength_in_cm 3.085678e24 % defines length unit of
tabulated
                                            % input spectrum in cm/h.
                                            % Note: This can be chosen
different from UnitLength_in_cm


ShapeGamma 0.21 % only needed for Efstathiou power spectrum
PrimordialIndex 0.967 % may be used to tilt the primordial index,
                           % primordial spectrum is k^PrimordialIndex


Seed 123456 % seed for IC-generator


NumFilesWrittenInParallel 1 % limits the number of files that are
                             % written in parallel when outputting


UnitLength_in_cm 3.085678e21 % defines length unit of output (in
cm/h)
UnitMass_in_g 1.989e43 % defines mass unit of output (in
g/cm)
UnitVelocity_in_cm_per_s 1e5 % defines velocity unit of output
(in cm/sec)

WDM_On 0
WDM_Vtherm_On 0
WDM_PartMass_in_kev 10.0
------------------------------------------------------------
------------------------------------

Am I missing something in my parameterfile or do I just need to request
more nodes?

Best,
Tom Kirsh
Received on 2018-05-11 18:45:42