- Mail actions: [ respond to this message ] [ mail a new topic ]
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]

From: Tom Kirsh <tk27182_at_gmail.com>

Date: Fri, 11 May 2018 11:45:00 -0500

Hello,

I'm currently trying to create the initial conditions file for a 1024^3

particle simulation. I'm using the NCSA XE nodes to run this simulation,

with 4096 nodes and 16 ppn. Previously, I had no trouble running the IC

file for a 512^3 simulation and 256^3 simulation, however, when I try to

create the IC file for the 1024^3 particle simulation, I get the following

output:

----------------------------------------

Begin Torque Prologue on nid25338

at Thu May 10 21:17:35 CDT 2018

Job Id: 8599200.bw

Username: kirsh

Group: ILL_baoy

Job name: ics1024

Requested resources: neednodes=4096:ppn=16:xe,

nodes=4096:ppn=16:xe,walltime=0

0:30:00

Queue: normal

Account: baoy

End Torque Prologue: 0.132 elapsed

----------------------------------------

Task=0 Local_nx=1024

allocated 8208 Mbyte on Task 0

reading Lagrangian glass file...

reading 'dummy_glass.dat' with 4096 particles

Nglass= 4096

1073741824 particles on task=0 (slabs=1024)

Total number of particles = 1073741824

start computing displacement fields...

vel_prefac= 2.13829, vel_prefac2= 4.27658, hubble_a=13.6921 fom=0.999977

allocated 72.0469 Mbyte on Task 0

starting axes=3...

Application 66871546 exit codes: 139

Application 66871546 resources: utime ~34s, stime ~26s, Rss ~33714504,

inblocks

~8849, outblocks ~21293

------------------------------------------------------------

------------------

The ics.param file is the same as the 512^3 IC file, where the only

difference is the number of particles and boxsize. I've included the

parameterfile below

Nmesh 1024 % This is the size of the FFT grid used to

% compute the displacement field. One

% should have Nmesh >= Nsample.

Nsample 1024 % sets the maximum k that the code uses,

% i.e. this effectively determines the

% Nyquist frequency that the code assumes,

% k_Nyquist = 2*PI/Box * Nsample/2

% Normally, one chooses Nsample such that

% Ntot = Nsample^3, where Ntot is the

% total number of particles

Box 1024000.0 % Periodic box size of simulation

FileBase ics1024 % Base-filename of output files

OutputDir ./ICs/ % Directory for output

GlassFile dummy_glass.dat % File with unperturbed glass or

% Cartesian grid

GlassTileFac 64 % Number of times the glass file is

% tiled in each dimension (must be

% an integer)

Omega 0.272 % Total matter density (at z=0)

OmegaLambda 0.728 % Cosmological constant (at z=0)

OmegaBaryon 0.045 % Baryon density (at z=0)

OmegaDM_2ndSpecies 0.0 % Omega for a second dark matter species (at

z=0)

HubbleParam 0.7 % Hubble paramater (may be used for power spec

parameterization)

Redshift 40.0 % Starting redshift

Sigma8 0.81 % power spectrum normalization

SphereMode 1 % if "1" only modes with |k| < k_Nyquist are

% used (i.e. a sphere in k-space), otherwise

modes with

% |k_x|,|k_y|,|k_z| < k_Nyquist are used

% (i.e. a cube in k-space)

WhichSpectrum 0 % "1" selects Eisenstein & Hu spectrum,

% "2" selects a tabulated power spectrum in

% the file 'FileWithInputSpectrum'

% otherwise, Efstathiou parametrization is used

FileWithInputSpectrum /u/sciteam/kirsh/2LPTic/matter2_psz40.txt %

filename of tabulated input

% spectrum (if used)

InputSpectrum_UnitLength_in_cm 3.085678e24 % defines length unit of

tabulated

% input spectrum in cm/h.

% Note: This can be chosen

different from UnitLength_in_cm

ShapeGamma 0.21 % only needed for Efstathiou power spectrum

PrimordialIndex 0.967 % may be used to tilt the primordial index,

% primordial spectrum is k^PrimordialIndex

Seed 123456 % seed for IC-generator

NumFilesWrittenInParallel 1 % limits the number of files that are

% written in parallel when outputting

UnitLength_in_cm 3.085678e21 % defines length unit of output (in

cm/h)

UnitMass_in_g 1.989e43 % defines mass unit of output (in

g/cm)

UnitVelocity_in_cm_per_s 1e5 % defines velocity unit of output

(in cm/sec)

WDM_On 0

WDM_Vtherm_On 0

WDM_PartMass_in_kev 10.0

------------------------------------------------------------

------------------------------------

Am I missing something in my parameterfile or do I just need to request

more nodes?

Best,

Tom Kirsh

Received on 2018-05-11 18:45:42

Date: Fri, 11 May 2018 11:45:00 -0500

Hello,

I'm currently trying to create the initial conditions file for a 1024^3

particle simulation. I'm using the NCSA XE nodes to run this simulation,

with 4096 nodes and 16 ppn. Previously, I had no trouble running the IC

file for a 512^3 simulation and 256^3 simulation, however, when I try to

create the IC file for the 1024^3 particle simulation, I get the following

output:

----------------------------------------

Begin Torque Prologue on nid25338

at Thu May 10 21:17:35 CDT 2018

Job Id: 8599200.bw

Username: kirsh

Group: ILL_baoy

Job name: ics1024

Requested resources: neednodes=4096:ppn=16:xe,

nodes=4096:ppn=16:xe,walltime=0

0:30:00

Queue: normal

Account: baoy

End Torque Prologue: 0.132 elapsed

----------------------------------------

Task=0 Local_nx=1024

allocated 8208 Mbyte on Task 0

reading Lagrangian glass file...

reading 'dummy_glass.dat' with 4096 particles

Nglass= 4096

1073741824 particles on task=0 (slabs=1024)

Total number of particles = 1073741824

start computing displacement fields...

vel_prefac= 2.13829, vel_prefac2= 4.27658, hubble_a=13.6921 fom=0.999977

allocated 72.0469 Mbyte on Task 0

starting axes=3...

Application 66871546 exit codes: 139

Application 66871546 resources: utime ~34s, stime ~26s, Rss ~33714504,

inblocks

~8849, outblocks ~21293

------------------------------------------------------------

------------------

The ics.param file is the same as the 512^3 IC file, where the only

difference is the number of particles and boxsize. I've included the

parameterfile below

Nmesh 1024 % This is the size of the FFT grid used to

% compute the displacement field. One

% should have Nmesh >= Nsample.

Nsample 1024 % sets the maximum k that the code uses,

% i.e. this effectively determines the

% Nyquist frequency that the code assumes,

% k_Nyquist = 2*PI/Box * Nsample/2

% Normally, one chooses Nsample such that

% Ntot = Nsample^3, where Ntot is the

% total number of particles

Box 1024000.0 % Periodic box size of simulation

FileBase ics1024 % Base-filename of output files

OutputDir ./ICs/ % Directory for output

GlassFile dummy_glass.dat % File with unperturbed glass or

% Cartesian grid

GlassTileFac 64 % Number of times the glass file is

% tiled in each dimension (must be

% an integer)

Omega 0.272 % Total matter density (at z=0)

OmegaLambda 0.728 % Cosmological constant (at z=0)

OmegaBaryon 0.045 % Baryon density (at z=0)

OmegaDM_2ndSpecies 0.0 % Omega for a second dark matter species (at

z=0)

HubbleParam 0.7 % Hubble paramater (may be used for power spec

parameterization)

Redshift 40.0 % Starting redshift

Sigma8 0.81 % power spectrum normalization

SphereMode 1 % if "1" only modes with |k| < k_Nyquist are

% used (i.e. a sphere in k-space), otherwise

modes with

% |k_x|,|k_y|,|k_z| < k_Nyquist are used

% (i.e. a cube in k-space)

WhichSpectrum 0 % "1" selects Eisenstein & Hu spectrum,

% "2" selects a tabulated power spectrum in

% the file 'FileWithInputSpectrum'

% otherwise, Efstathiou parametrization is used

FileWithInputSpectrum /u/sciteam/kirsh/2LPTic/matter2_psz40.txt %

filename of tabulated input

% spectrum (if used)

InputSpectrum_UnitLength_in_cm 3.085678e24 % defines length unit of

tabulated

% input spectrum in cm/h.

% Note: This can be chosen

different from UnitLength_in_cm

ShapeGamma 0.21 % only needed for Efstathiou power spectrum

PrimordialIndex 0.967 % may be used to tilt the primordial index,

% primordial spectrum is k^PrimordialIndex

Seed 123456 % seed for IC-generator

NumFilesWrittenInParallel 1 % limits the number of files that are

% written in parallel when outputting

UnitLength_in_cm 3.085678e21 % defines length unit of output (in

cm/h)

UnitMass_in_g 1.989e43 % defines mass unit of output (in

g/cm)

UnitVelocity_in_cm_per_s 1e5 % defines velocity unit of output

(in cm/sec)

WDM_On 0

WDM_Vtherm_On 0

WDM_PartMass_in_kev 10.0

------------------------------------------------------------

------------------------------------

Am I missing something in my parameterfile or do I just need to request

more nodes?

Best,

Tom Kirsh

Received on 2018-05-11 18:45:42

*
This archive was generated by hypermail 2.3.0
: 2022-09-01 14:03:43 CEST
*