Snapshot file format
The primary result of a simulation with GADGET-4 are snapshots, which are simply dumps of the state of the system at certain times. GADGET-4 supports parallel output by distributing a snapshot into several files, each written by a group of processors. This procedure allows an easier handling of very large simulations; instead of having to deal with one file of size of a dozens of GB, say, it is much easier to have several files with a smaller size of a few hundred MB to a couple of GB instead. Also, the time spent for I/O in the simulation code can be reduced if several files are written in parallel.
Each particle dump consists of
k files, where
k is given by
k > 1, the filenames are of the form
XXX stands for the number of the dump,
for the number of the file within the dump. Say we work on dump 7 with
k=16 files, then the filenames are
snapshot_007.15, and if HDF5 is in use (recommended!), they will be
snapshot_007.15.hdf5. They will actually be
stored in a directory called
snapdir_007, created in the output
k=1, the filenames will just have the form
snapshot_XXX and no separate subdirectory is created for the
snapshot. The base name "snapshot" can be changed by setting
SnapshotFileBase to another value.
Each of the individual files of a given set of snapshot files contains a variable number of particles, but the files all have the same basic format (this is the case for all three fileformats supported by GADGET), and all of them are in binary. A binary representation of the particle data is the preferred choice, because it allows much faster I/O than ASCII files, and in addition, the resulting files are much smaller while still providing loss-less storage of the data.
Legacy Format 1
In the original default file format of GADGET (selected with
SnapFormat=1), the data is organised in blocks, each containing a
certain information about the particles. For example, there is a block
for the coordinates, and one for the temperatures, etc. The sequence
of blocks in snapshots files of GADGET-4 (which for the most part is
compatible with older versions of GADGET) is given in the table
below. Not all blocks are necessarily present in all simulations, for
example, the blocks describing gas properties such as internal energy
or density will only be included in hydrodynamic simulations, and some
output blocks are an optional
Config.sh option. The presence of the
mass block depends on whether or not particle masses are defined to be
constant for certain particle types by means of the
Massarr table in
the file header. If a non-zero mass is defined there for a certain
particle type, particles of this type will not be listed with
individual masses in the mass block. If such fixed particle masses are
defined for all types that are present in the snapshot file, the mass
block will be completely absent.
|Nr||Format2-ID||HDF5 Identifier||Block contents|
|5||MASS||Masses||Masses (only for particle types with variable masses)|
|6||U||InternalEnergy||Thermal energy per unit mass (only SPH particles)|
|7||RHO||Density||Density of SPH particles|
|8||HSML||SmoothingLength||SPH smoothing length h|
|9||POT||Potential||Gravitational potential of particles|
|10||ACCE||Acceleration||Acceleration of particles|
|11||ENDT||RateOfChangeOfEn||Rate of change of entropic function of SPH particles|
|12||TSTP||TimeStep||Timestep of particles|
Within each block, the particles are ordered according to their particle type, i.e. gas particles will come first (type 0), then type-1 particles, followed by type-2 particles, and so on. However, it is important to realize that the detailed sequence of particles within the blocks may change from snapshot to snapshot. Also, a given particle may not always be stored in the snapshot file with the same sub-number among the files belonging to one set of snapshot files. This is because particles may move around from one processor to another during the course of a parallel simulation. In order to trace a particle between different outputs, one therefore has to resort to the particle IDs, which are intended to be used to label particles uniquely. (In fact, the uniqueness of the IDs in the initial conditions is checked upon start up of the code.)
The first block (number 1) has a special role, it is a header which contains global information about the particle set, for example the number of particles of each type, the number of files used for this snapshot set, etc. The fields of the file header for formats 1 and 2 in GADGET-4 is given in the table below:
|Header Field||Type||HDF5 name||Comment|
|Npart||uint||NumPart_ThisFile||The number of particles of each type in the present file.|
|Nall||uint64||NumPart_Total||Total number of particles of each type in the simulation.|
|Massarr||double||MassTable||The mass of each particle type. If set to 0 for a type which is present, individual particle masses are stored for this type.|
|Time||double||Time||Time of output, or expansion factor for cosmological simulations.|
|Redshift||double||Redshift||z=1/a-1 (only set for cosmological integrations)|
|BoxSize||double||BoxSize||Gives the box size if periodic boundary conditions are used.|
|NumFiles||int||NumFilesPerSnapshot||Number of files in each snapshot.|
In GADGET-1/2/3, the outline of the fields in the header has been
slightly different, in particular the number of particle types was
always fixed to 6 (now this is given by
NTYPES), and the total
particle number was stored as a 32-bit integer, such that simulations
exceeding particle numbers of a couple of billion needed as special
(and ugly) extension of the header, where the high-order bits in the
particle numbers where stored in a separate entry. In addition,
cosmological parameters like
Omega0 and various flags informing
about enabled/disabled code features were redundantly stored there as
well. Finally, the total header length was filled to a total length of
exactly 256 bytes, with a view to reserve the extra space for future
extensions. However, as the block-size guards (see below) anyhow store
the length of the header in the file format, this has been a
superfluous restriction. In GADGET-4, this is hence lifted, and also
other clean-ups of the header structure are implemented (such as going
to 64-bit integers for the particle numbers). While this makes the new
file format incompatible with older versions of GADGET, backwards
compatibility can be enforced by setting the
To allow an easy access of the data also in Fortran (this should not be misunderstood as an encouragement to use Fortran), the blocks are stored using the "unformatted binary" convention of most Fortran implementations. In it, the data of each read or write statement is bracketed by block-size fields, which give the length of the data block in bytes. These block fields (which are two 4-byte integers, one stored before the data block and one after) can be useful also in C-code to check the consistency of the file structure, to make sure that one reads at the right place, and to skip individual blocks quickly by using the length information of the block size fields to fast forward in the file without actually having to read the data. The latter makes it possible to efficiently read only certain blocks, for example, just temperatures and densities but no coordinates and velocities. Note however that this block size structure imposes the restriction that individual blocks may not be larger than 4 GB in file formats 1 and 2.
Assuming that variables have been allocated/declared appropriately, a possible read-statement of some of these blocks in Fortran could then for example take the form:
read (1) npart, nall, massarr, a, redshift read (1) pos read (1) read (1) id read (1) masses
In this example, the block containing the velocities, and several
fields in the header, would be skipped. Further read-statements may
follow to read additional blocks if present. In the file
read_snapshot.f included in the code distribution, you can find a
simple example of a more complete read-in routine. There you will
also find a few lines that automatically generate the filenames, and
further hints how to read in only certain parts of the data.
When you use C, the block-size fields need to be read or skipped
explicitly, which is quite simple to do. This is demonstrated in the
read_snapshot.c. Note that you need to generate the
appropriate block field when you write snapshots in C, for example
when generating initial conditions for GADGET.
GADGET-4 will check explicitly that the block-size fields contain the correct values and refuse to continue if not. It will also use the block-size information to convert between single and double precision (or vice versa), and between 32-bit and 64-bit, where appropriate, i.e. when reading in files the code will autodetect if single or double precision is used and do conversions if needed.
Legacy Format 2
Whether or not a certain block is present in GADGET's snapshot file format depends on the type of simulation, and the makefile options. This makes it difficult to write reasonably general I/O routines for analysis software when file formats 1 and 2 are used, capable of dealing with output produced for a variety of makefile settings. For example, if one wants to analyse the (optional) rate of entropy production of SPH particles, then the location of the block in the file changes if, for example, the output of the gravitational potential is enabled or not. This problem becomes particularly acute if more complicated baryonic physics modules or optional output fields are added to GADGET runs.
To remedy this problem, a variant of the default fileformat was
implemented in GADGET (based on a suggestion by Klaus Dolag), which
can be selected by setting
SnapFormat=2. Its only difference is that
each block is preceded by a small additional block which contains an
identifier in the form of a 4-character string (filled with spaces
where appropriate). This identifier is listed in the "Format2-ID" in
the above block table. Using it, one can write more flexible I/O
routines and analysis software that quickly fast-forwards to blocks of
interest in a simple way, without having to know where it is expected
in the snapshot file.
HDF5 file format
A yet more general solution is provided by HDF5, which is nowadays the strongly recommended format for the code. If the hierarchical data format is selected as format for snapshot files or initial conditions files, GADGET-4 accesses files with low level HDF5 routines. Advantages of HDF5 lie in its portability (e.g. automatic endianness conversion) and generality (which however results in somewhat more complicated read and write statements), and in the availability of a number of tools to manipulate or display HDF5 files. Also, attributes such as units or conversion factors can be stored along-side data-sets. Together with HDF5 utility commands like h5ls, h5dump, etc., this makes the file format self-documenting to a significant degree. A wealth of further information about HDF5 can be found on the website of the HDF5 library.
In the HDF5 file format of GADGET, the blocks are stored as datasets
with names as given in the column "HDF5 Identifiers" in the table
above. To make accessing different particle types easier, for each
particle type presented in the snapshot file, a separate data group is
ParticleType1, and so on, and
the blocks of the standard file format are appearing as datasets
within these groups. These data groups can be thought of as
subdirectories in the HDF5 file, with each being devoted to one
particular particle type. In addition, in HDF5 each of these datasets
is also equipped with attribute values that store conversion factors
to cgs units, as well as factors that inform whether one has to
multiply with a certain power of the scale factor and/or of the
dimensionless Hubble parameter
h to get to truly physical
Finally, the file header is also moved to a special
Header group in
the HDF5 output. This group does not contain datasets, but only
attributes that hold the values for all the fields defined for file
format 1 and 2 of GADGET-4. The names of these attributes are listed
in the column "HDF5 name" in the table above.
To graphically explore the contents of HDF5 files, the program HDFView, available for free from the HDF-Group, is one good possibility. It allows an easy exploration of the structure and the contents of the various HDF5 outputs produced by GADGET4. It also readily reveals the type of data set fields, the values of attributes, etc., and hence greatly facilitates the writing of corresponding analysis scripts.
Alternatively, one can also explore the contents of HDF5 files at the command line, using simple tools that are part of the HDF5 library. For example, to list contents of the header of a snapshot file on the command line, use the following command:
h5dump -g Header snapshot_007.3.hdf5
To list, for example, which blocks are available for type-0 particles, you can use the following command:
h5ls snapshot_007.3.hdf5 would tell you which particle types are
available as separate data groups.
Finally, GADGET-4 also stores all the parameters and config file
options with which it was run to produce the given snapshot in all its
HDF5 files. This is realized through Parameter attributes stored in
two special data groups called
To examine the parameter file settings that were used, you can hence issue the command:
h5dump -g Parameters snapshot_007.3.hdf5
And for retrieving the
Config.sh options, you can use:
h5dump -g Config snapshot_007.3.hdf5
Here are a couple of additions noted on the format, units, and variable types of the individual blocks in the snapshot files:
Particle positions: A floating point 3-vector for each particle, giving the comoving coordinates in internal length units (corresponds to kpc/h if the above choice for the system of units is adopted). N=sum(header.Npart) is the total number of particles in the file. Note: For file formats 1 and 2, the particles are ordered in the file according to their type, so it is guaranteed that particles of type 0 come before those of type 1, and so on. However, within a type, the sequence of particles can change from dump to dump. For file format 3, different particle types appear in separate particle groups.
Particle velocities: Particle velocities
uare in internal velocity units (corresponds to km/sec if the default choice for the system of units is adopted). For cosmological simulations, peculiar velocities
vare obtained from
uwith sqrt(a), i.e. v = u * sqrt(a).
Particle identifiers: These are unsigned 32-bit or 64-bit integers (if
IDS_64BITis set) and intended to provide a unique identification of particles. The order of particles may change between different output dumps, but the IDs can be easily used to bring the particles back into the original order for every dump.
Variable particle masses: Single or double precision floats, with a total length Nm. Only stored for those particle types that have variable particle masses (indicated by zero entries in the corresponding
massarrentry in the header). Nm is thus the sum of those
npartentries that have vanishing
Nm=0, this block is not present at all.
Internal energy: Single/double precision float values for Ngas. Internal energy per unit mass for the
Ngas=npart(0)gas particles in the file. The block is only present for
Ngas>0. Units are again in internal code units, i.e. for the standard system of units,
uis given in
Density: The comoving density of SPH particles. Units are again in internal code units, i.e. for the above system of units,
rhois given in
10^10 Msun/h / (kpc/h)^3.
Smoothing length: Smoothing length of the SPH particles. Given in comoving coordinates in internal length units.
Gravitational potential: This block will only be present if it is explicitly enabled in the makefile.
Accelerations: Likewise, only present if it is explicitly enabled in the makefile.
Rate of entropy production: The rate of change of the entropic function of each gas particles. For adiabatic simulations, the entropy can only increase due to the implemented artificial viscosity. This block will only be present if it is explicitly enabled in the makefile.
Timesteps of particles: Individual particle timesteps of particles. For cosmological simulations, the values stored here are d ln(a), i.e. intervals of the natural logarithm of the expansion factor. This block will only be present if it is explicitly enabled in the makefile.
Format of initial conditions
The possible file formats for initial conditions are the same as those
for snapshot files, and are selected with the
parameter. However, only the blocks up to and including the gas
temperature (if gas particles are included) need to be present; gas
densities, SPH smoothing lengths and all further blocks need not be
provided and are ignored.
In preparing initial conditions for simulations with gas particles,
the temperature block can be filled with zero values, in which case
the initial gas temperature is set in the parameterfile with the
InitGasTemp parameter. However, even when this is done, the
temperature block must still be present. Note that the field
in the header will be ignored when GADGET-4 is reading an initial
conditions file. Instead, you have to set the time of the start of the
simulation with the
TimeBegin parameter in the parameterfile.