Re: Corrupt Particle ID's in Snapshot Files

From: Yves Revaz <yves.revaz_at_obspm.fr>
Date: Wed, 08 Nov 2006 10:13:54 +0100

Dear Matt,

Its seems that the way you are reading your gadget file is wrong.
Gadget gives an Id to each particle between 0 and N-1, thus your Id's values
shoud be less than 2563 which is not the case in the output you get.

In order to be sure that you read correctely your snapshot, I propose
the following:

1) Check that you read correctely each block. In the gadget file, it is easy to check it, because
there is a header before each block (4 bytes), telling the total number of bytes in the block.
At the end of the block you will find exactly the same header. Check it !
Its the best way to avoid offsets.

2) Check the byte conversion for each block. The gadget default is that
real (pos,vel,mass) and integer (Id) are coded on 4 bytes.

3) Check that when you think to have read the whole file, you really are
at the end of the file.

On 64 bit machines, I get some problems linked to the size of the
structure header. The same header may take a different number of bytes on a 64 compared to a 32 bit
machine. This generates a shift of all blocks in your snaphsot file.
With gcc you can avoid this problem, using the option -fpack-struct.

Regards,

Yves










Matthew Francis wrote:

>Hi All,
>
> I'm getting an odd error in some of my GADGET2 snapshots. Occasionally
>I'll get a bunch of particle ID's that look corrupted. For example in a
>file containing 16777216 particles (256^3), I checked all the ID's to
>see if they lay between 1-N_particles, and found the following:
>
>Particle ID: Position in Snapshot:
>-2071986048 6014114
> 1521338276 6014115
> 33554432 6014117
> -1551499008 6014118
> 50331648 6014119
> 117440512 6014120
> 536870912 6014122
> -1003326207 6014123
> 100663296 6014124
> 1701080942 6014125
> -301793280 6014127
> -368902144 6014128
> -368902144 6014130
> 603979776 6014133
> 100663297 6014134
> 33556480 6014135
>
>All the other particles in the file had sensible ID's, so it's only ~20
>in ~16 million that are affect. As you can see the particles affected
>are next to each other in the list of particle in the snapshot file. The
>positions of the particles with the crazy IDs look sensible, i.e. they
>lie within the box, just the IDs are affected. The other strange thing
>about this is that in a given run it won't affect all the snapshots, so
>the corrupt ID's are not being carried through the simulation. Roughly 1
>in 5 snapshots I write out show this strange behaviour.
>
>I'm using the standard GADGET2 snapshot format, but not HDF5.
>
>I've recently started using GADGET2 on a new dedicated cluster, rather
>than the network I had been using. On the old system I had no problems,
>this has only arisen since moving to the new cluster. I'll try and give
>as much information about this system as I can. It's a 16 node cluster
>with each node containing 2 duel core AMD Opteron processors with 4GB
>per node. I'm running my simulations with 256^3 particles on 2 nodes
>each, so 8 processors and 8GB of RAM in total. The nodes are running
>Open SuSE Linux 10.1, using mpich2 to implement mpi.
>
>Has anyone seen an error like this before or can suggest a possible
>solution? It's not a terminal issue for me, since for the moment I'm not
>worried about tracking individual particles, but I may be in the future
>and in any case am concerned that this may be pointing to a deeper
>issue.
>
>Regards
>
>Matt Francis
>
>
>


-- 
                                                (o o)
--------------------------------------------oOO--(_)--OOo-------
  Yves Revaz
  Lerma Batiment A           Tel : ++ 33 (0) 1 40 51 20 79
  Observatoire de Paris      Fax : ++ 33 (0) 1 40 51 20 02 
  77 av Denfert-Rochereau    e-mail : yves.revaz_at_obspm.fr
  F-75014 Paris              Web : http://obswww.unige.ch/~revaz/
  FRANCE             
----------------------------------------------------------------
Received on 2006-11-08 10:13:28

This archive was generated by hypermail 2.3.0 : 2023-01-10 10:01:30 CET