Stephen Donnelly schrieb:
On Mon, 2008-01-21 at 22:00 +0100, Ulf Lamping wrote:
Gianluca Varenni schrieb:
I think the description of timestamp formats is quite bad in the specs.
The timestamps are represented as a 64bit quantity split into high and low
32 bits, that represent the number of microseconds/nanoseconds/??? from
1/1/1970 (that's the meaning of in "in standard unix format i.e. since
1/1/1970").
The reason behind using a single 64bit quantity instead of
seconds/subseconds is twofold:
1. if we use seconds and subseconds, 32bits don't allow to go under the
nanosecond.
2. several hardware-based capture cards represent timestamps as
nanoseconds/microseconds as a single 64bit quantity i.e. they don't split
them into seconds and subseconds.
BTW, there was a discussion on the timestamp format on the ntar-workers
mailing list, here
http://www.winpcap.org/pipermail/ntar-workers/2006-March/000122.html
Yes, the timestamp spec of the EPB (and PB) is *very misleading* here
and definitely needs a clarification! The structure - and the
descriptive text - looks far too much "libpcap like" to get an idea that
it's actually different.
Reading the text a few times now, I think it's even not very consistent
in itself ...
I believe part of the idea behind allowing the timestamp precision to be
specified as a binary fraction (2^-X) is to allow the use of Endace ERF
timestamps in pcapng/ntar natively.
Well, my complain in the first place is that the specification text is
pretty easy to misunderstand (just as I did), which is usually a very
bad thing for a specification.
Timestamps in network traces are typically 64-bit, comprising a 32-bit
'seconds' part with a UNIX epoch, and a 32-bit 'subsecond' field
counting milli/micro/nanoseconds withing the second. The most
significant few bits of the subsecond are not used as the conuter
saturates before using all the available bits.
I'm pretty much aware of that - I'm the author of
http://wiki.wireshark.org/Development/LibpcapFileFormat and I've
implemented the Wireshark switch from millisecond to nanosecond internal
resolution :-)
ERF timestamps are also 64-bits, and can be considered to be a 64-bit
value in units of 2^-32 seconds, with a UNIX epoch. Alternatively it can
be considered a fixed point count of seconds from the UNIX epoch.
Conveniently the 32 MSB are equivalent to the conventional 'seconds'
count, while the 32 LSB can be considered a binary fraction of seconds.
The lowest few LSB may not be used by specific hardware depending on the
available clock resolution.
Thanks for explaining it. But it just doesn't really help to explain it
here, it must be explained in the spec! Otherwise the next one that
graps the spec and implements stuff will be trapped.
Advantages include the ability to perform a single 64-bit integer
subtraction to find timestamp differences, and the ability to support
different hardware clock resolutions without changing timestamp format
or interpretation.
Using ERF timestamps directly in pcapng avoids the requirement to
convert the timestamp formats on the fly at capture time. This can be a
significant load at extreme packet rates.
Full ACK.
But as a drawback, now the former libpcap timestamp format is not
directly supported and needs conversion "on the fly" while writing
packet data.
It does suggest that timestamp
accessor functions should be available so that the any timestamp
(record/block) can be requested in any format.
That reminds me of the big/little endian problem, which is solved by
writing stuff in the native format and being able to detect and then
read (and probably convert) whatever comes in.
For timestamps this means: write libpcap, ERF, or whatever comes in and
being able to detect and then read (and probably convert) the timestamp
format.
I'm pretty much aware that this adds complexity to the programs reading
the format, but as you've said writing performance is the key.
Regards, ULFL
