Title: Message
see
below
John,
> I'm going to merge files
from captures on multiple interfaces.
> Unfortunately I have some duplicates
appearing in the traces I am
merging.
> Has anyone seen any utility
to do duplicate packet removal
?
I have tinkered with this sort of
problem when comparing two traces of the same task taken from client and
server. I was more interested in what sort of packets were
being delayed in
transmission. My naive
solution would be to:
-
mung the output of both files using tethereal -x and generate a single line of
text per frame
-
fudge the timestamps of one of the trace files so that traces are aligned
properly
- sort and pipe
the output through uniq
-
un-mung back into a suitable format for text2pcap to read.
But
that isn't going to work in general
- How do you ensure that the real world clocks are
synced for two traces?
- How much of fudge factor is needed to comparing the timestamps
of frames in the two traces?
- If
trace 1 sees 2 pings to an IP address and trace 2 sees 3. How do you
decide whether there were 3, 4 or 5 (or 8!) pings
really?
- What if trace 1 says packet A arrived before B
and trace 2 says B arrived before A?
Firstly I'm running several capture processes on one
system so timestamps are the same. If captured on a different system,
we use NTP/GPS timing sync from a local
server.
I am assuming that frames are unique, and a
retransmission will be different to the original
packet.
It might be a simple thing to do to modify the code to
keep a circular buffer of src, dst, CRC, and then scan it for a duplicates
before allowing a packet to be merged into the output file. The length of
the buffer will be the correlation window for the duplicate packet
check.
Read from input
files
Pick most recent packet
Check against buffer
If in buffer, then
ignore
If not in buffer
put src, dst, CRC into
buffer
write to output
file
Loop until all
input files read
Does this seem sensible
?
It may be easier to advise you if we understood what
sort of packets were being duplicated? Eg is it MAC level broadcasts, RIP
updates etc, HSRP hellos?
Alistair
Registered Office:
Marks & Spencer p.l.c
Michael House, Baker
Street,
London, W1U 8EP
Registered No. 214436 in England and
Wales.
Telephone (020) 7935 4422
Facsimile (020) 7487
2670
www.marksandspencer.com
Please note that electronic mail
may be monitored.
This e-mail is confidential. If you received it by
mistake, please let us know and then delete it from your system; you should
not copy, disclose, or distribute its contents to anyone nor act in reliance
on this e-mail, as this is prohibited and may be unlawful.
The
registered office of Marks and Spencer Financial Services PLC, Marks and
Spencer Unit Trust Management Limited, Marks and Spencer Life Assurance
Limited and Marks and Spencer Savings and Investments Limited is Kings Meadow,
Chester, CH99 9FB.
