Wireshark · Wireshark-dev: Re: [Wireshark-dev] Reordering capture files

[Martin Mathieson <martin.r.mathieson@xxxxxxxxxxxxxx>]

I think I will write a separate wiretap console app (tonight, if I get carried away).

My log files tend to be long (100s MB), so I don't think a script such as yours would work well for me.

Ultimately it would be good to have this functionality in (probably) editcap, and if there is demand, make it another option when exporting frames from wireshark.

Regards,

Martin

On Mon, Sep 17, 2012 at 3:15 PM, James Howard Young <jyoung@xxxxxxx> wrote:

Hello Martin,

I've used mergecap to do this.

It's a bit of a hacky solution but I used to split the
original out-or-order packets out to separate 1 packet
trace files.  I then used mergecap's default chronological
merging behavior to piece the sections back into a single
file.   The problem with this approach is that mergecap can
ONLY work with about 512 trace files at a time.   Depending
on how many out of order sections the trace file contains
this can get very tedious to do manually.

I eventually cooked up a shell script that helped automate
the process.  But the script was very slow, but it did
(eventually) get the work done.

The script's main loop would fix-up one negative delta
section at a time.  It used capinfos -o report to determine
if the trace file had any out-of-order packets.  If so it
ran a tshark -td report grepping for any negative delta
time records and piping it to head -1.  It would extract
the frame number of the 1st negative delta time packet
and use that particular frame number to split the trace
file into two parts. The first part would contain all the
frames up to the out-of-order frame.   The second part
would contain all frame from the out-of-order frame onward.
The two parts would be merged back together with mergecap.
Then the newly reconstituted trace file would be processed
again and again until the capinfos -o report indicated
that the was in strict chronological order.

Unfortunately I lost that particular script but it shouldn't
be too hard to re-create.

But I think augmenting editcap or mergecap or even adding
a new wiretap based console app to do this would be useful.
I've thought about this before but I simply have NOT had the
time to invest in following up.  I suspect the ability of
wireshark with pcapng to support multiple concurrent inputs
might make it more likely for to have trace files with
"out-of-order" packets.

I hope this helps,

Jim Y.


On 9/17/12 9:21 AM, "Martin Mathieson" <martin.r.mathieson@xxxxxxxxxxxxxx>
wrote:

>Hi,
>
>
>I work with log files that are created from multiple sources, and
>although the timestamps are in good synchronisation, they are often
>written to the log file slightly out of order (up to a few milliseconds).
>
>
>editcap -S supports rewriting the timestamps to be in strict
>chronological order (assuming the frame order will be correct), but this
>is the opposite of what I need.
>
>
>So, I'm considering adding a new flag to editcap, or maybe creating a
>separate console program using wiretap (I haven't thought through how
>difficult it might be to support other options at the same time).  It
>would work something like this:
>- specify a number N (say 200), that would serve as the number of frames
>in the ordering buffer
>- when a new frame is read from the input file, insert it into its
>correct time order within the (up-to) N frames in the buffer.  Hopefully
>not a linear search :)
>- write the earliest frame to the output file
>- when we reach the end of the input file, just flush the sorted buffer
>to the output file
>
>
>Not sure if I'd keep the buffered frames in memory, or go back and reread
>them when it was time to write them out.
>
>
>
>
>Am I missing an existing way to get this functionality?  Am I the only
>person who needs this functionality?
>
>
>Thanks,
>Martin
>

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe