Wireshark · Wireshark-dev: Re: [Wireshark-dev] Adding pcap-ng pipe support to dumpcap

[Anthony Coddington <Anthony.Coddington@xxxxxxxxxx>]

On Aug 31, 2017, at 1:31 PM, Guy Harris <guy@xxxxxxxxxxxx> wrote:
> On Aug 30, 2017, at 6:00 PM, Ed Beroset <beroset@xxxxxxxxxxxxxx> wrote:
> > There are some limitations.  Specifically, pipes don't allow random access, so any file formats that currently require that would need to either be rewritten

> Which, for at least one capture file format (Network Monitor format), would be impossible, as we don't define it, Microsoft does (and they're probably not very amenable to changing it, not least because they've deprecated NetMon in favor of Message Analyzer).  The only file formats *we* control to any degree are pcap and pcapng, neither of which require random access in order to read them sequentially.

> If we make extcap programs work like dumpcap, the only pipe involved is the control pipe between *shark and the program - the packets are written directly to a file - but that wouldn't make any difference for those file formats, as you can't, for example, read a NetMon file until it's *completely* written, with a frame table, and you can't do that until you've written all the packets to it.

Wtap filetypes could indicate that they do/do not support sequential only access with a flag in dump_open_table. There is already a flag for writing_must_seek. I'm sure at some point I saw some support in wtap internals for trying to read from pipes/stdin without random seeking support? If necessary wtap could be extended to have wtap filetypes return a 'need more data' rather than WTAP_ERR_SHORT_READ or blocking if they don't have the rest of the packet but have not reached EOF. There is also nothing stopping having an interface for extcap that requires dumping to an intermediate file for files that require seeking.

Also note that with the extcap toolbar support there is now optionally already a control pipe to and from extcap applications. One option would be to extend and use that interface to indicate when packets are available. In dumpcap as well ideally (replacing the capchild control pipe). Not sure if it is feasible to suggest extcap applications flush the output pipe only on packet boundaries?

Regards,
Anthony


From: Wireshark-dev <wireshark-dev-bounces@xxxxxxxxxxxxx> on behalf of Guy Harris <guy@xxxxxxxxxxxx>
Sent: Thursday, 31 August 2017 1:31 p.m.
To: Developer support list for Wireshark
Subject: Re: [Wireshark-dev] Adding pcap-ng pipe support to dumpcap
    
On Aug 30, 2017, at 6:00 PM, Ed Beroset <beroset@xxxxxxxxxxxxxx> wrote:

> One problem is that as dumpcap is currently written, it treats files and pipes very differently.

*Files* and pipes, or *capture devices* and pipes?

> but I can't help but think that the general approach you describe is the better long term strategy.

Probably.  It means that the interface between *shark and extcap programs would be different - but, while having extcap programs behave like dumpcap might complicate the extcap programs (although some of the code to do that could be in a library used by dumpcap  and by extcap programs), it might simplify the Wireshark capture code path.

> There are some limitations.  Specifically, pipes don't allow random access, so any file formats that currently require that would need to either be rewritten

Which, for at least one capture file format (Network Monitor format), would be impossible, as we don't define it, Microsoft does (and they're probably not very amenable to changing it, not least because they've deprecated NetMon in favor of Message Analyzer).   The only file formats *we* control to any degree are pcap and pcapng, neither of which require random access in order to read them sequentially.

If we make extcap programs work like dumpcap, the only pipe involved is the control pipe between *shark and the program - the packets are written directly to a file - but that wouldn't make any difference for those file formats, as you can't, for example, read  a NetMon file until it's *completely* written, with a frame table, and you can't do that until you've written all the packets to it.

However, I suspect Stephen is thinking of ERF format, which should be writable purely sequentially, so it shouldn't be a problem, whether you're writing to a pipe or to a file that's being read incrementally.
___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
              mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe