Hi Peter,
first: happy-shark is very good idea. All notes above I would like to
discuss and then I will try to add my samples with respect to conclusions...
> For a basic testing framework, have a look at
> https://github.com/wireshark/happy-shark
>
> To-do: populate the test/ directory with lots of specific samples :-)
This approach looks good to me, I will try to add samples.
What I'm missing there is idea how to add description for a sample - I
mean "description" - why sample is there and "specification" - what
wireshark should do with it.
Testing with PDML is good idea, but it tests dissectors only. Time to
time you want to store sample because GUI should do something with it
(e.g. show it in red).
1) Won't be good idea to ask for e.g. <sample>.description.txt and
<sample>.specification.txt for each sample?
2) Won't be good idea to allow skip a sample from automatic testing
(because it is for GUI demonstration)?
> I think that happy-shark (which uses PDML output with a XSL
> post-processor) does what you want.
I tried to run make in top directory and it failed for first try :-) I
checked the output and found that something changed in DNS dissector and
dns.flags.recdesired .. value="1" was in past and now value="FFFFFFFF".
It is probably about signed/unsigned value - not important now.
But this issue pointed to me more things:
3) There should be easy way how to describe which wireshark version
processed stored PDML file.
Probably different outputs should be stored for main branches (2.0, 2.2,
...). It is obvious that each version of wireshark will change (improve)
some outputs. Therefore older release will not be able to process it
correctly (PDML diff will fail).
I think that without output version you can't run regression tests
automatically.
4) There should be easy way how to run custom wireshark version without
modifying Makefile. I have multiple versions for testing. I think
environment variable solve it.
5) There should be easy way how to describe which filter should be used
for preprocessing PDML. Now only filter.xsl is used.
Maintaining such sample repository would be hard work. But I believe
that when there will be rules how to add sample and which information
must be added to it, it will be much easier.
6) Rules must be written.
6) There should be a procedure which check whether repository/new sample
contains expected information. Something like git review procedure for
wireshark source/gerrit.
Sincerely yours,
Jirka Novak
