Wireshark · Ethereal-dev: Re: [Ethereal-dev] issues with large tables in ethereal source...

[Nathan Neulinger <nneul@xxxxxxx>]

I wasn't figuring anything quite so complex for this, but I don't see
any reason that we'd have to limit it to building tables of
value-strings.

I wonder if it would be worthwhile to come up with a simple syntax for
describing these two tasks and write up perl script to do it.

I think I'd want to avoid the tabbing-syntax you used though...

What do y'all think of having a gentables.pl script that would be called
similar to:

gentables.pl packet-kerberos.tbl

that would auto-generate:

packet-kerberos-defs.h
	contains defs of ints for hf_ stuff, and #defines
packet-kerberos-hf.inc
	contains hf_ entries
packet-kerberos-val.inc
	contains value_string structures

I suppose it could have a few options to indicate that perhaps you
wanted to generate different file names, or even generate a single file
instead of three. How hard would it be to allow
proto_register_field_array to be called more than once? If you could
call it more than once, the hf_ stuff could be made to be static at the
file level instead of in the subroutine, eliminating the need to do
#includes in the middle of structure definitions. This would also make
it extremely easy to have gentables.pl generate a single file for
everything.

Now, something I've also wanted to do with this, but haven't thought
much about - if we generate many of the tables automatically - we could
likely dramatically improve parsing performance by either hashing or
sorting the tables for efficient lookup. I'm not sure of the best way to
implement this though as it either requires a special hf registration
flag of some sort or extra routines. Seems like this could be a later
goal in any case.

If you think you'd accept something like this into the sources (at least
the developer side, not necessarily the distributed tarball though) -
I'll go ahead and work something up for packet-kerberos and send a
diff/copy of the script to -dev for people to try out. 

-- Nathan


Christophe Tronche wrote:
> 
> I agree with you, not only for enumerated values but also for the
> hf_register_info / hf_ value pairs. For example, the packet-x11
> dissector uses 40+ tables, some with other 120 entries, but also
> declares more than 360 hf_ fields. The source was generated by a PERL
> script converting a text file such as
> 
> textitem        NONE    NONE
>            font    UINT32  HEX
>            string          NONE    NONE
>                    delta           INT8    DEC
> 
> (the amount of tabs at the beginning of the line in the description
> file gives the level)
> 
> into two .h files:
> 
>    static int hf_x11_textitem = -1;
>    static int hf_x11_textitem_font = -1;
>    static int hf_x11_textitem_string = -1;
>    static int hf_x11_textitem_string_delta = -1;
> 
> and
> 
>    { &hf_x11_textitem, { "textitem", "x11.textitem", FT_NONE, BASE_NONE, NULL, 0, "" } },
>    { &hf_x11_textitem_font, { "font", "x11.textitem.font", FT_UINT32, BASE_HEX, NULL, 0, "" } },
>    { &hf_x11_textitem_string, { "string", "x11.textitem.string", FT_NONE, BASE_NONE, NULL, 0, "" } },
>    { &hf_x11_textitem_string_delta, { "delta", "x11.textitem.string.delta", FT_INT8, BASE_DEC, NULL, 0, "" } },
> 
> (details and the script can be found in the archives at
> http://www.ethereal.com/lists/ethereal-dev/200006/msg00080.html)
> 
> Unfortunately, the script was just a quick hack to solve my problem
> and has no flexibility at all. It may give ideas, however.
> 
> Just my 0.02 french francs.
> 
> --
> Christophe Tronche      ch.tronche@xxxxxxxxxxxx
> marchFIRST (France)     http://tronche.com/
> 
> > I brought this up once in the past, never really heard anything back.
> >
> > I think ethereal source needs some facility for creating/maintaining large
> > tables. For example, the kerberos dissector has error codes - about 60 of
> > them. It's highly annoying to have to #define every one of those codes, and
> > then use it later to create the lookup table, especially since it all has to
> > be done by hand initially.
> >
> > I'd like to see something like a preprocessed (hand-run by developers and
> > not generated at build time) table include that you could cleanly define
> > lists of 'code', 'value', 'text'. With 'code' being optional - since alot of
> > people don't need that in their sources. You'd only need the code if you're
> > actually comparing against that constant anywhere directly as opposed to
> > everything being table lookups.
> >
> > Some sort of standardized naming for these tables would be good as well.
> >
> > ------krb5_tables.tbl-------
> > table krb5_error_codes
> > KRB5_ERROR_1, 1, this is the first kerberos error
> >
> > table krb5_encryption_types
> > 1, this is the first kerberos encryption type
> > ------
> >
> > would generate either into krb5_tables.c AND krb5_tables.h, or just
> > krb5_tables.c, depending on how we wanted to do things.
> >
> > ------krb5_tables.c--------
> > /* Definitions for table 'krb5_error_codes' */
> > #define KRB5_ERROR_1  1
> >
> > /* Table content for 'krb5_error_codes' */
> > static const value_string krb5_error_codes[] = {
> >       { KRB5_ERROR_1, "this is the first kerberos error" }
> > };
> >
> > /* Definitions for table 'krb5_encryption_types' */
> > /* no definitions provided */
> >
> > /* Table content for 'krb5_encryption_types' */
> > static const value_string krb5_encryption_types[] = {
> >       { 1, "this it the first encryption type" }
> > };
> > ----------------------------
> >
> > If split into two files, the #define's would be in the .h.
> >
> >
> > Just an idea. I think it would clean up the code for kerberos, and a number
> > of the other large-table dissectors as well.
> >
> > -- Nathan
> >
> > ------------------------------------------------------------
> > Nathan Neulinger                       EMail:  nneul@xxxxxxx
> > University of Missouri - Rolla         Phone: (573) 341-4841
> > Computing Services                       Fax: (573) 341-4216

-- 


------------------------------------------------------------
Nathan Neulinger                       EMail:  nneul@xxxxxxx
University of Missouri - Rolla         Phone: (573) 341-4841
CIS - Systems Programming                Fax: (573) 341-4216