Wireshark-dev: Re: [Wireshark-dev] RFD: The Future of Memory Management in Wireshark
On Wed, Oct 24, 2012 at 8:10 AM, Sébastien Tandel
<sebastien.tandel@xxxxxxxxx> wrote:
>
>
> On Wed, Oct 24, 2012 at 1:10 AM, Guy Harris <guy@xxxxxxxxxxxx> wrote:
>>
>>
>> On Oct 18, 2012, at 6:01 PM, Evan Huus <eapache@xxxxxxxxx> wrote:
>>
>> > I have linked a tarball [2] containing the following files:
>> > - wmem_allocator.h - the definition of the allocator interface
>> > - wmem_allocator_glib.* - a simple implementation of the allocator
>> > interface backed by g_malloc and a singly-linked list.
>>
>> Presumably an implementation of the allocator could, instead of calling a
>> lower-level memory allocator (malloc(), g_malloc(), etc.) for each
>> allocation call, allocate larger chunks and parcel out memory from the
>> larger chunks (as the current emem allocator does), if that ends up saving
>> enough CPU, by making fewer allocate and free calls to the underlying memory
>> allocator, so as to make it worth whatever wasted memory we have at the ends
>> of chunks?
>>
>
> One step further, instead of mempools, I think wireshark could have great
> interest in implementing slabs (slab allocator). Slabs had initially been
> designed for kernel with several advantages over traditional allocators in
> terms of resources needed to allocate (CPU), (external / internal)
> fragmentation and also cache friendliness (most of the traditional
> allocators don't care). I've attached some slides about a high-level
> description of slab.
>
> Since then, another paper has been written showing some improvements and
> what it took to write a slab for user-space (libumem). There is another
> well-known exampel out there, called memcache, that implements its own
> version (and could be a good intial point for wireshark implementation, who
> knows? :))
If I understand correctly, a slab allocator provides the most benefit
when you have to alloc/free a large number of the same type of object,
but I don't know if this is necessarily the case in Wireshark. There
are probably places where it would be useful, but I can't think of any
off the top of my head. TVBs maybe? I know emem is currently used all
over the place for all sorts of different objects...
You could certainly shoehorn a slab allocator into wmem's current
architecture, but you'd have to abuse the wmem_allocator_t interface
to do it. I suspect that it would make more sense to implement slabs
separately anyways - since their goal is primarily performance you
would want to cut out the function pointers that wmem currently uses.
It's definitely worth thinking about though.
Thanks,
Evan
