Re: MemPools rewrite

Andres Kroonmaa wrote:

> search for right pool based on ptr would have much less overhead
> with chunks than it would be with current Pools. This could even
> be considered.

Currently there is no search AFAIKT.

> I don't think so. each separate malloc has its own memory overhead,
> which is much smaller with memPools, especially for small object sizes.
> For large objects you may be right, but direct reduction in overhead
> is only part of the improvement.

Preallocation would fix the initial fragmentation issue.

> Because there are many Pools in separate memory areas, overall system
> memory has quite different fragmentation issues. I hope to see better
> fragmentation figures that would give most of the improvement.

Fragmentation can be addressed without going to chunked allocation
simply by allocating several entries at a time like Alex suggested, and
then keep them relatively sorted and free from the "tail" (other end
from where you are picking entries to use)

> hmm, how does compiler handle struct:
> struct {
> char b;
> double x;
> struct {
> int i;
> short s;
> char c;
> double y;
> }
> }
> does it really reorder them?

No, it can't unless you allow it to.

However, the compiler will know the required alignment of the structure
(simple max of the alignment of all members).

> malloc is known to return 8-aligned pointers. If there is an array of
> struct above, does compiler add alignment overhead between items?

Yes.

> btw, does "double" need to be aligned only on 64-bit cpus?

double needs to be aligned to the alignment required by the CPU.
Different for each, varying from 1 byte to 8 byte alignment if I
remember correctly.

> what else besides malloc? squid depends on malloc like nothing else
> already now. malloc is top-5 lib calls squid is making per second.
> If malloc is not safe, then how can squid survive at all?

malloc is safe for all C allocations.

There are a few OS dependent allocations where malloc cannot be used,
but Squid does not use any such OS features.

> Should add comments into memPools sources to warn against problems.

Anyone writing a memory allocator should know about alignment.

Currently MemPools are not a memory allocator, only a memory tracker, so
it does not need to care about alignment.

/Henrik
Received on Thu Oct 19 2000 - 00:42:44 MDT