RE: Memory usage fix (patch)

> -----Original Message-----
> From: Steven Wilton [mailto:swilton@q-net.net.au]
> Sent: Thursday, March 17, 2005 10:57 AM
>
> > -----Original Message-----
> > From: Henrik Nordstrom
> > Sent: Thursday, March 17, 2005 9:12 AM
> >
> > On Thu, 17 Mar 2005, Steven Wilton wrote:
> >
> > > The problem is that uncacheable objects (ie size >
> > maximum_object_size,
> > > or download managers doing multiple partial requests on
> > large files) are
> > > always held in memory. Squid does free this memory as the
> > data is sent
> > > to the client, but it doesn't look like there's a backoff
> > mechanism when
> > > the data is arriving at a much faster rate than it is being
> > sent to the
> > > client.
> >
> > Normally this is dealt with by the fwdCheckDefer function.
> Maybe your
> > epoll implementation does not use the filedescriptors defer
> > function to
> > back off when needed?
>
> I did make some changes to my original epoll patch, and the
> epoll patch now
> work with the commSetDefer() function....

Actually.. How is the fwdCheckDefer function meant to slow this down? The
way I read the code is that it follows this logic:

httpReadReply
- check aborted && return
- read data from socket
- if length > 0 and we have processed headers
  - storeAppend
  - switch httpPconnTransferDone (check whether the transfer is comlete)
     - if transfer not complete, queue fd for read with callback to
httpReadReply

So, if the headers have been procesed, and the read call returns data, we
queue another read without checking whether to defer.

The patch that I submitted checks that a swap object is not first. If
entry->mem_obj->swapout.sio is not set, is it possible for another request
to be fetching from this object? I was pretty sure that I had tested this
case, and found that without a sio, there was no reference for squid to
generate a cache hit. (It was 7 months ago, and I could be mistaken.)

Having said this, the patch I originally posted did not seem to work well
with my testing (100% cpu usage with epoll). I am going to do further work
on it.

Steven

-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.308 / Virus Database: 266.7.3 - Release Date: 3/15/2005