Re: Improving qos_flows mark feature - obtaining mark later

On Tue, 2011-11-01 at 12:12 +1300, Amos Jeffries wrote:
> On Mon, 31 Oct 2011 22:19:12 +0000, Andrew Beverley wrote:
> > On Mon, 2011-10-31 at 15:42 -0600, Alex Rousskov wrote:
> >> On 10/31/2011 03:03 PM, Andrew Beverley wrote:
> >>
> >> > Having thought about this further, I think what I was trying to
> >> achieve
> >> > was getting the mark every time a packet was received.
> >>
> >>
> >> > I have tried putting the relevant code into
> >> > ServerStateData::addVirginReplyBody() which seems to work for
> >> different
> >> > protocols as well as being called for each packet received.
> >> >
> >> > Is there any problems with this approach? If not, I will refine
> >> and
> >> > submit a patch.
> >>
> >> Yes, if you do not care about HTTP response header or FTP control
> >> messages: ServerStateData::addVirginReplyBody() is not called when
> >> the
> >> HTTP header or FTP control messages are received. If you want to
> >> catch
> >> those cases (it sounds like you do based on your "every packet"
> >> description above), then you need HttpStateData::readReply() and
> >> equivalent read handlers in other protocols.
> >
> > Ah, okay. Thanks for the explanation.
> >
> >> I do not think there is a single Server method that is always called
> >> for
> >> every Squid read. You should probably add it and call it from the
> >> corresponding HTTP and FTP-specific code.
> >
> > I assume that I should add it to all the protocols? I may need some
> > advice as to the correct functions within each one. Which one do you
> > think for FTP? dataRead()? processReplyBody()?
>
> All the read handlers. With type IOCB or taking CommIoCb* handlers
> needs a loot at.

Okay.

>
> If you really want it every packet you can't get any closer than
> commHandleRead() in comm.cc which is what schedules those handlers. The
> ccb->conn available in there is the connection descriptor being read.

It's been established that trying to achieve this is pointless, so I'll
either stick with the above, or alternatively reduce it further as
below.

>
> >
> >> Please keep in mind that Squid does not work on a TCP packet level.
> >> It
> >> uses the TCP socket interface and the read(2) may happen when there
> >> are
> >> multiple packets in the buffer already.
> >
> > Got you. I was wondering that, as it seemed that the data was being
> > buffered somewhere, hence the reason I kept testing my code further
> > and
> > further back, but to no avail.
>
> Yes, its buffered in TCP, its buffered in squid, its processed in async
> chunk in squid (adaptation means a few more buffers to transit) and its
> buffered on write, then its buffered in TCP again.
> Each of these steps has a 'random' asynchronous delay as well as
> possibly aggregating (parser, chunking) and/or splitting (de-chunking,
> adaptation) the data size.
>
> Overall, I'm a little mystified as to why you need per-packet marks at
> all. Some real-time QoS or such? That is all simply outside of Squids
> scope of operation.

I'm doing both routing and traffic shaping with the mark value.
Different bits in the mark value affect each of them. I only want the
routing bits present in the connection to the server, otherwise they
mess up the routing to the client connection. Routing is done before the
packets have hit iptables, so it's too late to change the mark value
there.

It soon became apparent to me that trying to achieve the above by
constantly changing the connection mark value was stupid, and that a
better way was to mask the bits instead on the outbound packet, hence
the patch I have just submitted.

However, having looked into it, I thought that obtaining the mark value
could be better anyway, so I thought I would work on that as well.

> Squid working with the HTTP per-request metric focus does make a lot
> more sense normally to grab the marks at or just after significant MiME
> boundaries. Before request-line, before headers, before body, and after
> body. Possibly at chunk boundaries.

Okay, I'll take your recommendation on that. I don't suppose there is
any particular single function that I can target, but instead that it
needs to be done within each of those?

Thanks,

Andy
Received on Tue Nov 01 2011 - 00:09:33 MDT