Re: Improving qos_flows mark feature - obtaining mark later

 On Mon, 31 Oct 2011 22:19:12 +0000, Andrew Beverley wrote:
> On Mon, 2011-10-31 at 15:42 -0600, Alex Rousskov wrote:
>> On 10/31/2011 03:03 PM, Andrew Beverley wrote:
>>
>> > Having thought about this further, I think what I was trying to
>> achieve
>> > was getting the mark every time a packet was received.
>>
>>
>> > I have tried putting the relevant code into
>> > ServerStateData::addVirginReplyBody() which seems to work for
>> different
>> > protocols as well as being called for each packet received.
>> >
>> > Is there any problems with this approach? If not, I will refine
>> and
>> > submit a patch.
>>
>> Yes, if you do not care about HTTP response header or FTP control
>> messages: ServerStateData::addVirginReplyBody() is not called when
>> the
>> HTTP header or FTP control messages are received. If you want to
>> catch
>> those cases (it sounds like you do based on your "every packet"
>> description above), then you need HttpStateData::readReply() and
>> equivalent read handlers in other protocols.
>
> Ah, okay. Thanks for the explanation.
>
>> I do not think there is a single Server method that is always called
>> for
>> every Squid read. You should probably add it and call it from the
>> corresponding HTTP and FTP-specific code.
>
> I assume that I should add it to all the protocols? I may need some
> advice as to the correct functions within each one. Which one do you
> think for FTP? dataRead()? processReplyBody()?

 All the read handlers. With type IOCB or taking CommIoCb* handlers
 needs a loot at.

 If you really want it every packet you can't get any closer than
 commHandleRead() in comm.cc which is what schedules those handlers. The
 ccb->conn available in there is the connection descriptor being read.

>
>> Please keep in mind that Squid does not work on a TCP packet level.
>> It
>> uses the TCP socket interface and the read(2) may happen when there
>> are
>> multiple packets in the buffer already.
>
> Got you. I was wondering that, as it seemed that the data was being
> buffered somewhere, hence the reason I kept testing my code further
> and
> further back, but to no avail.

 Yes, its buffered in TCP, its buffered in squid, its processed in async
 chunk in squid (adaptation means a few more buffers to transit) and its
 buffered on write, then its buffered in TCP again.
  Each of these steps has a 'random' asynchronous delay as well as
 possibly aggregating (parser, chunking) and/or splitting (de-chunking,
 adaptation) the data size.

 Overall, I'm a little mystified as to why you need per-packet marks at
 all. Some real-time QoS or such? That is all simply outside of Squids
 scope of operation.

 Squid working with the HTTP per-request metric focus does make a lot
 more sense normally to grab the marks at or just after significant MiME
 boundaries. Before request-line, before headers, before body, and after
 body. Possibly at chunk boundaries.

 Amos
Received on Mon Oct 31 2011 - 23:12:11 MDT