Re: [squid-users] tuning an overloaded server

Gah, they way they work is really quite simple.

* ufs does the disk io at the time the request happens. It used to try
using select/poll on the disk fds from what I can gather in the deep,
deep dark history of CVS but that was probably so the disk io happened
in the next IO loop so recursion was avoided.

* aufs operations push requests into a global queue which are then
dequeued by the aio helper threads as they become free. The aio helper
threads do the particular operation (open, close, read, write, unlink)
and then push the results into a queue so the main squid thread can
handle the callbacks at a later time.

* diskd operations push requests into a per storedir queue which is
then dequeued in order, one operation at a time, by the diskd helper.
The diskd helper does the normal IO operations (open, close, read,
write, unlink) and holds all the disk filedescriptors (ie, the main
squid process doesn't hold open the disk FDs; they're just given
"handles".) The diskd processes do the operation and then queue the
result back to the main squid process which handles the callbacks at a
later time.

AUFS works great where the system threads allow for concurrent
blocking syscalls. This meant Linux (linuxthreads being just
processes) and Solaris in particular worked great. The BSDs used
userland threads via a threading library which "wrapped" syscalls to
try and be non-blocking. This wouldn't work for disk operations and so
a disk operation stalled all threads in a given process. diskd, as far
as I can gather (duane would know better!) came into existance to
solve a particular problem or two, and one of those problems was the
lack of scalable disk IO available in the BSDs.

FreeBSD in particular has since grown a "real" threading library which
supports disk IO happening across threads quite fine.

The -big- difference right now is how the various disk buffer cache
and VM systems handle IO. By default, the AUFS support in Squid only
uses the aio helper threads for a small subset of the operations. This
may work great under Linux but operations such as write() and close()
block under FreeBSD (think 'writing out metadata', for example) and
this mostly gives rise to the notion of Linux "being better" by most
people who haven't studied the problem in depth. :)

hope that helps,

Adrian

2008/11/27 Amos Jeffries <squid3_at_treenet.co.nz>:
> B. Cook wrote:
>>
>> On Nov 22, 2008, at 7:30 AM, Amos Jeffries wrote:
>>
>> 8< -- snip -- >8
>>
>>>>
>>>
>>> That said BSD family of systems get more out of diskd than aufs in
>>> current Squid.
>>>
>>>>>
>>> --
>>> Please be using
>>> Current Stable Squid 2.7.STABLE5 or 3.0.STABLE10
>>> Current Beta Squid 3.1.0.2
>>
>> Hello,
>>
>> Sorry to bother..
>>
>> so even in any FreeBSD (6.3, 7.0, etc..) diskd is still better than aufs?
>>
>> and if so,
>>
>> http://wiki.squid-cache.org/Features/DiskDaemon
>>
>> this page talks about 2.4
>>
>> and I can't seem to find an aufs page.. I can find coss, but coss has been
>> removed from 3.0..
>>
>> so again, diskd should be what FreeBSD users use? As well as the kernel
>> additions? Even on 6.3 and 7.0 machines amd64 and i386 alike?
>
> Yes. We have some circumstantial info that leads to believe its probably a
> bug in the way Squid uses AUFS and the underlying implementation differences
> in FreeBSD and Linux. We have not yet had anyone investigate deeply and
> correct the issue. So its still there in all Squid releases.
>
>
>>
>> Thanks in advance..
>>
>> (I would think a wiki page on an OS would be very useful.. common configs
>> for linux 2.x and BSD, etc.. )
>>
>> Many people are not as versed in squid as the developers, and giving them
>> guidelines to follow would probably make it easier for them to use.. imho.
>>
>> They don't understand coss vs aufs vs diskd vs ufs.. ;)
>
> We are trying to get there :). It's hard for just a few people and
> non-experts in many areas at that. So if anyone has good knowledge of how
> AUFS works jump in with a feature page analysis.
>
> What we have so far in the way of config help is explained at
> http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#head-ad11ea76c4876a92aa1cf8fb395e7efd3e1993d5
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE5 or 3.0.STABLE10
> Current Beta Squid 3.1.0.2
>
>
Received on Fri Nov 28 2008 - 04:10:16 MST