Re: [Fwd: pls give me some suggestion!]

Wen,

(all, your help in clearing up my own misconceptions is appreciated)

I'm not sure of your level of awareness of current kernel space web
server implementations. In some respects, these bear many similarities
to the style of functionlity required by a proxy server as well. The
current implementation in Linux kernel v2.4 is khttpd. I don't believe
this is in active development any longer, rather the focus is on 'Tux'.
You can read about Tux from its author on this slashdot article:
http://slashdot.org/articles/00/07/20/1440204.shtml.

http://kt.linuxcare.com/kernel-traffic/kt20000807_79.epl#7 suggests that
putting some stuff inside the kernel might offer a different way to do
asynchronous I/O on Linux.

I would ask a few questions up font about what might be achieved by
kernel space caching. Historically I think that the biggest area of
performance difficulties with Squid (and caches in general?) has been
disk subsystems, in particular seek time across platters. However if you
consider the possibility of perhaps only handling requests from RAM then
maybe things are a little different... In order to see some stuff where
ticks are currently spent have a look at Andre's profiling work at
http://www.online.ee/~andre/squid/profil/

In the tests that Tux performed so well on, the fileset that would could
in some respect associate with the currently cached set was in the order
of 10GB, the same order of magnitude as the amount of memory - for the
purposes of a cache equivalent you might possibly just drop the idea of
persistant storage and go with memory only caching. In this case current
RAM prices are in the order of $100US / GB? (c.f. maybe $5US/GB? for
disk) . The only problem is that most boards won't take too many
sticks... Anyway, you might see the RAM cost curve continuing to drive
down and density going up quite quickly. If you can get 10GB RAM into a
'low end' (64 bit address space) box in a few years time, then you might
actually be able to see not unreasonable cache hit rates on this type of
box - and of course response time and throughput out of the machine
should be superb.

In this eventuality the benefits of having kernel mode implementation of
large portions of the proxy server are then exactly as described in the
Tux article - lower 'system call' costs due to no context switch, more
efficient dispatching of incoming connections.

The really interesting thing is though: what gains could be gotten with
the current _squid_ codebase in situations where it has quite a small
disk store in comparison to available memory if the existing comm.c API
(which I know just a little about) and the disk subsystem (which I know
nothing about) could be hooked into the Tux framework? Have a look at
the programmers guide and also
http://devel.squid-cache.org/projects.html#tcp_async and
http://devel.squid-cache.org/eventio/.

One final issue I throw in the grab bag is the assumptions that Tux may
make about threading could well be all wrong for Squid and caching in
general.

Roger.

Henrik Nordstrom wrote:

>
>-------- Original Message --------
>Subject: pls give me some suggestion!
>Date: Mon, 15 Oct 2001 22:22:13 +0800
>From: wen xiulin <xlwen@uestc.edu.cn>
>To: "hno@squid-cache.org" <hno@squid-cache.org>
>
>Mr Henrik:
>hi,
>Recently i read the source code Squid, thanks for so many pepole working
>to do this.
>i want to implementation a Squid like proxy server under linux(of
>course, a simpler one without so many functions such as Hierarchical
>Caching).the main idea is that i want put the whole or part of the
>program into kernel-- work as linux kernel module.
>
>but i have no idea of that, can it really improvment the server's
>performance by that?
>and how difficult would it be to do that?
>
>it would very appreciate of you to read this letter and have some
>suggestion to me!
>
>wish everything going well with you.
>
>
> wen xiulin
>
Received on Tue Oct 16 2001 - 06:54:57 MDT