Re: Squid and SSI

On 28.01.2012 09:46, goro goro wrote:
> Dear Sir(s)
>

Greetings and welcome.

>     I Am a system administrator at an ISP, Squid is at the Heart of
> it, Caching is important to me , as is Through Put.
>
> I have Been trying for months now , fiddiling with /Proc anyway I
> can, and with the squid.conf, also with the Limits.conf
>
> I finaly was able to reach a throughput of 80Mb which is not even
> close
> to what I really need, One of the Server's Reached 120Mb, but i wasnt
> able to know why!!!.

Mbps is not a good rating for HTTP software. One of the major limits is
parsing the requests and responses. One of those may have as few as 16
bytes or as large as 64 KB of headers to parse.

80+Mbps is pretty good traffic rate for an individual Squid. The better
real-world traffic rates we have seen mention of by multiple ISP have
usually been around the 50Mbps rates with peak traffic getting a bit
higher for specific situations.

>
> I tried fiddiling with Squid's Source code to remove any un-needed
> stuff, but it was too complicated for my programming knowldge.
>

Start with the features available for --disable using configure
options. That cuts out a lot of extra code checks to see if those
features are needed on each request. Beyond that we need someone to
profile the parse and processing code logics and streamline them. This
is an ongoing work item for all us dev.

> I hope you can help me with some of the questions I have, for I am
> desprate at this time to find a decent answer, The Web didnt really
> give me a stright answer.
>
> Please note that i Have an ACL RegEx of almost 2000 lines
>

regex is another speed bottleneck. Reducing this to dstdomain and other
faster ACL test types may improve the speed.

The recent 3.2 beta releases have some regex optimizations which should
improve that situation a lot, particularly on longs regex lists. But we
are still limited by regex testing libraries speeds.

>
> 1 : What are the main configuration parameters that have a major
> effect
> on through put ( in sysctl.conf , squid.conf , limits.conf and /proc
> )
>
> 2 : I am thinking of using OpenSSI to use its Load Levelling
> functionallity , But I am not sure that Squid will benift from it,
> Will
> it increase the Thoughput of my Squid or limit it ?

Depends on what criteria the load balancing is using.

You can safely and quickly balance new TCP connections between multiple
Squid instances with good speed increases linear to the number of Squid
available.

Delving down into packet contents for HTTP level switching will only
add speed overheads. We don't have any good measurements to say whether
the improved HIT ration that can lead to is worth the trouble of packet
inspection in the LB.

>
> 3 : I am looking for a patch that can Write the Access Log Directlly
> to
> Oracle or Mysql , unlike squid2Mysql , I dont want it to read the
> file Access.log and then write it, I need squid to write it to the DB
> directlly, Is that even possible ?

Squid 2.7 and 3.2 offer daemon logger helpers for fast processing of
log data outside of the main squid process to any backend logging
system. One of the daemon helpers publicly available does MySQL logging.
Script daemons can easily be created for any backend.
Be aware that database logging can be speed limited by the database
write speed, and uniqueness limits imposed by the database can drop data
or cause problems related to wrapping of line ID counters under high
traffic loads.

>
> 4 : Any way to speed up anything with Squid . I would really
> appreciate it.
>
> I wait for any answers.
>
> Thanks in advance.

You did not say which version of squid you are using right now. From
what you list as your requirements I believe you will benefit a lot from
the features in 3.2 beta releases.
  Unfortunately 3.2.0.14 is not quite up to production readiness yet.
But if you could test the latest daily bundle and help us determine what
bugs are affecting your installation worst it would help move towards a
stable release.

Hope this helps.
Amos
Received on Mon Jan 30 2012 - 23:12:31 MST