Re: [squid-users] Squid 3.0 Stable1 with MySql Logging

Cassiano Martin ha scritto:
> Hi Marcello!
>
> I'im interested in rewrite my squid importer to use this new feature, it
> will be much
> faster, as I see. How squid behaves when there is a large flow of data
> between squid and
> the log reader daemon? I wrote a Squid quota daemon, but it reads the
> log from the
> text file, and updates a mysql table. Its fast, but when there is a lot
> of data, it takes some
> time.. specially when updating the quota table, some users can go over
> their limit.
>
> Sorry about asking this, but could you show me the socket part of your
> code?
> I have not searched about this, yes I'm lazy :-) and I dont have a newer
> squid version
> to test.
>
> Thanks!
> Cassiano Martin
>
> Marcello Romani wrote:
>> Marcello Romani ha scritto:
>>> Adrian Chadd ha scritto:
>>>> On Mon, Feb 18, 2008, Marcello Romani wrote:
>>>>
>>>>> Hi, I have some experience in Perl and mysql. I can't guarantee a
>>>>> timely implementation, but I'm interested in this feature and I'm
>>>>> willing to contribute.
>>>>>
>>>>> Could you give me some pointers for where to start ?
>>>>
>>>> look for "logfile_daemon" in squid-2.7 snapshots. Its relatively
>>>> easy from
>>>> there.
>>>>
>>>>
>>>>
>>>> Adrian
>>>>
>>>
>>> I've written a very small perl script which reads stdin and dumps it
>>> to a text file. I've set it to be the logfile_daemon, but it seems it
>>> doesn't get called.
>>>
>>> I've not found much info on this configuration directive.
>>>
>>> I must be missing something stupid...
>>>
>>> Can you help me ? Thanks
>>>
>>
>>
>> Ok I found it. I think the comments in the squid.conf file should
>> contain a paragraph like the following:
>>
>> # And priority could be any of:
>> # err, warning, notice, info, debug.
>> #
>> # To log the request via a logfile writing daemon, specify a
>> filepath of "daemon", e.g.:
>> # access_log daemon:/var/logs/access.log squid
>> # You may also want to change the default value of logfile_daemon to
>> point to
>> # a custom logfile-writing daemon
>>
>> HTH
>>
>
>

There is no "socket part" of any code. I'll post my logfile daemon
implementation in perl as soon as I clean it up a little (right now it's
very primitive and virtually non-configurable, but it works ;-). In the
meanwhile, here's what I did:

1) download squid source code for the latest 2.7 series;
2) look at src/logfile-daemon.c to learn how a logfile daemon should
work (it's actually quite simple)
3) write a script which continuously reads from stdin and does something
with each line it reads (the first byte is the command, the rest is the
actual log line), e.g. prints it to a file or splits it up in fields and
sotres them in a mysql table :-)
4) configure --prefix=/somewhere/in/your/home/dir, then make && make
install, and apply the config i mentioned in my previous post.

After this, you won't be able to call yourself lazy anymore :-)

-- 
Marcello Romani
Responsabile IT
Ottotecnica s.r.l.
http://www.ottotecnica.com