Re: store.log bloat possible fix?

That is because it was I who threw it there.

This section is a bit magic, and should perhaps be documented better.

the storeReleaseRequest takes care of unregistering the swap file
number, and the storeRelease actually releases the storeentry (or
something like that. Was after all about a year ago so my memory is a
bit dim on the details now...)

If you don't call storeRelease here, then any cancelled objects will
stay in memory. The full story should be available on my squid-2.2 patch
page. (if I remember correcly there was two problems in this area.. one
causing a huge temporary memory usage during rebuild, and one causing a
storeentry leak during rebuild..)

The whole procedure of storeLog while reqbuilds needs to be carefully
reviewed. The current design is flawed in more than one way.

* Amount of entries logged
* How to deal with crashes/restarts during the rebuild

My proposal is to not log actions from the rebuild, but keep logging
normal actions to the current swap.state file. When the rebuild is
completed (original end of swap.state found) then write out a clean log
file.

/Henrik

Adrian Chadd wrote:
>
> I forget the reasons I threw a storeRelease() in there, but from my
> quick testing locally its not needed as above it is a storeReleaseRequest().
> Can someone actually tackle this and make sure this doesn't screw up
> rebuilds ?
>
> The trouble with calling storeRelease() is that it calls storeLog() and
> whilst rebuilding it causes store.log to be VERY full.
>
> Comments?
>
> Adrian
>
> Index: fs/aufs/store_dir_aufs.c
> ===================================================================
> RCS file: /server/cvs-server/squid/squid/src/fs/aufs/store_dir_aufs.c,v
> retrieving revision 1.7
> diff -u -r1.7 store_dir_aufs.c
> --- fs/aufs/store_dir_aufs.c 2000/06/27 22:06:23 1.7
> +++ fs/aufs/store_dir_aufs.c 2000/10/04 05:11:42
> @@ -566,7 +566,6 @@
> e->swap_filen = -1;
> e->swap_dirn = -1;
> }
> - storeRelease(e);
> rb->counts.objcount--;
> rb->counts.cancelcount++;
> }
> Index: fs/diskd/store_dir_diskd.c
> ===================================================================
> RCS file: /server/cvs-server/squid/squid/src/fs/diskd/store_dir_diskd.c,v
> retrieving revision 1.17
> diff -u -r1.17 store_dir_diskd.c
> --- fs/diskd/store_dir_diskd.c 2000/10/03 15:31:40 1.17
> +++ fs/diskd/store_dir_diskd.c 2000/10/04 05:11:47
> @@ -754,7 +754,6 @@
> e->swap_filen = -1;
> e->swap_dirn = -1;
> }
> - storeRelease(e);
> rb->counts.objcount--;
> rb->counts.cancelcount++;
> }
> Index: fs/ufs/store_dir_ufs.c
> ===================================================================
> RCS file: /server/cvs-server/squid/squid/src/fs/ufs/store_dir_ufs.c,v
> retrieving revision 1.7
> diff -u -r1.7 store_dir_ufs.c
> --- fs/ufs/store_dir_ufs.c 2000/06/27 22:06:26 1.7
> +++ fs/ufs/store_dir_ufs.c 2000/10/04 05:11:50
> @@ -566,7 +566,6 @@
> e->swap_filen = -1;
> e->swap_dirn = -1;
> }
> - storeRelease(e);
> rb->counts.objcount--;
> rb->counts.cancelcount++;
> }
>
> ----- End forwarded message -----
>
> --
> Adrian Chadd "If a butterfly flaps its wings in China,
> <adrian@creative.net.au> will a women get naked in Amsterdam?"
> -- Ashley Penney on Chaos Theory
Received on Wed Oct 04 2000 - 01:35:05 MDT