Re: [squid-users] rock questions ?

On 10/23/2013 03:56 AM, Omid Kosari wrote:
> I am using rock on one of SSD drives to check its performance .
>
>
> before choosing rock the filesystem was reiserfs because it shows good
> performance in huge number of little files but i read somewhere the rock
> uses one big file . so i choose ext4 with discard to got the benefits of
> discard/trim .
>
> the rock creates one file with the name rock in /cache2 directory with the
> size 105906MB from begin and still it has same size . but "df " shows 27%
> used and grows each day .

This is normal. The db file is initialized using ftruncate(). Until the
db file is filled with content, there will be a difference between the
size of the db file shown by "ls" and the amount of space actually
occupied by that file on disk (shown by "df").

> 1. is the max-size=31000 is the maximum size rock may store ?

Max-size is the maximum size of a single cache_dir entry. In SMP
environment (i.e. multiple workers and/or multiple Rock diskers), the
maximum usable max-size value for official Rock implementations is
approximately 32752 bytes (32KB - sizeof(Rock::DbCellHeader), to be
precise).

This ~32KB limit comes from SMP sharing requirements. It is eliminated
by the Large Rock project:
http://wiki.squid-cache.org/Features/LargeRockStore

I do not think there is currently any diagnostic that will warn you that
your max-size exceeds the largest supported value. This a bug,
complicated by the fact that the largest supported value depends on
whether your configuration uses diskers, which, in turn, is a
non-trivial question to answer in general. If you use 32000 or a similar
value, you should be OK regardless of these complications.

> 2. is my fstab line shows bellow optimal for rock ? i have created it with
> "mkfs.ext4 /dev/sdb" on OCZ-VERTEX3 120GB

As discussed further below, I would not be able to answer that question
anyway, but please note that you are posting from a Nable.com interface
that eats all quoted lines such as your "fstab lines shown below". We do
not know what you actually posted because of that nibbling by Nable
software outside our control.

[ Rant: I wonder if this mailing list should start rejecting all Nable
posts both because those posts often come with critical information
stripped and because they lack context? Perhaps somebody who likes Nable
can approach Nabble.com and ask them to at least fix their nibbling bugs?]

> 3. it seems the the disk is more idle since switched to rock . how to force
> it use the disk more aggressive ?

You may need to define "more aggressive" for the above question to be
clearer.

In general, Squid will cache everything that HTTP rules and your
configuration allow it to cache. If you want to cache more, you either
need more traffic or a more "permissive" configuration. Please keep in
mind that max-size is one of those configuration limits. Due to this
size limit, an official SMP Rock cache_dir will not cache [large]
objects that a UFS-based cache_dir without max-size will cache.

However, one should always start from higher level metrics that are
important to one's deployment environment (response time, bandwidth
savings, robustness, etc.). Making sure that Squid caches everything is
often (but not always) the wrong optimization approach.

> 4. the following is output of dumpe2fs of that drive . is it better to
> resize the block size to 32KB which is the size of rock store size ? any
> other suggestion for filesystem ?
>
> dumpe2fs /dev/sdb | more

Same "Nable nibbling at your post" problem as discussed in question #2.

In my experience, preventing disk I/O queue overflows (and resulting
blocking of Squid worker processes) is the key here. There are many
parameters that affect that, especially parameters that deal with
OS-initiated flushing of disk write buffers. The optimal settings are
going to depend on your specific environment.

Will 32KB block size be always better than 16KB? Hard to say. In an
average environment, more than half of the stored objects are going to
be smaller than 16KB so it may not make sense to force the OS to load
all 32KB all the time (if that is what 32KB fs blocks will do).

Similarly, is ext4 better than ext2 or Reiser for Rock? I personally
find ext2 easier to tune correctly but YMMV.

For more specific (but far from comprehensive!) advice, please see
"Performance Tuning" at http://wiki.squid-cache.org/Features/RockStore

HTH,

Alex.
Received on Wed Oct 23 2013 - 15:38:05 MDT