Re: [squid-users] Not able to apply maximum_object_size_in_memory

 On Mon, 21 Feb 2011 11:52:11 -0500, John Craws wrote:
> Hi,
>
> Thank you for the clarification. Maybe I'm just not correctly
> interpreting whether the object is in the cache or not.
> Here's the info you asked for, based on the config I posted
> previously. I'm downloading a +- 16M file.
>
> 1. Before downloading the object:
>
> john.craws@jjj:~/wget$ curl -I http://172.16.199.150/popeye.mp4
> HTTP/1.0 200 OK
> Date: Mon, 21 Feb 2011 16:46:56 GMT
> Server: Apache/2.2.3 (Red Hat)
> Last-Modified: Thu, 24 Sep 2009 19:22:32 GMT
> ETag: "e2800c-1013726-47457c0c5ae00"
> Accept-Ranges: bytes
> Content-Length: 16856870
> Content-Type: video/mp4
> X-Cache: MISS from jnk
> Via: 1.0 jnk (squid/3.1.11)
> Connection: keep-alive
>
>
> john.craws_at_jjj:~/wget$ /opt/squid/bin/squidclient mgr:objects
> HTTP/1.0 200 OK
> Server: squid/3.1.11
> Mime-Version: 1.0
> Date: Mon, 21 Feb 2011 15:51:02 GMT
> Content-Type: text/plain
> Expires: Mon, 21 Feb 2011 15:51:02 GMT
> Last-Modified: Mon, 21 Feb 2011 15:51:02 GMT
> X-Cache: MISS from jnk
> Via: 1.0 jnk (squid/3.1.11)
> Connection: close
>
> (lists cached objects, no trace of the object -- normal).
>
> 2. Downloading the object
>
> john.craws@jjj:~/wget$ curl http://172.16.199.150/popeye.mp4 -o
> popeye.mp4
> % Total % Received % Xferd Average Speed Time Time
> Time Current
> Dload Upload Total Spent
> Left Speed
> 100 16.0M 100 16.0M 0 0 2446k 0 0:00:06 0:00:06
> --:--:-- 1634k
>
> 3. This time the object appears in the list
>
> john.craws_at_jjj:~/wget$ /opt/squid/bin/squidclient mgr:objects
> HTTP/1.0 200 OK
> Server: squid/3.1.11
> Mime-Version: 1.0
> Date: Mon, 21 Feb 2011 15:51:18 GMT
> Content-Type: text/plain
> Expires: Mon, 21 Feb 2011 15:51:18 GMT
> Last-Modified: Mon, 21 Feb 2011 15:51:18 GMT
> X-Cache: MISS from jnk
> Via: 1.0 jnk (squid/3.1.11)
> Connection: close
>
> (...)
> KEY 669AB801B7640FA80E4BA73193FDAC2A
> STORE_OK IN_MEMORY SWAPOUT_NONE PING_DONE
> CACHABLE,DISPATCHED,VALIDATED
> LV:1298303469 LU:1298303469 LM:1253820152 EX:-1
> 0 locks, 0 clients, 1 refs
> Swap Dir -1, File 0XFFFFFFFF
> GET http://172.16.199.150/popeye.mp4
> inmem_lo: 0
> inmem_hi: 16857134
> swapout: 0 bytes queued
> (...)
>
>
> 4. This time it's a HIT
>
> john.craws@jjj:~/wget$ curl -I http://172.16.199.150/popeye.mp4
> HTTP/1.0 200 OK
> Date: Mon, 21 Feb 2011 15:51:09 GMT
> Server: Apache/2.2.3 (Red Hat)
> Last-Modified: Thu, 24 Sep 2009 19:22:32 GMT
> ETag: "e2800c-1013726-47457c0c5ae00"
> Accept-Ranges: bytes
> Content-Length: 16856870
> Content-Type: video/mp4
> Age: 16
> X-Cache: HIT from jnk
> Via: 1.0 jnk (squid/3.1.11)
> Connection: keep-alive
>
> 5. access.log
>
> 1298304218.303 19 127.0.0.1 TCP_MISS/200 322 HEAD
> http://172.16.199.150/popeye.mp4 - DIRECT/172.16.199.150 video/mp4
> 1298304222.694 0 127.0.0.1 TCP_MEM_HIT/200 329 HEAD
> http://172.16.199.150/popeye.mp4 - NONE/- video/mp4
>
> I also notice the difference the major time difference between the
> two
> curl operations.
>

 Thanks for the detail. You are right about it being in the memory
 cache.

 What I expect to see with your config is that file pushed to disk,
 since it is within the 17MB but over the 32KB. But you have no on-disk
 cache right?

 Something funky is going on with the swapout.

 I think there are 2 bugs visible here, the easy one is that the config
 parser is not detecting and warning about the global limit being larger
 than the biggest specific limit. Second being the object not discarded
 when over memory size and push to disk not possible.

 Amos

> Thanks!
>
> John
>
>
> On Fri, Feb 18, 2011 at 5:56 PM, Amos Jeffries <squid3_at_treenet.co.nz>
> wrote:
>> On 19/02/11 07:28, John Craws wrote:
>>>
>>> Hi,
>>>
>>> I have a squid 3.1.11 instance configured with no disk cache.
>>> Stripped down configuration below.
>>>
>>>
>>>
>>> #--------------------------------------------------------------------------------
>>> # squid.conf
>>>
>>>
>>> #--------------------------------------------------------------------------------
>>>
>>> shutdown_lifetime 0 seconds
>>> http_port 3128
>>> http_access allow all
>>> forwarded_for transparent
>>>
>>> acl VIDEO-CONTENT           rep_header Content-Type video/.+
>>>
>>> maximum_object_size_in_memory 32 KB
>>> maximum_object_size 17 MB
>>> cache_mem 4 GB
>>> cache allow all
>>> debug_options ALL,1
>>>
>>>
>>>
>>> #--------------------------------------------------------------------------------
>>> # end squid.conf
>>>
>>>
>>> #--------------------------------------------------------------------------------
>>>
>>> I use curl to request a file through the proxy.
>>>
>>> What I understood from the docs is that maximum_object_size limits
>>> the
>>> size of objects to disk cache and maximum_object_size_in_memory
>>> does
>>> so for RAM cache.
>>>
>>> I have no disk cache, so I tried to apply a limit using
>>> maximum_object_size_in_memory. It didn't work. However, it does
>>> work
>>> when I specify a limit using maximum_object_size.
>>>
>>> What am I doing wrong?
>>
>> You are almost correct.
>>  maximum_object_size is a global limit which nothing stored anywhere
>> can
>> evade.
>>  maximum_object_size_in_memory is a RAM-only limit. The smaller of
>> memory
>> and global limits is used for cache_mem.
>>
>>  There are matching *-size limits on individual cache_dir in the
>> latest
>> Squid that do the same for each disk dir.
>>
>>
>> It should be preventing long-term storage of anything over 32KB,
>> provided
>> the size info is known in the headers.
>>  Note that this does not limit objects which are currently in
>> transit. They
>> are stored for as long as that transit use needs, then discarded.
>>
>> Do you have a copy of the headers and a display of how you are
>> identifying
>> the failure please?
>>
>> Amos
>> --
>> Please be using
>>  Current Stable Squid 2.7.STABLE9 or 3.1.11
>>  Beta testers wanted for 3.2.0.5
>>
Received on Mon Feb 21 2011 - 22:37:14 MST