profiling information from the wikipedia squids

Here's some profiling information from one of the Wikipedia squids:

CPU: AMD64 processors, speed 2009.3 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 50000
samples % image name symbol name
29968423 53.3123 no-vmlinux (no symbols)
2302640 4.0963 libc-2.4.so memcpy
1667371 2.9662 libc-2.4.so memset
1368925 2.4352 squid storeKeyHashCmp
786430 1.3990 libtcmalloc.so.0.0.0 (no symbols)
730871 1.3002 libc-2.4.so re_search_internal
719279 1.2796 libc-2.4.so memchr
641089 1.1405 squid memPoolAlloc
554176 0.9859 libc-2.4.so vfprintf
540236 0.9611 libc-2.4.so strlen
497282 0.8846 libc-2.4.so _IO_vfscanf
481931 0.8573 squid httpHeaderIdByName
442834 0.7878 squid comm_select
378810 0.6739 squid rfc1738_do_escape
349059 0.6210 squid memPoolFree
276477 0.4918 libm-2.4.so floor
273867 0.4872 squid headersEnd

There's more where that came from. I'll put it up at
http://www.creative.net.au/diffs/20070129-wiki-oprofile-1.txt .
That'll include the somewhat confusing call graph available from oprofile.

I'll keep on with my plans to introduce ref-counted buffers and buffer ranges
to Squid-2 and start using them for the http header entries and request buffers.
We'll see if it cuts back on the amount of code thats involved in the http request
and reply path. There's still a hell of a lot though..

I'd also like to augment storeClientCopy with a version that simply returns a
reference to a storage page, so the client-side can write that directly out to
the FD. Any comments/suggestions?

Adrian

Squid Object Cache: Version 2.HEAD-CVS
Start Time: Mon, 29 Jan 2007 09:54:56 GMT
Current Time: Mon, 29 Jan 2007 13:47:43 GMT
Connection information for squid:
        Number of clients accessing cache: 0
        Number of HTTP requests received: 2662113
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Number of HTCP messages received: 6527400
        Number of HTCP messages sent: 6070169
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 11436.4
        Average ICP messages per minute since start: 0.0
        Select loop called: 114474042 times, 0.122 ms avg
Cache information for squid:
        Request Hit Ratios: 5min: 71.0%, 60min: 71.1%
        Byte Hit Ratios: 5min: 73.9%, 60min: 74.0%
        Request Memory Hit Ratios: 5min: 75.5%, 60min: 75.3%
        Request Disk Hit Ratios: 5min: 18.3%, 60min: 18.6%
        Storage Swap size: 4358856 KB
        Storage Mem size: 1573044 KB
        Mean Object Size: 10.11 KB
        Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 0.00000 0.00000
        Cache Misses: 0.58309 0.58309
        Cache Hits: 0.00000 0.00000
        Near Hits: 0.55240 0.49576
        Not-Modified Replies: 0.00000 0.00000
        DNS Lookups: 0.00000 0.00278
        ICP Queries: 0.00041 0.00039
Resource usage for squid:
        UP Time: 13966.500 seconds
        CPU Time: 1526.440 seconds
        CPU Usage: 10.93%
        CPU Usage, 5 minute avg: 10.94%
        CPU Usage, 60 minute avg: 11.39%
        Process Data Segment Size via sbrk(): 1034096 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
        Total space in arena: -1850060 KB
        Ordinary blocks: -1864261 KB 0 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 0 KB 0 blks
        Free Small blocks: 7616 KB
        Free Ordinary blocks: 6584 KB
        Total in use: -1864261 KB 101%
        Total free: 14200 KB 0%
        Total size: -1850060 KB
Memory accounted for:
        Total accounted: 1891951 KB
        memPoolAlloc calls: 443755928
        memPoolFree calls: 437980679
File descriptor usage for squid:
        Maximum number of file descriptors: 65536
        Largest file desc currently in use: 3597
        Number of file desc currently in use: 3551
        Files queued for open: 0
        Available number of file descriptors: 61985
        Reserved number of file descriptors: 100
        Store Disk files open: 0
        IO loop method: epoll
Internal Data Structures:
        431207 StoreEntries
        153550 StoreEntries with MemObjects
        153476 Hot Object Cache Items
        431110 on-disk objects
Received on Mon Jan 29 2007 - 06:55:18 MST