Re: [squid-users] Squid with Perl script are not working

fadi.sbat wrote:
> Hello,
>
>
> I am trying to Cache Youtube Videos with my SquidNT 2.7STABLE6 but there is nothing working ..
>
I also installed strawberry-perl-5.10.0.4.exe .. I wish if anyone can
help it .
>

SquidNT? no such thing any more. Not since early Squid 2.6 integrated
Windows support. There are a few fakes of extremely dubious origin
floating around under that banner though.

Please ensure you have the official Squid binary built for windows
either by yourself or http://squid.acmeconsulting.it/. They are the
_only_ third-party supplier of Windows Squid binaries currently known
and accepted by the Squid project team.

Also for your youtube rules, check the youtube caching discussion page.
It's being updated all the time as youtube change their systems.
It seems to have a newer set of config and rewriter than you are using.
http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussion

Amos
Squid-3 Maintainer

>
> Here is my squid.conf :
> ----------------------
> # ACCESS CONTROLS
> acl all src all
> acl manager proto cache_object
> acl localhost src 127.0.0.1/255.255.255.255
> acl to_localhost dst 127.0.0.0/8
> acl admin src 192.168.1.65/255.255.255.255
> acl our_networks src 192.168.5.0/24
> acl SSL_ports port 443 563 # https, snews
> acl SSL_ports port 873 # rsync
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 563 # https, snews
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 5004 #
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl Safe_ports port 631 # cups
> acl Safe_ports port 873 # rsync
> acl Safe_ports port 901 # SWAT
> acl CONNECT method CONNECT
> acl YMPort port 5050
> acl youtube dstdomain .youtube.com
> acl PURGE method purge
>
> # TAG: http_access
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow our_networks
> http_access allow localhost
> http_access deny all
> icp_access allow all
> icp_access deny all
>
> # TAG: http_port
> http_port 3128 transparent
>
> cache_mem 1000 MB
> maximum_object_size_in_memory 64 KB
> memory_replacement_policy heap GDSF
> cache_replacement_policy heap LFUDA
> cache_dir ufs E:/SquidCache/cache 60000 64 256
> maximum_object_size 300 MB
> #half_closed_clients off
> server_persistent_connections off
> #client_persistent_connections off
> cache_swap_low 98
> cache_swap_high 99
>
> #peer parent
> #cache_peer random.us.ircache.net parent 3128 3130 login=tawonxx_xx_at_yahoo.co.id:xxxxxxxBicbin
>
> # LOGFILE OPTIONS
> access_log c:/squid/var/logs/access.log squid
> cache_log c:/squid/var/logs/cache.log
> cache_store_log none
> logfile_rotate 2
> emulate_httpd_log off
> log_fqdn off
> ftp_passive on
> storeurl_rewrite_program C:/strawberry/perl/bin/perl.exe c:/squid/etc/store_url_rewrite.pl
> acl store_rewrite_list url_regex \/(get_video\?|videodownload\?|videoplayback\?id) \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv)\? \/ads\?
> acl store_rewrite_list url_regex ^http://(.*?)/get_video\?
> acl store_rewrite_list url_regex ^http://(.*?)/watch\?
> acl store_rewrite_list url_regex ^http://(.*?)/videodownload\?
> acl store_rewrite_list url_regex ^http://i(.*?).photobucket.com/albums/(.*?)/(.*?)/(.*?)\?
> acl store_rewrite_list url_regex ^http://vid(.*?).photobucket.com/albums/(.*?)/(.*?)\?
> acl store_rewrite_list url_regex ^http://.*?.\files\.youporn\.com/.*?/.*?/.*?\.flv\?.*
> acl store_rewrite_list url_regex mt.google.com mt0.google.com mt1.google.com mt2.google.com
> acl store_rewrite_list url_regex mt3.google.com
> acl store_rewrite_list url_regex kh.google.com kh0.google.com kh1.google.com kh2.google.com
> acl store_rewrite_list url_regex kh3.google.com
> acl store_rewrite_list url_regex kh.google.com.au kh0.google.com.au kh1.google.com.au
> acl store_rewrite_list url_regex kh2.google.com.au kh3.google.com.au
>
> acl store_rewrite_list url_regex ^http:\/\/([A-Za-z-]+[0-9]+)*\.[A-Za-z]*\.[A-Za-z]*
> acl store_rewrite_list url_regex ^http:\/\/[a-z]+[0-9]\.google\.com doubleclick\.net
>
> # This needs to be narrowed down quite a bit!
> acl store_rewrite_list url_regex .youtube.com
>
> acl youtube_query url_regex -i \.youtube\.com\/get_video
> acl youtube_query url_regex -i \.youtube\.com\/watch
> acl youtube_query url_regex -i \.cache[a-z0-9]?[a-z0-9]?[a-z0-9]?\.googlevideo\.com\/videoplayback
> acl youtube_query url_regex -i \.cache[a-z0-9]?[a-z0-9]?[a-z0-9]?\.googlevideo\.com\/get_video
> acl youtube_query url_regex -i \.cache[a-z0-9]?[a-z0-9]?[a-z0-9]?\.googlevideo\.com\/watch
> acl youtube_deny url_regex -i http:\/\/[a-z][a-z]\.youtube\.com
> acl metacafe_query url_regex v.mccont.com
> acl dailymotion_query url_regex -i proxy\-[0-9][0-9]\.dailymotion\.com\/
> acl google_query url_regex vp.video.google.com
> acl redtube_query url_regex dl.redtube.com
> acl xtube_query url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\.xtube\.com\/(.*)flv
> acl vimeo_query url_regex -i bitcast\.vimeo\.com\/vimeo\/videos\/
> acl wrzuta_query url_regex -i va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
> url_rewrite_access deny youtube_deny
> url_rewrite_access allow youtube_query
> url_rewrite_access allow metacafe_query
> url_rewrite_access allow dailymotion_query
> url_rewrite_access allow google_query
> url_rewrite_access allow redtube_query
> url_rewrite_access allow xtube_query
> url_rewrite_access allow vimeo_query
> url_rewrite_access allow wrzuta_query
>
>
> cache allow youtube
> cache allow store_rewrite_list
> cache allow all
>
> acl Facebook urlpath_regex .facebook.com
> acl Ghaneli urlpath_regex .ghaneli.net
> acl Nogomi urlpath_regex .nogomi.com
> acl FBCDN urlpath_regex .fbcdn.net
> acl TAGSTAT urlpath_regex .tagstat.com
> acl Yahoo urlpath_regex .yahoo.com
> acl TBNGoogle urlpath_regex tbn3.google.com
> acl TBNGoogle urlpath_regex tbn2.google.com
> acl TBNGoogle urlpath_regex tbn1.google.com
> acl TBNGoogle urlpath_regex tbn0.google.com
> acl BABEL urlpath_regex .babel.com
> cache allow BABEL
> cache allow Ghaneli
> cache allow Nogomi
> cache allow TBNGoogle
> cache allow Yahoo
> cache allow TAGSTAT
> cache allow Facebook
> cache allow FBCDN
>
> #hierarchy_stoplist cgi-bin ?
> #acl QUERY urlpath_regex cgi-bin \?
> #cache deny QUERY
>
>
> refresh_pattern ^http://*.friendster.*/.* 720 100% 10080 override-expire override-lastmod reload-into-ims ignore-reload
> refresh_pattern ^http://*.swf.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.nogomi.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.ghaneli.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.fbcdn.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.snap.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.2mdn.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.z5x.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.harrenmedianetwork.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.scanmedios.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.adbrite.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.musicxposure.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.pkg.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.connextra.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.4shared.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.facebook.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.playfish.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.ak.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.google.*/.* 1440 100% 4320
> refresh_pattern ^http://*.akamai.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.youtube.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.salmiya.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.googlesyndication.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.yieldmanager.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.googlevideo.*/.* 999999 50000% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.dailymotion.*/.* 999999 50000% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.xtendmedia.*/.* 999999 50000% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.clipfish.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.youporn.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.redtube.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.ytimg.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://mail.yahoo.com/.* 1720 100% 10080
> refresh_pattern ^http://*.yahoo.*/.* 1720 100% 7200
> refresh_pattern ^http://*.google-analytics.*/.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
>
> refresh_pattern ^http://*.wordpress.com/.* 720 80% 10080
> refresh_pattern -i .google.co.id$ 1440 100% 3500 override-expire override-lastmod reload-into-ims ignore-reload
> refresh_pattern -i \.flv$ 999999 90% 999999 ignore-no-cache override-expire ignore-private
> refresh_pattern -i .co.id$ 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload
> refresh_pattern -i .mail.yahoo$ 2440 100% 35000 override-expire override-lastmod reload-into-ims ignore-reload
> #youtube's videos
> refresh_pattern ^http://(.*?)/get_video\? 999999 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://(.*?)/watch\? 999999 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://(.*?)/videoplayback\? 161280 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://.*?.\files\.youporn\.com/.*?/.*?/.*?\.flv\?.* 10080 90% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://(.*?)/videodownload\? 161280 90% 999999 override-expire ignore-no-cache ignore-private
> refresh_pattern ^http://i(.*?).photobucket.com/albums/(.*?)/(.*?)/(.*?)\? 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://vid(.*?).photobucket.com/albums/(.*?)/(.*?)\? 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.blogsome.com/.* 7200 90% 10080
> refresh_pattern ^http://*.gmail.*/.* 720 100% 4320
> refresh_pattern ^http://*.plasa.*/.* 720 100% 4320
> refresh_pattern ^http://*.telkom.*/.* 720 100% 4320
> refresh_pattern ^http:\/\/74\.125 86400 50% 86400 override-expire override-lastmod
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320 reload-into-ims
> refresh_pattern -i \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv)(\?|$) 161280 3000% 525948 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(class|css|js|gif|jpg)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(jpe|jpeg|png|bmp|tif)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(tiff|mov|avi|qt|mpeg)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(mpg|mpe|wav|au|mid)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(zip|gz|arj|lha|lzh)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(rar|tgz|tar|exe|bin)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(hqx|pdf|rtf|doc|swf)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(inc|cab|ad|txt|dll)$ 10080 100% 43200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern ^http://*.adframe.php.* 999999 100% 999999 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
> refresh_pattern -i .(asp|acgi|pl|shtml|php3|php)$ 22 50% 43200 reload-into-ims
> refresh_pattern -i ? 22 50% 4320 reload-into-ims
> refresh_pattern -i cgi-bin 22 50% 4320 reload-into-ims
> refresh_pattern -i (/cgi-bin/|\?) 22 50% 4320
>
> quick_abort_min 0
> quick_abort_max 0
> quick_abort_pct 98 %
> negative_ttl 5 minutes
>
>
> # HTTP OPTIONS
> acl apache rep_header Server ^Apache
> broken_vary_encoding allow apache
> vary_ignore_expire on
> cache_mgr cache5_at_globalone.net.lb
> cache_effective_user squid
> cache_effective_group squid
> visible_hostname cache5.globalone.net.lb
> shutdown_lifetime 8 seconds
>
> # HTTPD-ACCELERATOR OPTIONS
>
>
> # TAG: snmp_port
> # snmp_port 3401
>
> # ICP OPTIONS
> #icp_port 3130
> #icp_hit_stale on
> #log_icp_queries on
> #never_direct deny all
> dns_nameservers 192.168.1.69 208.67.222.222 208.67.220.220
> ipcache_size 2048
> ipcache_low 98
> ipcache_high 99
> memory_pools off
> reload_into_ims on
> offline_mode on
> pipeline_prefetch off
>
>
> #delay pools
> # ACL time
> #acl sepi time SMTWHFA 00:00-09:59
> #acl lumayan time SMTWHFA 10:00-11:59
> #acl padat time SMTWHFA 12:00-16:59
> #acl lumayan time SMTWHFA 17:00-18:59
> #acl padat time SMTWHFA 19:00-21:59
> #acl lumayan time SMTWHFA 22:00-23:59
>
>
> acl download url_regex -i ftp \.exe$ \.mp3$ \.mp4$ \.tar.gz$ \.gz$ \.tar.bz2$ \.rpm$ \.zip$ \.rar$ \.sh$
> acl download url_regex -i \.avi$ \.mpg$ \.mpeg$ \.rm$ \.iso$ \.wav$ \.mov$ \.dat$ \.mpe$ \.mid$ \.rmvb$
> acl download url_regex -i \.midi$ \.rmi$ \.wma$ \.wmv$ \.ogg$ \.ogm$ \.m1v$ \.mp2$ \.mpa$ \.wax$ \.mpg$
> acl download url_regex -i \.m3u$ \.asx$ \.wpl$ \.wmx$ \.dvr-ms$ \.snd$ \.au$ \.aif$ \.asf$ \.m2v$ \.qt$
> acl download url_regex -i \.m2p$ \.ts$ \.tp$ \.trp$ \.div$ \.divx$ \.mod$ \.vob$ \.aob$ \.dts$ \.xpi$ \.jar$
> acl download url_regex -i \.ac3$ \.cda$ \.vro$ \.deb$ \.spl$ \.swf$ \.3gp$ \.flv$ \.7z$ \.cab$ \.vqf$ \.mpe$
> acl download url_regex -i \.flv$ \.fla$ \.tbz$ \.tgz$ \.bz2$ \.bz$ \.pdf$ \.chm$ \.ppt$ \.doc$ \.odt$
> acl download url_regex -i \.raw$ \.3gp$ \.dll$ \.mdb$ \.mda$ \.mde$ \.mdt$ \.mdw$ \.vxd$ \.hqx$ \.sis$
> acl download url_regex -i \.wsc$ \.wsf$ \.wsh$ \.xls$ \.tar$ \.dmg$ \.sit$ \.bin$ \.cue$ \.wmf$ \.ram$
> acl download url_regex -i get_video\? watch\? video_id\? videodownload\?
>
> #delay_pools 4
> #delay_class 1 1
> #delay_parameters 1 2300/4050
> #delay_access 1 allow our_networks
> #delay_access 1 allow padat download
> #delay_access 1 deny all
> #delay_class 2 1
> #delay_parameters 2 52000/640000
> #delay_access 2 allow our_networks
> #delay_access 2 allow lumayan download
> #delay_access 2 deny all
> #delay_class 3 1
> #delay_parameters 3 -1/-1
> #delay_access 3 allow our_networks
> #delay_access 3 allow sepi download
> #delay_access 3 deny all
>
> #delay_class 4 1
> #delay_parameters 4 -1/-1
> #delay_access 4 allow admin
> #delay_access 4 allow localhost
> #delay_access 4 deny all
>
> #delay_pools 2
> #delay_class 1 3
> #delay_parameters 1 -1/-1 24000/64000 16000/32000
> #delay_access 1 allow our_networks download
> #delay_access 1 deny all
>
> #delay_class 2 2
> #delay_parameters 2 -1/-1 -1/-1
> #delay_access 2 allow admin our_networks
> #delay_access 2 deny all
>
> -------------------------------
> store_url_rewrite.pl :
> --------------------
> $| = 1;
>
> while (<>) {
> chomp;
> # print STDERR $_ . "\n";
> if (m/kh(.*?)\.google\.com(.*?)\/(.*?) /) {
> print "http://keyhole-srv.google.com" . $2 . ".SQUIDINTERNAL/" . $3 . "\n";
> # print STDERR "KEYHOLE\n";
> } elsif (m/mt(.*?)\.google\.com(.*?)\/(.*?) /) {
> print "http://map-srv.google.com" . $2 . ".SQUIDINTERNAL/" . $3 . "\n";
> # print STDERR "MAPSRV\n";
> } elsif (m/^http:\/\/([A-Za-z]*?)-(.*?)\.(.*)\.youtube\.com\/get_video\?video_id=(.*) /) {
> # http://lax-v290.lax.youtube.com/get_video?video_id=jqx1ZmzX0k0
> print "http://video-srv.youtube.com.SQUIDINTERNAL/get_video?video_id=" . $4 . "\n";
> } else {
> print $_ . "\n";
> }
> }
> --------------------------------
>
> Many thanks
>
> Regards,

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE16
   Current Beta Squid 3.1.0.9