Re: [squid-users] slow internet browsing.

On 25/04/2012 3:34 a.m., Eliezer Croitoru wrote:
> On 24/04/2012 18:14, Muhammad Yousuf Khan wrote:
>> ok i trim down config file to this as you suggested of blocking
>> whitelist to local net.. let see how things work tommorw. ill update.
>> but block list is like 10MB big do you think it could be the
>> problem.as every query has to be matched with 10 MB database.
>>
>> ?
> in any case a dstdomain of 10MB is a very bad idea from what i know.
> one thing about dstdomain is that squid must validate the request dns
> records and it will take more bandwidth on dns queries.

Only if comparing a raw-IP to a domain name. If the raw-IP is on teh
tested URL it is faster as the DNS result gets re-used for all tests.
The common case though is straight domain-vs-domain comparisons.

Amos

> if you still dont have local dns server for cahing only this is the
> time to add it.
>
> i think that 10MB of domains can be optimized into some basic DST
> DOMAINS REGEX and some blacklist DSTDOMS REGEX.
>
> i think that some db application for this kind of amount of dstdoms
> can much more effective.
> you can also use squidguard for that.
>
> if you can share some (1MB) of the dstdoms of the whole list i might
> be able to try to optimize it in a way.
>
>
> Regards,
> Eliezer
>
>>
>>
>>
>> #-------------Allow All ACL-------------
>> acl aci_lan src 10.51.100.0/24
>> acl aci_general src 10.51.100.0/24
>>
>> #---------------------Assurety Whitelist---------------
>> acl aci_whitelist dstdomain "/blocklist/aci_list/whitelist"
>> http_access allow aci_whitelist aci_general
>>
>> #----------TimeDomainBlock
>> acl aci_dest dstdomain "/blocklist/aci_list/time_block_domains"
>>
>> #--General Timing------------ Normal Days Working hours--------------
>> acl aci_working_hours time MTWH 10:04-13:04
>> acl aci_working_hours time MTWH 14:04-18:04
>> #--General Timing-------------Friday------------------------
>> acl aci_working_hours time F 10:04-13:04
>> acl aci_working_hours time F 15:04-18:04
>>
>> http_access deny aci_dest aci_working_hours aci_general
>>
>>
>> On Tue, Apr 24, 2012 at 1:11 PM, Eliezer
>> Croitoru<eliezer_at_ngtech.co.il> wrote:
>>> are you taking about the delay pools rules?
>>> also if it's a proxy that is open to the internet i would limit the
>>> access
>>> to port 3128 to only lan.
>>> your http_access rules are allowing anyone to use the proxy for the
>>> whitelist.
>>>
>>> Regards,
>>> Eliezer
>>>
>>>
>>>
>>> On 24/04/2012 09:06, Muhammad Yousuf Khan wrote:
>>>>
>>>> ok i just disabled all the rules and it works for me now ill test
>>>> which rule is making a problem and let you know also.
>>>>
>>>> Thanks
>>>>
>>>> On Mon, Apr 23, 2012 at 11:20 PM, Muhammad Yousuf
>>>> Khan<sirtcp_at_gmail.com>
>>>> wrote:
>>>>>
>>>>> here is the log for bbc.co.uk . first and last msg of log
>>>>>
>>>>> so you can see the time delay.
>>>>>
>>>>> 335205033.183 841 10.51.100.240 TCP_MISS/200 24506 GET
>>>>> http://www.bbc.co.uk/ - DIRECT/212.58.244.66 text/html
>>>>> 1335205057.936 328 10.51.100.240 TCP_REFRESH_HIT/304 435 GET
>>>>> http://static.bbci.co.uk/wwhomepage-3.5/1.0.41/img/broadcast-sprite.png
>>>>>
>>>>> - DIRECT/80.239.148.70 image/png
>>>>>
>>>>>
>>>>> On Mon, Apr 23, 2012 at 11:12 PM, Muhammad Yousuf
>>>>> Khan<sirtcp_at_gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Here you go with my squid.conf
>>>>>>
>>>>>> acl all src all
>>>>>> acl manager proto cache_object
>>>>>> acl localhost src 127.0.0.1/32
>>>>>> acl to_localhost dst 127.0.0.0/8
>>>>>> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
>>>>>> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
>>>>>> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
>>>>>> acl SSL_ports port 443 # https
>>>>>> acl SSL_ports port 563 # snews
>>>>>> acl SSL_ports port 873 # rsync
>>>>>> acl Safe_ports port 80 # http
>>>>>> acl Safe_ports port 21 # ftp
>>>>>> acl Safe_ports port 443 # https
>>>>>> acl Safe_ports port 70 # gopher
>>>>>> acl Safe_ports port 210 # wais
>>>>>> acl Safe_ports port 1025-65535 # unregistered ports
>>>>>> acl Safe_ports port 280 # http-mgmt
>>>>>> acl Safe_ports port 488 # gss-http
>>>>>> acl Safe_ports port 591 # filemaker
>>>>>> acl Safe_ports port 777 # multiling http
>>>>>> acl Safe_ports port 631 # cups
>>>>>> acl Safe_ports port 873 # rsync
>>>>>> acl Safe_ports port 901 # SWAT
>>>>>> acl purge method PURGE
>>>>>> acl CONNECT method CONNECT
>>>>>>
>>>>>> # sqstat
>>>>>> acl manager proto cache_object
>>>>>> acl webserver src 10.51.100.206/255.255.255.255
>>>>>> http_access allow manager webserver
>>>>>> http_access deny manager
>>>>>>
>>>>>>
>>>>>>
>>>>>> # Skype
>>>>>> acl numeric_IPs dstdom_regex
>>>>>>
>>>>>> ^(([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)|(\[([0-9af]+)?:([0-9af:]+)?:([0-9af]+)?\])):443
>>>>>>
>>>>>> acl Skype_UA browser ^skype
>>>>>> acl validUserAgent browser \S+
>>>>>>
>>>>>> # for cheetah only
>>>>>>
>>>>>> #acl usman src 10.51.100.107
>>>>>> #delay_pools 1
>>>>>> #delay_class 1 1
>>>>>> #delay_parameters 1 22000/22000
>>>>>> #delay_access 1 allow usman
>>>>>>
>>>>>>
>>>>>>
>>>>>> #-------------Allow All ACL-------------
>>>>>> acl aci_lan src 10.51.100.0/24
>>>>>> acl aci_general src 10.51.100.0/24
>>>>>>
>>>>>>
>>>>>> #----My ip
>>>>>> acl my_ip src 10.51.100.240
>>>>>> http_access allow my_ip
>>>>>>
>>>>>>
>>>>>>
>>>>>> # Testing delay pool
>>>>>> delay_pools 1
>>>>>> delay_class 1 1
>>>>>> delay_parameters 1 22000/10240000
>>>>>> delay_access 1 allow aci_general
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> #---------------------Assurety Whitelist---------------
>>>>>> acl aci_whitelist dstdomain "/blocklist/aci_list/whitelist"
>>>>>> http_access allow aci_whitelist
>>>>>>
>>>>>> #--Senior Allow Domainlist------------------------------
>>>>>> acl aci_seniors dstdomain "/blocklist/aci_list/whitelist_seniors"
>>>>>> #---------------------------------------------------------#See
>>>>>> implimentation in ACI implimentation section
>>>>>>
>>>>>> #--------------------Assurety Hard_Block--------------
>>>>>> acl aci_hard_block dstdomain
>>>>>> "/blocklist/aci_list/hard_block_domains"
>>>>>> http_access deny aci_hard_block
>>>>>>
>>>>>> #--------------------Hard_Block EXE and E.T.C---------------------
>>>>>> #acl mime_block_hard rep_mime_type -i
>>>>>> "/blocklist/aci_list/hard_mime_block"
>>>>>> #http_reply_access deny mime_block_hard
>>>>>>
>>>>>>
>>>>>> #--General------Streaming Block------------------------------
>>>>>> acl mime_block rep_mime_type -i
>>>>>> "/blocklist/aci_list/time_mime_block"
>>>>>>
>>>>>> #--General Domainlist------------------------------
>>>>>> acl aci_dest dstdomain "/blocklist/aci_list/time_block_domains"
>>>>>>
>>>>>> #--Seniors MAC list mouting------------------------------
>>>>>> acl aci_mac_seniors arp "/blocklist/aci_list/mac_list_seniors"
>>>>>>
>>>>>> #--General Timing------------ Normal Days Working
>>>>>> hours--------------
>>>>>> acl aci_working_hours time MTWH 10:04-13:04
>>>>>> acl aci_working_hours time MTWH 14:04-18:04
>>>>>> #--General Timing-------------Friday------------------------
>>>>>> acl aci_working_hours time F 10:04-13:04
>>>>>> acl aci_working_hours time F 15:04-18:04
>>>>>>
>>>>>> #--General/Seniors-------------Implimentation------------------
>>>>>> http_access allow aci_seniors aci_mac_seniors
>>>>>> http_access deny aci_dest aci_working_hours aci_general
>>>>>> http_reply_access deny mime_block aci_working_hours aci_general
>>>>>> !my_ip
>>>>>>
>>>>>> #skype deny
>>>>>> http_access deny numeric_IPS aci_working_hours
>>>>>> http_access deny Skype_UA aci_working_hours
>>>>>> http_access deny !validUserAgent aci_working_hours
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> #Error Directory by Ykhan
>>>>>> error_directory /usr/share/squid/errors/en-us/
>>>>>> #------------------------TheEnd----------------------
>>>>>> http_access allow aci_lan
>>>>>>
>>>>>>
>>>>>>
>>>>>> http_access allow manager localhost
>>>>>> http_access deny manager
>>>>>> http_access allow purge localhost
>>>>>> http_access deny purge
>>>>>> http_access deny !Safe_ports
>>>>>> http_access deny CONNECT !SSL_ports
>>>>>> http_access allow localhost
>>>>>> http_access deny all
>>>>>> icp_access allow localnet
>>>>>> icp_access deny all
>>>>>> http_port 3128
>>>>>> hierarchy_stoplist cgi-bin ?
>>>>>> access_log /var/log/squid/access.log squid
>>>>>> refresh_pattern ^ftp: 1440 20% 10080
>>>>>> refresh_pattern ^gopher: 1440 0% 1440
>>>>>> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>>>>>> refresh_pattern (Release|Package(.gz)*)$ 0 20% 2880
>>>>>> refresh_pattern . 0 20% 4320
>>>>>> acl shoutcast rep_header X-HTTP09-First-Line ^ICY\s[0-9]
>>>>>> upgrade_http0.9 deny shoutcast
>>>>>> acl apache rep_header Server ^Apache
>>>>>> broken_vary_encoding allow apache
>>>>>> extension_methods REPORT MERGE MKACTIVITY CHECKOUT
>>>>>> hosts_file /etc/hosts
>>>>>> coredump_dir /var/spool/squid
>>>>>>
>>>>>> ##ykhan squid redirection to squidguard
>>>>>>
>>>>>> #redirect_program /usr/bin/squidGuard
>>>>>> #url_rewrite_program /usr/bin/squidGuard
>>>>>> #url_rewrite_children 5
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 23, 2012 at 8:42 PM, Eliezer
>>>>>> Croitoru<eliezer_at_ngtech.co.il>
>>>>>> wrote:
>>>>>>>
>>>>>>> On 23/04/2012 18:38, Muhammad Yousuf Khan wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> well i have been experiencing slow Internet browsing. not very
>>>>>>>> slow
>>>>>>>> but comparatively slower then IPCOP firewall. i can not
>>>>>>>> understand how
>>>>>>>> come i diagnose the issue.
>>>>>>>> i mean. i increase the RAM , i checked the DNS every thing is
>>>>>>>> fine but
>>>>>>>> my browser stuck at "connecting" ones it start download it do
>>>>>>>> it fast
>>>>>>>> but then stop for something then start. i am not getting the clear
>>>>>>>> picture. can anyone help
>>>>>>>>
>>>>>>>> i am suing debian 6.0.4 with 2.7 stable squid.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> MYK
>>>>>>>
>>>>>>>
>>>>>>> what is your exact problem? slow downloads?
>>>>>>> what is your squid setup?transparent ?regular forward proxy?
>>>>>>> what browser are you using?
>>>>>>> do you have some squid logs? or squid.conf?
>>>>>>> what dns server are you using?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Eliezer
>>>>>>>
>>>>>>> --
>>>>>>> Eliezer Croitoru
>>>>>>> https://www1.ngtech.co.il
>>>>>>> IT consulting for Nonprofit organizations
>>>>>>> eliezer<at> ngtech.co.il
>>>
>>>
>>>
>>> --
>>> Eliezer Croitoru
>>> https://www1.ngtech.co.il
>>> IT consulting for Nonprofit organizations
>>> eliezer<at> ngtech.co.il
>
>
Received on Wed Apr 25 2012 - 02:38:59 MDT