[squid-users] store_url_rewriting on squid 2.7 completed. thanks all (there is much more to do)

Thank amos and everyone that helped me.

today i finished my youtube caching project using squid 3.1.11 and squid2.7

it took me awhile to understand how everything works and all the weird
stuff that i got while rewriting the address and other stuff.
André Luiz dos Santos helped me very very much seeing some things that i
haven't seen about some end-points\states on youtube and other video sites.

i had a lot of trouble cause the urls for the videos are hell and i dont
like regular expressions.

i had an exam on java programming so to prepare for the test i played
with java and made the store_url_rewrite program on java.
the nice thing about this software is the simplicity of that i could
make things understandable and make the debugging steps for problems
much more effective.

André gave me the nice link to his work on:

http://code.google.com/p/youtube-cache/

i prefer the store_url_rewrite cause it's a cache server and not video
storage one so nginx and other stuff are nice but not exactly what i and
many other want to achieve.

problems i encounter on the way:
- identifying youtube dynamic urls (there are many of the) patterns.

- return =NULL logged as return from the storeurl_program
          the culprit was too much storeurl_rewrite_children
           using java storeurl_rewrite_children 2 is more then anyone
can need and more then 15 causing the problem.

- i got weird things like "rewritten url part one" + IP/- - GET -
myip=IP myport=3128"second end of the url"
     until now i really dont know the cause for it and the only option
is the my store_rewrite program was analysing wrong the input url
     and for that i have changed my code to make sure that the output is
fixed.

- sometimes the youtube links are not the videos them-self but a
redirection link to other domains and to avoid gettings "error loading"
      i needed to change the minimum object size to cache.

- the best way i have found till now to make the cache reliable is to
actually KNOW what the refresh rules are and all the acls system works.
     if you will add the correct rules you will benefit a lot.

- in the squid-cache site\wiki some of the explanations are old\not
understable.

- i didnt look that much but i am looking for a documented VISUAL IMAGE
of the squid system such as in iptables in here:
      http://www.linuxhomenetworking.com/wiki/images/f/f0/Iptables.gif
     i would be glad to make one if there isn't.

- i am looking for a way to extend an object cache lease\life span based
on usage.
     is there a way to "mangle" a single object cache validity without
releasing and reloading it into the cache?

thanks Eliezer
Received on Tue Feb 22 2011 - 03:41:31 MST