Re: [squid-users] Youtube, storeurl and 302 redirection

the RANGE ins included in the request URL , not in the request header :

http://o-o.preferred.lhr14s07.v4.lscache4.c.youtube.com/videoplayback?ptk=WMG&algorithm=throttle-factor&ipbits=8&id=4afe9d30517fcadb&sver=3&keepalive=yes&ip=95.0.0.0&range=3563520-5345279&upn=bcUthT7SlxY&signature=8E4C1D0F0157382C5B24AD9A1650E09AEE4A3B19.3C2018BBC937F347D256AE2344D75396DB16037A&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&key=yt1&source=youtube&burst=40&expire=1334086473&ptchn=FueledByRamen&cp=U0hSSVBTVF9MTUNOMl9ITVhFOnNtQXY0ZkUwTmJE&itag=34&factor=1.25&cm2=0

the above link is a request for 1.7M video segment I was tracing this morning.

On Tue, Apr 10, 2012 at 4:13 PM, Christian Loth
<c.loth_at_phase2-networks.com> wrote:
> On Tuesday 10 April 2012 14:48:36 Eliezer Croitoru wrote:
>> On 10/04/2012 03:41, Paolo Malfatti wrote:
>> > Yes, but i saw that the secuence of parameters changes : sometimes "id"
>> > is before "range" and sometimes after.
>> >
>> > -----Mensaje original----- From: Hasanen AL-Bana
>> > Sent: Monday, April 09, 2012 2:33 PM
>> > To: Paolo Malfatti
>> > Subject: Re: [squid-users] Youtube, storeurl and 302 redirection
>> >
>> > ahh...so if we include the range parameter in the storeurl , we might
>> > get something working.
>>
>> what you should do is use some regex patterns to limit the usage of the
>> storeurl..
>> this will give you much more effective url rewriting.
>> dont ever cache the "range" urls.
>> and for this.. nginx very very good.
>
> Why not? The range-parameters seem predictable enough. In the squid/nginx
> setup we use we see a caching effect of stored chunks -  the range chunks are
> requested multiple times by multiple clients. In our setup, where disk space
> is cheap and bandwidth is expensive this is well justified, in my opinion.
>
> However: I have started only recently to learn about this proxy setup. If
> there are reasons against caching range-URLs, please do tell! I'm always
> willing to learn :)
>
> Best Regards,
> - Christian Loth
>
>
>> i must say that nginx did the job so good.
>> in a case of redirection it will not cache it.
>> and so on many cases i have tested.
>> i have used two squid instances with main squid3.2 branch and the second
>> instance squid2.7 for usage with store_url.
>> there is also a "redirect" argument that is being used by some ISP's.
>> you can try to analyze the urls a bit more to get some accurate data on it.
>> i was just sitting on my ass for a long time while trying couple methods
>> with store_url and i wrote my own (dont know where it is)
>> store_url_rewite helper on java.
>> it is indeed was hell of a project but nginx does it so good.. so i just
>> used it.
>>
>> Regards,
>> Eliezer
>>
>> > On Mon, Apr 9, 2012 at 7:10 PM, Paolo Malfatti <paolo_at_cidis.ws> wrote:
>> >> I know, but when the flash player asks a segment, it puts a range
>> >> parameter
>> >> in the GET request. The script captures and saves these fragments in the
>> >> cache individually.
>> >>
>> >>> From: Hasanen AL-Bana
>> >>> Sent: Monday, April 09, 2012 1:19 AM
>> >>> To: Paolo Malfatti
>> >>> Cc: squid-users_at_squid-cache.org
>> >>>
>> >>> Subject: Re: [squid-users] Youtube, storeurl and 302 redirection
>> >>>
>> >>> you will face a major problem , recently youtube decided to split
>> >>> videos into 1.7MB segments,making it harder for people to download the
>> >>> streamed videos, most of the newly uploaded videos are coming
>> >>> into segments now, causing squid to cache only one segment and it
>> >>> thinks that the video has been fully downloaded, this will either give
>> >>> you error in
>> >>> flash player or you might get few seconds of the video
>> >>> only.
>> >>>
>> >>>> On Mon, Apr 9, 2012 at 4:34 AM, Paolo Malfatti <paolo_at_cidis.ws> wrote:
>> >>>>
>> >>>> Hi, i’m using storeurl.pl script to cache youtube videos files and I
>> >>>> followed instructions in:
>> >>>>
>> >>>>
>> >>>> http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube#Conf
>> >>>>igExamples.2BAC8-DynamicContent.2BAC8-YouTube.2BAC8-Discussion.FixedThi
>> >>>>s
>> >>>>
>> >>>> patch don't worked for me, so I changed it a bit:
>> >>>> 1) I'd like to ignore redirects, only when request is a storeurl
>> >>>> request.
>> >>>> 2) I dont like to force a MISS to "304 NOT MODIFIED" responses
>> >>>>
>> >>>> What do you think about?
>> >>>> Thanks
>> >>>>
>> >>>> Paolo Malfatti
>> >>>> CIDIS Camiri
>> >>>> Index: src/client_side.c
>> >>>> ===================================================================
>> >>>> --- src/client_side.c (revision 134)
>> >>>> +++ src/client_side.c (working copy)
>> >>>> @@ -2408,6 +2408,17 @@
>> >>>> is_modified = 0;
>> >>>> }
>> >>>> }
>> >>>> + /* bug fix for 302 moved_temporarily loop bug when using
>> >>>> storeurl*/
>> >>>> + if (r->store_url && rep->sline.status >= 300 && rep->sline.status
>> >>>> <400 && rep->sline.status != 304) {
>> >>>> + if (httpHeaderHas(&e->mem_obj->reply->header, HDR_LOCATION))
>> >>>> {
>> >>>> + debug(33, 2) ("clientCacheHit: Redirect Loop
>> >>>> Detected:%s\n",http->uri);
>> >>>> + http->log_type = LOG_TCP_MISS;+ clientProcessMiss(http);
>> >>>> + return;
>> >>>> + }
>> >>>> + }
>> >>>> + /* bug fix end here*/
>> >>>> stale = refreshCheckHTTPStale(e, r);
>> >>>> debug(33, 2) ("clientCacheHit: refreshCheckHTTPStale returned
>> >>>> %d\n",stale);
>> >>>> if (stale == 0) {
>>
Received on Tue Apr 10 2012 - 13:23:24 MDT