Re: [squid-users] Youtube, storeurl and 302 redirection

On 10/04/2012 03:41, Paolo Malfatti wrote:
> Yes, but i saw that the secuence of parameters changes : sometimes "id"
> is before "range" and sometimes after.
>
> -----Mensaje original----- From: Hasanen AL-Bana
> Sent: Monday, April 09, 2012 2:33 PM
> To: Paolo Malfatti
> Subject: Re: [squid-users] Youtube, storeurl and 302 redirection
>
> ahh...so if we include the range parameter in the storeurl , we might
> get something working.
what you should do is use some regex patterns to limit the usage of the
storeurl..
this will give you much more effective url rewriting.
dont ever cache the "range" urls.
and for this.. nginx very very good.
i must say that nginx did the job so good.
in a case of redirection it will not cache it.
and so on many cases i have tested.
i have used two squid instances with main squid3.2 branch and the second
instance squid2.7 for usage with store_url.
there is also a "redirect" argument that is being used by some ISP's.
you can try to analyze the urls a bit more to get some accurate data on it.
i was just sitting on my ass for a long time while trying couple methods
with store_url and i wrote my own (dont know where it is)
store_url_rewite helper on java.
it is indeed was hell of a project but nginx does it so good.. so i just
used it.

Regards,
Eliezer

>
> On Mon, Apr 9, 2012 at 7:10 PM, Paolo Malfatti <paolo_at_cidis.ws> wrote:
>> I know, but when the flash player asks a segment, it puts a range
>> parameter
>> in the GET request. The script captures and saves these fragments in the
>> cache individually.
>>
>>> From: Hasanen AL-Bana
>>> Sent: Monday, April 09, 2012 1:19 AM
>>> To: Paolo Malfatti
>>> Cc: squid-users_at_squid-cache.org
>>>
>>> Subject: Re: [squid-users] Youtube, storeurl and 302 redirection
>>>
>>> you will face a major problem , recently youtube decided to split videos
>>> into 1.7MB segments,making it harder for people to download the streamed
>>> videos, most of the newly uploaded videos are coming
>>> into segments now, causing squid to cache only one segment and it thinks
>>> that the video has been fully downloaded, this will either give you
>>> error in
>>> flash player or you might get few seconds of the video
>>> only.
>>
>>
>>
>>>> On Mon, Apr 9, 2012 at 4:34 AM, Paolo Malfatti <paolo_at_cidis.ws> wrote:
>>>>
>>>> Hi, i’m using storeurl.pl script to cache youtube videos files and I
>>>> followed instructions in:
>>>>
>>>>
>>>> http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube#ConfigExamples.2BAC8-DynamicContent.2BAC8-YouTube.2BAC8-Discussion.FixedThis
>>>>
>>>> patch don't worked for me, so I changed it a bit:
>>>> 1) I'd like to ignore redirects, only when request is a storeurl
>>>> request.
>>>> 2) I dont like to force a MISS to "304 NOT MODIFIED" responses
>>>>
>>>> What do you think about?
>>>> Thanks
>>>>
>>>> Paolo Malfatti
>>>> CIDIS Camiri
>>>> Index: src/client_side.c
>>>> ===================================================================
>>>> --- src/client_side.c (revision 134)
>>>> +++ src/client_side.c (working copy)
>>>> @@ -2408,6 +2408,17 @@
>>>> is_modified = 0;
>>>> }
>>>> }
>>>> + /* bug fix for 302 moved_temporarily loop bug when using
>>>> storeurl*/
>>>> + if (r->store_url && rep->sline.status >= 300 && rep->sline.status
>>>> <400 && rep->sline.status != 304) {
>>>> + if (httpHeaderHas(&e->mem_obj->reply->header, HDR_LOCATION))
>>>> {
>>>> + debug(33, 2) ("clientCacheHit: Redirect Loop
>>>> Detected:%s\n",http->uri);
>>>> + http->log_type = LOG_TCP_MISS;+ clientProcessMiss(http);
>>>> + return;
>>>> + }
>>>> + }
>>>> + /* bug fix end here*/
>>>> stale = refreshCheckHTTPStale(e, r);
>>>> debug(33, 2) ("clientCacheHit: refreshCheckHTTPStale returned
>>>> %d\n",stale);
>>>> if (stale == 0) {
>>
>>
>>
>>
>>
>
>

-- 
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer <at> ngtech.co.il