Re: [squid-users] Generell Squid setup

On 21/09/2012 4:26 a.m., Farkas H wrote:
> Hi Amos,
>
> thank you for your response.
> I think I didn't explain the problem correctly. Let me mention that
> the http-Post request contains
> - one or two URLs,
> - name and parameters of one mathematical function from a set of
> server functions,
> - mime-type of the response.
>
> The client sends a http-Post request to the server
>
> The server ...
> (1) fetches the data (xml, up to 100 MB) via http-Get using the
> reference which is embedded in the client http-Post request,
> (2) computes a complete new dataset (xml, up to 100 MB) using the
> fetched data and the requested function and parameters,
> (3) converts the dataset to the requested mime-type,
> (4) returns the resulting dataset as a response of the http-Post
> request to the client.

Aha. So you are *not* transforming a POST into a GET as earlier stated.

You are generating an entirely new resource (at the web service and URLs
POST'ed too), from what happens to be remote data sources contacted via GET.

Because the front end requests are POST there is no caching of them by
Squid. The web service is free to run its own cache to optimize
computation responses, but that is outside of HTTP.

The back end requests being GET can be freely cached by a Squid right
next to the calculator web service. Erasing the latency to original
sources and de-duplicating repeat requests by the Web Service.

In theory the body response from a POST with Content-Location: header
could be cached under the Content-Location URL for later GET requests to
it, but Squid does not yet support that and it sounds like your system
the clients would not re-request using GET anyway. (Sponsorship welcome
if you want to go that way regardless).

> If we would implement a simple server cache the process would be like this:
> (- client sends request via http-Post to the server)
> - server generates a unique key of the client http-Post request,
> because the URL of the http-Post request is always the same,
> - server checks if the unique key / filename is in the cache; no:
> normal process,
> - server checks via http-Head (origin server) if the cached data is
> fresh: no normal process,
> - server sends the cached data associated to the unique key / filename
> as response of the http-Post request to the client; end,
> - normal process: server fetches the data via http-Get; server
> computes the result dataset using the fetched data and the requested
> function and parameters; server stores data in cache; server sends the
> result data as response of the http-Post request to the client; end.
>
> I wonder if Squid could help us to cache the data accosiated to the
> client http-Post request. My idea is to generate a unique key of the
> http-Post request to make it cacheable.

POST can be returned with Content-Length and ETag unique location and
identifier. Which technically make the response body cacheable, but only
when fetched from the Content-Location with GET requests later. All POST
requests still are not able to be responded with a HIT, only with MISS.

Amos

>
> Thanks so much for your response.
> Farkas
>
>
> On 8 September 2012 04:48, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> On 7/09/2012 9:39 a.m., Farkas H wrote:
>>> Hi Amos,
>>>
>>> thanks for your response.
>>> I modified the web application. Now we have the following infrastructure.
>>> client --> http-Post [embedded http-Get] --> Server / web application
>>> --> http-Get --> Squid -> Servers (-> Squid -> Server / web
>>> application -> client)
>>> Advantage: The Server / web application doesn't have to request data
>>> from the remote servers if it's in the Squid cache.
>>>
>>> Additionally I want to cache the http-Post requests.
>>> client --> http-Post --> Squid --> Server / web apllication /
>>> processing the response (-> Squid -> client)
>>
>> Requests body is not cacheable in HTTP.
>>
>> The body content is the state to be changed at end-server resource in the
>> URL. Caching it is meaningless, since next time the state needs to be
>> changed you CANNOT simply reply from a middleware cache saying "server state
>> now changed" without passing any of those details to the server.
>>
>> Additionally, your translation service is re-writing the POST into GET
>> requests so the POST ceases to exist at your gateway service. There is
>> nothing to cache.
>>
>>
>>
>>> The idea: We modify the header of the http-post request to make it unique.
>>> The information whether Squid has a stored response to the modified
>>> request (true or false) should be added to the request / should be
>>> forwarded to the destination server. There are two possibilities.
>>> (1) The modified request is not stored in Squid (new request).
>>> (2) The modified request is stored in Squid. We don't know yet if the
>>> data is still fresh.
>>> The request should be forwarded in both(!) possibilities, (1) and (2),
>>> to the destination server.
>>> Is that possible with Squid?
>>
>> Of course. That is dependent on the Cache-Control: headers the server
>> supplies to Squid with its responses and applies to each response
>> independent of anythign else.
>>
>> Send "Cache-Control: must-revalidate" and Squid will query the server for
>> each request asking if there are updates.
>>
>> Amos
Received on Fri Sep 21 2012 - 08:35:14 MDT