Re: [Mod_gzip] Re: Squid and Compression

Hi you all,

I just want to sum up the discussion a little,
as IMHO there are points not yet mentioned.

-----------------------------------------------

Can't I just upgrade the browser to solve the
whole problem?

Unlike Peter stated, there is no solution to
"simply upgrade your browser, it's free".
First of all the software may be free, but
upgrading a "fleet strategy" company browser
installation on thousands of PCs with a common
software infrastructure, and maybe even some
intranet applications depending on specific
browsers, is _definitely_ not 'free'.

My daily business is to write HTML/JavaScript
pages for a group of large-company customers
(German and Austrian Banks) and they are run-
ning 25% Netscape 4, many of them on OS/2 PCs.
You may guess which hardware they are using.
You would't want to install an operating system
from the current millennium on these machines,
and the same applies to a current browser.
Mozilla is a great browser, but be sure you
have at least 256 MB RAM if you want to run
it on any Windows platform.

These ancient browsers will not vanish quickly.
You don't need Mozilla to do online business,
Netscape 4 and M$IE 5 will do nicely.
It is much more likely that one server (or proxy)
will be upgraded than a thousand clients.

-----------------------------------------------

What is the real problem?

The real problem is that there is _one_single_
browser (known to me) that understands compres-
sed content _even_ if it didn't ask for it.

The name of this browser is Opera 6, and its
market share is somewhere in the 1% range -
because it has huge problems with JavaScript
and DOM (Opera 6 is lying about DOM as much as
Netscape 4 is lying about compression).

Neither M$IE nor Mozilla (!) will handle gzipped
content correctly if they didn't send the HTTP
header "Accept-Encoding: gzip".
Try it yourself: Load a compressed file into some
proxy cache, then turn off the header (M$IE:
Disable HTTP/1.1; Mozilla: Edit the preferences,
Advanded, HTTP networking) and you will see byte
garbage. I can reproduce both under Windows NT.

There is no real problem with Mozilla, as there
is no reason to ever switch off handling gzipped
content there. Even Netscape 7.0PR1, the "official"
version of Mozilla, _has_ this preferences option
available, but you just don't need it.

M$IE is a different story. Sending the "Accept-
Encoding" header is implicitly depending on the
Internet Options setting of "Use HTTP/1.1" and
"Use HTTP/1.1 through proxy connections".
As if this wasn't bogus enough, Micro$oft even
decided to change that behaviour incompatibly
when moving from M$IE 5.0 to M$IE 5.5.
While 5.0 did send "Accept-Encoding" when the
"Use HTTP/1.1" was checked, 5.5 now is sending
this header only if "Use HTTP/1.1 through proxy
connections" is checked.
And "is sending the header" is a synonym for
"is decompressing gzipped content"!

But most important is that M$IE is installed
with "Use HTTP/1.1" checked but "Use HTTP/1.1
through proxy connection" disabled by default.

So what we now have is a situation where the
browsers that make up two thirds of the current
internet market share, i. e. M$IE 5.5 and 6.0,
both have installation default settings where
HTTP/1.1 is checked for use but not via proxy.
This means that two thirds of all browsers
around don't understand compressed content if
sent without permission!
And there is no version to "upgrade to".

Even worse, every company that is migrating
from M$IE 5.0 to 5.5 or 6.0 had better drop
their proxy caches if using the installation
default Internet Options (which is not un-
likely at all).
Because what happens is that "in 5.0 days"
they happily used compressed data sent by
mod_gzip without a "Vary:" header and stored
in Squid caches, and now all the new M$IE
5.5 don't support uncompressing these cache
contents any more!
We are now one step further away (!) from
using compressed content everywhere. :-(

And another problem is that there indeed are
proxies around that don't work nicely if you
try to access them via HTTP/1.1.
So how do you convince some company admin to
configure all M$IE as to "use HTTP/1.1 via
Proxy"? Let there be _one_ mission critical
application that does't work with this setting,
the admin will make all browser installations
return to the installation default at once.

-----------------------------------------------

What is the primary goal?

I suggest the primary goal to be "correct data",
not "fast transmission".
I am fully aware of the trade-off between both,
and I have the opinion that ultimately only the
server administrator (either Apache or Squid)
really can decide what he wants to get.
There are as many different goals as scenarios,
and I don't think that either mod_gzip or squid
should unnecessarily prevent the admin from doing
what he decides to be correct. After all, he could
always take the source code and hack in what he
likes. So I better give him a piece of software
that allows him to handle all reasonable scenarios,
of which sending the "Vary:" is definitely one,
and not sending it may well be another.
Therefore my patched "mod_gzip 1.3.19.1b" may well
be a solution for many users, but I would prefer
some configuration directive here. At the moment
I am not able to implement it, but I don't rule
this out for the future.

Default values are a different story; here it
is obvious that a "cooperative" attempt should
be made, i. e. make the behaviour so that nor-
mally everything will work, maybe at the expense
of speed, and provide documentation to explain
the pros and cons of configuration options.
(Which is exactly what I am trying on my pages
- I am not the C coder anyway.)

-----------------------------------------------

Whose job is it to take care?

This depends upon what you want to get.

Today I had a case of a customer whose Micro$oft
Proxy Server was running with installation
default settings, which means "ignore all HTTP
headers and cache everything you can, except
for URLs ending in *.asp, which are known to
me as dynamic content".
Believe it or not, this proxy did cache pages
where I sent "Cache-Control" AND "Expires" AND
"Vary:", and these pages did contain real-time
financial informations. :-(

I just tell this story to show which software
world we are living in.
Whoever is able to take care of protocol vio-
lations of "partner" software has an edge over
someone who doesn't. This is the reason of the
success of M$IE, and while I don't like the
stategy of setting defaults in Micro$oft pro-
ducts, I understand their effort to support as
much as they can (rendering broken HTML pages
and the like, guessing MIME types instead of
trusting the HTTP headers and the like).
They do have a lot of experience with broken
software. ;-)

In this area I consider Squid to be much better
than the aforementioned M$ proxy, and I consider
mod_gzip to be much better than mod_deflate from
Apache 2.0. They both try to do it right, but
they care about others who don't do it right.
Please, you both, keep on doing it this way.

mod_gzip is already loaded with configuration
options to handle a lot of strange situations,
especially excluding the "broken lier" Netscape
4 from things it had better never seen.
Therefore my point of view is that it really
pays to handle errors of communication partners,
but you also need an option to switch off such
a feature if you want.
This is why mod_gzip indeed needs to be _able_
to send a "Vary:" header as well as not doing
so, depending on the scenario it will be used in.

-----------------------------------------------

How can both products help each other?

Ar least, by providing some reference implemen-
tation for a correctly working partner.

If mod_gzip supports HTTP/1.1 to the level that
Squid expects it to, then the Squid people can
easily tell everyone that there is a compression
implementation that works together with Squid.
And given the market share of Squid, this may
well be what mod_gzip can help a lot.

And if mod_gzip doesn't support HTTP/1.1 to
the level Squid expects it to, then there may
be the option for the Squid people to promote
mod_deflate, which _is_ sending the "Vary:"
header already (I have the Apache 2.0.36 source
here).

I suggest to not give them this option. ;-)

This is why I would vote for making mod_gzip
1.3.19.1b (or whichever implementation might
be better than mine) something like an official
upgrade.
I would prefer having a configuration option,
but as for now I am not able to implement it
myself. I believe the "Vary:" to be an improve-
ment right now. And if not, one is always free
to use 1.3.19.1a.
I would very much prefer all users to use "the
official mod_gzip 1.3.19.1b" than some patched
version by whom ever. So please give the Squid
people some official URL as to link to it,
instead of my humble private home page.

Regards, Michael
Received on Tue Aug 27 2002 - 21:17:10 MDT