Re: Meaning of "Page faults with physical IO" ?

Jonathan Larmour wrote:
> At 15:39 14/08/97 +0200, Andreas Reeh wrote:
> >user : 0:22:46.54 1.3% page in : 966636 disk 1: 28680r 58203w
> >nice : 0:00:00.00 0.0% page out: 467220 disk 2: 201747r 92831w
> >system: 0:28:03.66 1.6% swap in : 5
> >idle : 1d 5:03:55.40 97.2% swap out: 21
> >uptime: 1d 5:54:45.58 context : 2432173
> >
> >If what procinfo says is true there were only 5 swap in and 21 swap out -
> so
> >why has squid 51826 page faults with physical i/o ?!
>
> Maybe I _slightly_ oversimplified, because OS's these days (not counting
> MeSsy Windoze) only swap in and out _data_. They page fault when loading
> executables and whenever malloc'ed regions are actually _used_ (rather than
> just allocated) (generally). Executable code gets discarded rather than
> swapped when memory is short, and reloaded directly from the executable.
> That's why deleting an executable while its running is a hazardous activity.

There's two different operations at work, Paging and swapping. Paging is
when individual pages are shuffled (either discarded or swapped to/
from disk), while "swapping" _generally_ means the entire process got sent
to/from disk.

Needless to say, swapping a process is a pretty drastic event, and usually
only reserved for when there's a memory crunch and paging out cannot free
enough memory quickly enough. Also, there's some variation on how
swapping is implemented in OS's. Some don't do it at all or do a hybrid
of paging and swapping instead.

As you say, paging out doesn't necessarily involve disk IO, eg: text (code)
pages are read-only and can simply be discarded if they are not used (and
reloaded if/when needed). Data pages are also discarded if unmodified, and
paged out if there's been any changes. Allocated memory (malloc) is always
saved to disk since there's no executable file to recover the data from.
mmap() memory is variable.. If it's backed from a file, it uses the same
rules as the data segment of a file - ie: either discarded if unmodified or
paged out.

There's also "demand zeroing" of pages as well that cause faults.. If you
malloc memory and it calls brk()/sbrk() to allocate new pages, the chances
are that you are allocated demand zero pages. Ie: the pages are not
"really" attached to your process yet, but when you access them for the
first time, the page fault causes the page to be connected to the process
address space and zeroed - this saves unnecessary zeroing of pages that are
allocated but never used.

The "page faults with physical IO" comes from the OS via getrusage(). It's
highly OS dependent on what it means. Generally, it means that the process
accessed a page that was not present in memory (for whatever reason) and
there was disk access to fetch it. Many OS's load executables by demand
paging as well, so the act of starting squid implicitly causes page faults
with disk IO - however, many (but not all) OS's use "read ahead" and
"prefault" heuristics to streamline the loading. Some OS's maintain
"intent queues" so that pages can be selected as pageout candidates ahead
of time. When (say) squid touches a freshly allocated demand zero page and
one is needed, the OS can page out one of the candidates on the spot,
causing a 'fault with physical IO' with demand zeroing of allocated memory
which doesn't happen on many other OS's. (The other OS's generally put
the process to sleep while the pageout daemon finds a page for it).

The meaning of "swapping" varies. On FreeBSD for example, swapping out is
implemented as unlocking upages, kernel stack, PTD etc for aggressive
pageout with the process. The only thing left of the process in memory is
the 'struct proc'. The FreeBSD paging system is highly adaptive and can
resort to paging in a way that is equivalent to the traditional swapping
style operation (ie: entire process). FreeBSD also tries stealing pages
from active processes in order to make space for disk cache. I suspect
this is why setting 'memory_pools off' on the non-NOVM squids on FreeBSD is
reported to work better - the VM/buffer system could be competing with
squid to cache the same pages. It's a pity that squid cannot use mmap() to
do file IO on the 4K chunks in it's memory pool (I can see that this is not
a simple thing to do though, but that won't stop me wishing. :-).

> You must be hardly using your swap at all given your statistics. If you
> type "free", or "top" you should get statistics saying that your swap usage
> is near zero - at least according to what you've said above.

Not necessarily.. There have been half a million pageouts with about 150K
of physical writes (ie: modified data pages and malloc space). There have
been about 230K of read page faults with physical IO on the system so far,
of which about 52K are attributed to squid.

> Jonathan L.

Cheers,
-Peter
Received on Thu Aug 14 1997 - 20:58:33 MDT