|
http://lists.freebsd.org/pipermail/...May/000695.html
On 6 May 2003, at 9:55, Bogdan TARU wrote:
> /kernel: pmap_collect: collecting pv entries -- suggest increasing
> PMAP_SHPGPERPROC
>
> The machine being more than 200km away, I cannot compile a new kernel
> right away. So I was wondering if there are any tweaks that I can do
> 'remotely', to solve the problem at least temporary.
We ran into this problem on a similar system to your setup. Here
is what we learned:
PV Entries are directly related the number of simultaneous children
processes Apache is allowed to spawn.
To figure out how many PV entries your system is using per
Apache process, run this command:
ipcs -a ; sysctl vm.zone | grep PV
Take a look at the "shared" memory section and find the NATTCH
value for Apache. That number is the total number of Apache
processes that have attached to that shared memory segment.
The last line of output is your PV entry stats. The third number in
from the left is the number of PV entries currently in use.
Since there are other things using PV entries on the system
(usually very few - Apache is the big consumer on a web server),
this isn't an exact science, but take the number of PV entries in
use and divide it by the number of Apache processes. That will
give you a rough estimate of how many PV entries are in use per
Apache process.
I don't fully understand the code (pageout daemon) that calls the
code that generates that warning, but in practical use, the warning
usually means that you don't have enough pages on the PV entry
"free" list and it has to compete with other page consumers to get
more pages from the system.
Generally, it isn't a serious problem unless you see 5 of those
messages since the last boot or you're seeing panics from running
out of PV entries. Five is the maximum warnings you'll see of that
type. Any activity after that that would have generated the warning
has the warning suppressed, so if you hit five of those messages
from boot to present time, you have a problem that needs to be
addressed.
To address the problem, you'll need to tweak your Apache config
and restart Apache. Decrease KeepAliveTimeOut or turn off
KeepAlive. Set MaxClients to a number that will prevent your
system from being crashed by having too many PV entries in use
and yet still handle the load after reducing KeepAliveTimeout.
To determine MaxClients, take your maximum PV entries and
divide it by the number of PV entries per Apache process that you
calculated above. Then, subtract a few clients from the result (10
or 20) and that should be a good number for MaxClients.
The Apache status page will be your friend when tweaking your
Apache config so that it efficiently handles the load without hitting
MaxClients. It's off by default, so you'll have to turn it on in your
Apache config if it isn't already on.
To view your current maximum for PV entries, look at 'sysctl
vm.zone' and look at the "limit" column for PV ENTRY.
Determining the maximum PV entries your system will support
after tweaking PMAP_SHPGPERPROC isn't the easiest of tasks,
but here's the basic idea:
pv_entry_max = (shpgperproc * maxproc) + (((((availmem - 1) * 2) +
1) / pagesize) - firstpage)
Availmem can be found by doing a 'less /var/run/dmesg.boot' and
looking at the available memory figure in bytes (not Kbytes).
Pagesize is typically 4096 bytes on FreeBSD 4.7 i386. I'm not
certain on this, but I believe firstpage is the same as pagesize.
Maxproc can be found with a 'sysctl kern.maxproc'. Shpgperproc
defaults to 200 on FreeBSD 4.7, but may be higher depending on
your kernel config.
Just remember that you can't set PMAP_SHPGPERPROC
arbitrarily high or you could overflow your KVA without ever hitting
the limit for PV ENTRY. It's best to nudge it upwards gradually
until the "collecting PV entries" goes away, provided you've
tweaked Apache as suggested above.
I know it's kludgy and annoying, but it's just what us heavy Apache
users have to deal with to avoid crashes due to the state of the
FreeBSD VM system. The general idea is that FreeBSD is
designed for systems that don't do a lot of memory sharing
between overgrown parent and children processes.
Here's an article I found awhile back by Matthew Dillon (FreeBSD
VM guru) that outlines the issue with FreeBSD and memory
sharing between large numbers of large processes:
http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/x106.html
Most of these types of problems will go away when Apache 2.0
becomes useable by most people (missing many vital modules at
the present time) due to it's threading model (vs. spawning
children). For now, we've just got to deal with it, unless someone
with the right expertise is willing to create a kernel config option
that changes how PV Entries are allocated for systems that have
many big fat memory sharing processes.
Sincerely,
Andrew Kinney
President and
Chief Technology Officer
Advantagecom Networks, Inc.
http://www.advantagecom.net
|
[FreeBSD] collecting pv entries -- suggest increasing PMAP_SHPGPERPROC
