i was trying to tickle a kernel panic, but instead
i think i found a bug. this program was run on
a machine with 1800 MB user space available.
(3552/464510 user)

#include
#include

enum{
Big = 1024*1024*1790,
};

void
main(void)
{
char *p;
ulong i;

you know what's going on here. read the subject you wrote.

the problem is not really as easy as it might seem at first.
malloc just moves the brk, but the backing pages don't get
allocated until the pages are accessed (during memset).

you could argue for some kind of accounting that would
ensure pages were available, but this could only be
terribly pessimistic, especially in the case of stacks
and fork.

russ

i'm just suprised that plan 9 overcommits. this makes
this code nonsensical from user space

if((p = malloc(Size)) == 0)
/* malloc recovery code */
/* why bother? the kernel could be lying to us anyway. */

also, by default plan 9 assumes that there are 16 MB of memory
more than actually exist. i would think with today's large memories
typically, one would want to assume the worst and underestimate
the amount of memory required so malloc would fail rather than
getting a page fault later.

Russ:
| you could argue for some kind of accounting that would
| ensure pages were available, but this could only be
| terribly pessimistic, especially in the case of stacks
| and fork.

Still, that's the way unix worked. You can deal with the pessimism by
allocating lots of backing store, whereas with overcommit (at least on
linux) you just have to learn to live with processes dying randomly.

But you knew all that. :)

but most people can live with the overcommit, as witness the fact that
most of us do and never know it.

If you can't live with overcommit, maybe you need a wrapper that:
sets up to catch the note (I am assuming here that you get one; do you?)
malloc
zero memory you malloc'ed (which will get the pages in)
die reasonably if you get the note

Would that work?

ron

but why introduce unpredictability? who really programs as if
memory is not overcommited? i would bet that acme and most
residents of /sys/src/cmd could do quite bad things to you in these
cases. there's no waserror() in userland to wrap around memset.

how much memory can be wasted by assuming that all brk'ed memory
can be used?

- erik

but why introduce unpredictability? who really programs as if
memory is not overcommited? i would bet that acme and most
residents of /sys/src/cmd could do quite bad things to you in these
cases. there's no waserror() in userland to wrap around memset.

how much memory can be wasted by assuming that all brk'ed memory
can be used?

- erik

On Sun, Sep 02, 2007 at 06:47:17PM -0700, ron minnich wrote:

That's still a race. Getting all the memory at once is different from
probing for one page at a time and dealing with failures in the middle.

"erik quanstrom" quanstro.net> wrote in message
news:d7c705e9c3355e723c8f69677bf2d851@quanstro.net...

Indeed, when I discovered that Linux was overcommitting memory
in much the same way, which in my view is erroneous design, I
added an optional feature to my portable malloc implementation
that (on systems neeing it) would touch every allocated page
before reporting success. (During the touch loop a trap catcher
would set a flag that would be tested after the loop to determine
whether there had ben any faults.) malloc really shouldn't have
to do this, but if you're insisting on overcommitment in the OS
then the library malloc() needs to do it. Otherwise, applications
that try to be careful may still fail "randomly", which is intolerable.

"erik quanstrom" quanstro.net> wrote in message
news:d81b7ef468d08ecf9e31985f92a9b0c4@quanstro.net...

Practically everybody.

You don't have to do that if malloc takes are of it. (See my nearby
posting.)

If you allocate the storage, presumably you should expect to use it.

Anyway, it's better to be wasteful and correct than randomly failing.