Wednesday, July 04, 2018

Re: "Cannot allocate memory" error when memory is enough

HI Philip,

Thanks very much for your detailed explanation!

My OS is 6.3. I already use "pkg_add -u" to upgrade all installed
packages. cmake and egdb are are installed by "pkg_add", not compiled
by me.

"vmstat -m" gives some information:

$ vmstat -m
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 752 2832 2659213 1280 3
32 482 1054 691465 640 99
64 622 4114 1023682 320 191519
128 4496 560 27265523 160 3596104
256 164 364 107121 80 19059
512 387 197 63372 40 18454
1024 1507 5 125086 20 0
2048 36 4 2193 10 0
4096 555 1 80673 5 0
8192 207 1 450 5 0
16384 10 0 15 5 0
32768 9 0 29163 5 0
65536 9 0 21946371 5 0
262144 3 0 3 5 0
524288 2 0 2 5 0

......

Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
devbuf 2991 7220K 7220K 78644K 146347 0 0
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,262144,524288
......
dirhash 678 130K 239K 78644K 18387 0 0
16,32,64,128,256,512
......
ttys 408 1724K 1724K 78644K 408 0 0
512,1024,4096,8192
......
VM swap 7 299K 299K 78644K 7 0 0
16,64,2048,262144
UVM amap 295 12K 441K 78644K 4302813 0 0
16,32,64,128,256,512
......
temp 54 2082K 2211K 78644K 22808735 0 0
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,524288
......
DRM 275 114K 116K 78644K 1410 0 0
16,32,64,128,256,512,1024,2048,4096,16384

Memory Totals: In Use Free Requests
12378K 619K 53994333
Memory resource pool statistics
Name Size Requests Fail InUse Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
phpool 112 84953 0 4236 125 1 124 124 0 8 0
extentpl 40 126 0 48 1 0 1 1 0 8 0
pmappl 192 66221 0 34 105 103 2 3 0 8 0
......
In use 34552K, total allocated 40384K; utilization 85.6%


It seems kernel dynamic memory is run out, and devbuf and temp consume
most of the space.

Could you give some suggestions? Thanks very much in advance!
Best Regards
Nan Xiao


On Wed, Jul 4, 2018 at 10:57 AM, Philip Guenther <guenther@gmail.com> wrote:
> On Tue, 3 Jul 2018, Philip Guenther wrote:
> <nothing of interest>
>
> Flakey button on my mouse; time to clean it again and throw it out if it
> keeps glitching. Sorry about that.
>
>
>> On Tue, Jul 3, 2018 at 4:53 PM Nan Xiao <xiaonan830818@gmail.com> wrote:
>> > Thanks for your reply! The "ulimit -a" outputs following:
>> >
>> > $ ulimit -a
>> > time(cpu-seconds) unlimited
>> > file(blocks) unlimited
>> > coredump(blocks) unlimited
>> > data(kbytes) 33554432
>> > stack(kbytes) 8192
>> > lockedmem(kbytes) 1332328
>> > memory(kbytes) 3978716
>> > nofiles(descriptors) 128
>> > processes 1310
>> >
>> > It seems should be enough to launch cmake or egdb.
>
> But it wasn't and the kernel can only indicate that with a single error
> code, so now you have to actually dig into what's going on. There are
> many possibilities, as a search for ENOMEM in /usr/src/sys/kern/*exec*.c
> will show.
> 1) the ELF interpreter (normal ld.so) could be too large
> 2) the PT_OPENBSD_RANDOMIZE segment could be larger than permitted by the
> kernel
> 3) program's text segment could exceed the maximum for the arch, MAXTSIZ
> 4) the program's vnode couldn't be mmaped for some reason
> 5) the argument list and environment were together too big for the stack
> 6) the signal trampoline couldn't be mapped into the process VM
> 7) other random memory allocation problems
>
> Of those, (1), (4), and (6) are *really* unlikely. (3) is possible if
> you're building a debugging binary that's *huge* as a result. (5) would
> result in _all_ programs failing in that shell. I think (7) would show up
> in a close examination of the "vmstat -m" output.
>
> (2) is perhaps the most likely, as recent compiler changes have increased
> the expected size of the PT_OPENBSD_RANDOMIZE segment and while the kernel
> limit on that was also increased recently, you didn't provide any
> information about your setup: are your kernel, userland, and ports all in
> sync?
>
>
> Philip Guenther

No comments:

Post a Comment