On Wed, Apr 27, 2022 at 11:32:06AM -0400, Nick Holland wrote:
> On 4/25/22 1:23 PM, Peter J. Philipp wrote:
> > Hi,
> >
> > I have an openbsd amsterdam vps and KARL is using up so much RAM that it
> > causes the system to swap. I recently upgraded it to 7.1 and it's the first
> > time I had a problem with this (that I noticed). I have tried to put KARL
> > into a login.conf'ed (32 MB data limit) user but ld doesn't like that at all
> > and exits with a memory allocation failure.
> >
> > What can I do to make KARL reorder_kernel use less memory without buying more
> > RAM? I've turned KARL off for now but that's not a real solution and I hate
> > it.
> >
> > Is there no option in the clang 13.0.0 linker to store what it would normally
> > store in memory to disk? I know it would be slow but KARL doesn't need to
> > be fast if it's backgrounded.
>
> yep. It is called "swap". You just reinvented swap. :)
> And KARL is backgrounded already.
Oh goody! When I look at this VPS it has 1 GB of RAM, 1/2 of all resources
are used up by daemons (around 512 MB) and the rest is in buffer cache and
free memory. I don't care much about 45 MB being swapped out to disk really
but if I'm correct this does have a negative impact on openbsd.amsterdam and
was the reason they gave us a free RAM upgrade before.
> > I've done some homework googling and found this:
> > https://stackoverflow.com/questions/25197570/llvm-clang-compile-error-with-memory-exhausted
> >
> > in the checked solution, 1 and 2 are sorta out of the question, but question is
> > whether we're using a Debug build of clang? Does anyone know off hand?
> >
> > While I'm here thinking about possible solutions it would be cool if I could
> > allocate a 128 MB vmm inside this vmm (cascaded vmm's?) with a stripped down
> > KARL building kernel and lots of swap, then it can swap all it wants to while
> > linking and it leaves the system in reasonable memory without swapping in
> > the main vm. Perhaps I'm thinking in over-engineering terms here?
>
> "I have a problem with memory consumption. I know! I'll solve it adding a VM!"
> Now you have many problems. I really don't think this is a good idea.
>
> How tiny is this VM??? My smallest intel box currently sitting around and
1 GB RAM.
> ready to go is a 400MHz celeron with 512MB RAM, i386 platform, so I just
> fired it back up and did a few sysupgrades to bring it up to 7.1-current (ok,
> "just" isn't applicable here, I started this test yesterday). I did a reboot
You can't compare i386 to amd64 the memory footprint is a lot smaller in i386.
But I see where you're going with this: <=512 MB RAM is too low for a machine.
> and as soon as I could log back in, did so and watched top -- ld topped out
> at about 270MB. That is admittedly huge for an OS I used to do builds on
> with 128MB and run in production with 32MB but a couple releases ago, I
> found that 384MB was the minimum needed to avoid swap on boot. Doesn't look
> much worse now (granted, i386 platform. I don't know what you are running).
>
> If you are trying to run <512MB RAM, I would politely suggest reconsidering
> some life choices here. :)
Not the case here. Although I'm constantly searching for new opportunities
which affect my life choices regardless.
> Alternatively, you might want to think about other options.
> KARL is great, but even without it, I think you will find OpenBSD is still far
> more robust and secure than the systems your bank runs on, so disabling KARL
> is not fatal in my mind for otherwise fairly secure systems. If you wish to
> get overly complicated, you could disable KARL on the production machine and
> relink a kernel periodically on ANOTHER machine and put it on the prod
> machine after it is built (there's your VM. Just don't put it on an already
> resource-starved system!)
This is a good idea and I haven't thought of it. I may do this.
> Another idea might be to slip "disknice" into /etc/rc where it rebuilds the
> kernel. It is a cute little bit of code TedU@ wrote a number of years ago,
> you can find it here:
> https://marc.info/?l=openbsd-misc&m=126526614419455&w=2
> It won't stop swapping, but *may* help other tasks get some time. I've found
> it useful on disk I/O tied tasks, but never tried it with a swap-bound task.
> I have no idea how it would impact a swapping process. Might solve your
> problem, might do nothing ("doing nothing" counts as hurting when you make
> changes to system scripts).
I like your first recommendation better somehow, thanks very much though.
> Nick.
>
Best Regards,
-peter
No comments:
Post a Comment