Re: Memory upgrade

On Tue, Oct 15, 2024 at 02:35:03PM +0200, Christian Schulte wrote:
> On 10/15/24 12:45, Claudio Jeker wrote:
> > On Tue, Oct 15, 2024 at 12:28:20PM +0200, Christian Schulte wrote:
> >> On 10/15/24 12:09, Stuart Henderson wrote:
> >>> On 2024-10-15, Zé Loff <zeloff@zeloff.org> wrote:
> >>>> On Tue, Oct 15, 2024 at 10:14:42AM +0200, Christian Schulte wrote:
> >>>>> ulimit -d `ulimit -aH | grep data | awk '{print $2}'`
> >>>>> ulimit -n `ulimit -aH | grep nofiles | awk '{print $2}'`
> >>>
> >>> ulimit -d `ulimit -dH` etc... but then there's no point setting a
> >>> separate hard limit in login.conf.
> >>
> >> Of course. I am the only user on that system and the only limits I want
> >> "my" xsession to be in effect on that system are the hard limits setup
> >> by the kernel. Those make the system swap for no apparent reasons. So.
> >> Why is this thing swapping?
> >
> > Because you are out of memory (most probably the usual amd64 problem of
> > running out of dma reachable memory and the pagedaemon going berserk about
> > that). You have plenty of ram just in the wrong spot.
> >
>
> According to the readings of top(1) or vmstat(8) I am not hitting any
> physical RAM limits. Still. The system starts swapping and I am yet to
> find out why it does.

See Otto's reply. At some point *in the past*, you ran out of dma
reachable memory. To make room for the "request for a larger chunk of
memory" (which perhaps is larger than it needed to be, given that
ulimit) the system moves some stuff to swap. Then the greedy process
exits or simply frees up the "larger chunk of memory" it requested, and
you end up with free physical memory. If the swapped pages aren't
accessed, the system won't spend time and I/O moving them out of swap.

> Maybe it just cannot fulfill requests for larger chunks of memory but
> does not "tell" an application about it and just commits itself to
> swapping?

If the request doesn't exceed any (u)limit, and if there is swap
available, the system will find a way to accommodate that request, and
send some stuff to swap. That's exactly what swap space is there for.

> Makes no sense to me reading output of
> top(1) or vmstat(8) displaying that the system has swapped out more than
> half a GB to disk when nearly half of the RAM available to the system
> (8GB) is not even wired up. The system reports nearly 4GB of physical
> RAM available for allocation together with more than half of a GB
> swapped out to disk. Makes no sense.

Yes it does make sense. There is no point in spending resources to
preemptively move things back from swap. When those pages are needed,
and only then, they'll be read from the swap space, and moved back to
RAM. The system had to read them anyway, so it might as well write them
elsewhere, now that there's room available.

What makes no sense is to spend time reading a few hundred MB (GB?) of
from disk, just for the sake of moving things back to RAM, and then
risking having to repeat this whole process again, when a "large chunk
of memory" is requested. If a process repeatedly reserved and freed
large amounts of memory, you'd be moving pages in and out of swap again
and again, even if those pages weren't accessed at all. Just leave them
there quietly (you're not using them anyway), and when you *do* have to
read them, then take the opportunity to move them out.

>
> --
> Christian
>

--