Transferring ownership of SSH connection from process A to B, letting A quit nicely?

Hello. I have an interesting conundrum I'm dealing with, and
would like some ideas on getting it to work.

I'm making a useless service for recreational purposes,
and it involves holding many SSH connections at once. Ideally
said service should be composed of only one process, accepting multiple clients, since they interact with each other in (mostly)
real-time, unlike a typical web server.

But because of how SSH works, each connection sits behind yet
another process. I'd have to employ some IPC for it to work,
but would end up making it a lot less scalable. What I wanted was
so that each connection would invoke the SSH shell (process A),
and said shell would somehow pass ownership of the connection
to the main server process (process B). Afterwards, process A
could safely exit, leaving the SSH connection intact. This would
mean process B would be in charge of multiple SSH connections.

I have tried sending the file descriptors associated with the connection to process B via sendmsg, thinking that maybe the
file descriptors are reference-counted. It's a logical
assumption, but it didn't work - the connection closed with
process A.

I have also tried opening a /dev/ttyp* file corresponding to the connection, but that didn't work either. So is this not possible under OpenBSD's semantics?

Scalability is not a real issue, but having IPC adds a load
of complexity I'd rather not get involved with.

Thanks.