Perf: Lower FD use/avoid too-many-FDs-open: Specific FD savings + survey + possible Native_socket_stream_acceptor internal rejigger. #161

This ticket does concern generally one thing, but it has various aspects to it and various degrees of freedom plus a further survey component. So in that sense, apologies to reader/self to the catch-all nature of the ticket. Mitigating facts:

• Idea(s) below will be able to cut-down on FD use quickly, so we wouldn't need to do all of this right away, but if we were to do just idea(s) below, it might already be essentially fine.
• It's easier IMO to keep all the stuff in one place as opposed to shuffling around different inter-related tickets anyway.

Basic goal: Lower FD use so as to avoid too-many-FDs-open (FD, socket, handle, whatever) <=> EMFILE in Linux/probably POSIX.

Priority: It's not hair-on-fire, in that generally in modern OS including modern Linux one can raise the limit (default 1024), and this is done very typically in e.g. high-load server applications. It can these days be raised to millions without a problem IIRC. Moreover, while (as noted below) our FD use per-object (object being things like single Native_socket_stream, or single Session, or single Channel with NSS and/or MQs) sounds rough (IIRC in some contexts tens of FDs per object), it is still unlikely to be thing that drains a high-performance server of FDs. Usually only something much grander than IPC-channel use would be able to "achieve" that: Too many open WebSocket connections, say. So... not an emergency, but: This is a library, middleware or what not, and such things should be meek and careful in how they use resources. We aren't really, on this front. To summarize: Let's say medium-priority. Strong candidate for work.

--

First let's list likely high-impact ideas to actualize, probably, first. Low-hanging fruit.

The core thing is sync_io::Native_socket_stream; slightly less core are the sync_io::Blob_stream_mq_sender/receiver dudes. Here there's an immediate mini-explosion of FDs. Now, that isn't to say the FDs simply need not exist at all. Such a change is much more of an architectural change, if it's even practical at all; that'll fall under the open-ended "survey" task listed lower-down. No: here what I mean is that FDs are used in a PEER-state -- maybe even NULL-state in the case of Nss (but check both it and the MQ-dudes) -- object even before that object is at all used. And such an object may never be used. (Example: Nss_acceptor, as currently written, queues up any incoming connect()s without corresponding async_accept()s in a "surplus queue," and when it does so it keeps constructed-in-PEER-state sync_io::Nss objects. A haphazardly written application-set, such as a unit_test we've seen in error conditions, would mean these surplus-queue Nss objs would never actually get accepted, so they just sit around and then never bring any value. But they still make all them FDs.)

Also, it's not just FDs (but let's keep it in this ticket too). Associated (never used) threads are also started in some cases.

So, specifically:

• X=Nss + each MQ-dude: Keeps a Timer_event_emitter, which in ctor starts a thread with a boost.asio Timer. One schedules events, but instead of firing a handler as normal, it'll write a byte into a pipe, which sync_io::X user is to wait-upon and inform sync_io::X on pipe-readable. (sync_io::X, when informed, reads the byte and then proceeds to act on the timer event on which it was waiting.)
• Nss + each MQ-dude: Keeps a pipe-FD-pair for each timer-y task of interest. Namely there is one for the optional-to-even-use auto_ping() feature, and another for idle_timer_run() (ditto optional).

Thusly:

• Immediate improvement: Don't start Timer_event_emitter thread until a timer-wait is even needed (in practice: auto_ping() called, idle_timer_run() called).
• Immediate improvement: Don't create the pipe-FD-pair for feature F until feature F is invoked.
• More? Check Nss and separately each MQ-dude's impl: Are they creating FDs even before an API requires it? start_*ops(), replace_*()... those things. Need to be careful but generally just remember: These are linear state machines; they don't do anything to advance the state machine until a user API call, and the aforementioned one(s) is/are needed before even a PEER-state sync_io::X does anything at all.

Next: Native_socket_stream_acceptor: As previewed/spoiled above: It keeps that surplus queue of incoming connects, and if a user system (2-ish applications usually) issues connects without equal async_accept()s, then it'll create a PEER-state sync_io::Nss for each surplus connect. Through the idea above, this can cost mere 1 FD instead of the mini-explosion... so that's great; no Nss_a change needed.

However, even after the multiplier effect, one can still bombard it with connects up to EMFILE. Hence consider rejiggering Nss_a. Two ideas on that front. No matter what, though, consider the effect on the one known required use of Nss_a and Nss::sync_connect() (the others in Flow-IPC use the socketpair()+FD-transmit combo, not affected by this): Client_session::sync_connect() (<=> Nss::sync_connect()) and Session_server::async_accept() (<=> Nssa::async_accept()).

Whatever rejigger happens here can affect how that system works, so just contemplate that too. And... there's Flow-IPC/ipc_session#27 -- probably best to think about that one too here. So that said: the two ideas:

• Use kernel backlog as the surplus queue, instead of keeping our own (or in addition to it, per next bullet). By default in modern Linux IIRC that's 4096. But, not actually accepting for a full sync_io::Nss could affect some functionality expectations for sessions, so again, be careful. Goes without saying maybe, but there is commentary on Nssa, probably, as to why we chose to keep our own surplus queue. Read that before just blowing it away. But also context: Nssa is one of the oldest things written in Flow-IPC; those decisions may have lacked the benefit of subsequent relevant experience.
• Do still keep a surplus queue, but cap it. Once cap reached, revert to keeping it in kernel backlog. Fiddly for sure; if it can be avoided in favor of just bullet 1 (or... no bullet), then that's best, all else equal.

--

Beyond that: survey for unnecessary FD use. Start with the things already mentioned, sure, but there is more: In SHM-land at least there are various shm_open()-related FDs (SHMJ much more so than SHMC probably). In MQ-land there are those aux mini-SHM-pools used to guarantee a single reader + writer per MQ at most. In struc::-land -- who knows; look around.

P.S. This ticket is about quite a few thing; however it is not about handling actual EMFILE-or-equivalent more gracefully. Filing that separately; it really is basically orthogonal. Except that success in here makes success in there have less of an impact (lowers priority).