Add io_uring event loop (linux) (#16264)
Implements an event loop that leverages io_uring on Linux targets.
Requirements
The event loop requires different features that have been added in different versions of the kernel. At a minimum Linux 5.19 is required, while the recent Linux 6.13 is recommended. It is thus compatible with Linux 6.1 SLTS but not previous (S)LTS kernels.
The io_uring event loop is disabled by default. It must be enabled manually at compile time with the -Devloop=io_uring flag.
The SQPOLL feature is support but disabled by default. It allows to avoid syscalls on submissions & completions which is very cool… but it uses lots of CPU 🔥. It can be enabled at compile time with the IORING_SQ_THREAD_IDLE environment variable (in milliseconds) that sets the idle time for the SQPOLL thread.
For example:
export IORING_SQ_THREAD_IDLE=200
crystal build app.cr -Devloop=io_uring
Implementation details
The basic implementation was straightforward. It’s basically an async framework: submit an operation, suspend the fiber, and resume it when the operation has completed.
This is also the second event loop that uses blocking IO after IOCP on Windows, and the first one on UNIX.
The main issue is a Linux limitation where close doesn’t interrupt pending operations in the kernel, so we must shutdown sockets and cancel pending ops on files for example.
Threads Support & Safety
The MT safe implementation (preview_mt, execution_context) was much more complex. Unlike the other event loops, we can’t have a single ring as it would require to lock on every submit, and with multiple threads it would create a contention and would likely require syscalls (that would defeat the point), so we need a ring per thread (sharing the same kernel resources).
There’s thus a new API to register execution context schedulers to the event loop, so we can create/close rings as needed. Since a scheduler can shutdown (e.g. after a resize down), the execution context must also drain its ring before the scheduler can stop: all the pending operations must have completed and all the pending fibers be enqueued.
We need cross rings communication for a couple scenarios: to interrupt a thread waiting on the event loop, and for cancelling pending read/write file operations (the serial R/W of #16209 is required). At worst, this communication needs a lock on submit (which is avoided on Linux 6.13+). Unlike the single ring, the lock should usually not be contented in practice (unless you open lots of files, read/write from many fibers to the same file and close from whatever fiber).
Unlike the other event loops, there isn’t a single system instance for the whole event loop (e.g. one epoll, kqueue or IOCP), and each scheduler is responsible for its own completion queue… which means that we’re back into the “a busy thread can block runnable fibers” in its completion queue while there might be starving threads. A busy thread can be a CPU bound fiber, or a pair of fibers that keep re-enqueue each other.
To avoid this situation, once in a while + every time a scheduler would wait on the event loop (starving), the event loop will instead iterate the completion rings and try to steal runnable fibers from other threads. That requires a lock on the completion queue, that should also usually not be contended (only once in a while).