kal_net_close closes the descriptor before retiring the handle, so a file opened at the same moment gets a stale handle and its I/O goes to descriptor 0 #53

With openkal-llvm-runtime 0.20.1 (openkal-musl 0.29.2 over openkal-linux 0.26.0, x86_64-linux-musl), closing a connected TCP socket on one thread while another thread opens a regular file sometimes gives the new file a handle that is already stale. Reads on that file then return 0 bytes, and writes on it go to descriptor 0.

kal_net_close closes the descriptor and only then retires the handle (net.cpp L247-L252). kal_net_close_listener (L254-L259) and kal_datagram_close (datagram.cpp L138-L143) do the same, while kal_fs_close_file retires first and closes second (fs.cpp L600-L603). Between the close and the retire, another thread's openat can be given the same descriptor number. pack gives the new handle the current generation of that number, and the retire then increments it (handle.h L20-L49), so from that point unpack refuses the new file's handle.

openkal-musl takes the file's stream under its descriptor table lock, after the open (okm_syscall.c L397-L418). A socket is closed and retired under the same lock (okm_fd.c L309-L316), so a file opened in the window always has a stale handle by the time its stream is taken. kal_fs_stream turns a handle that does not unpack into stream 0 (fs.cpp L609-L612), and from then on every read and write on the new C descriptor reaches descriptor 0. kal_fs_close_file skips a handle that does not unpack, so the file's own descriptor is never closed.

The order itself is old; openkal-linux 0.16.1 already has it. For regular files it started to matter with openkal-musl 0.26.0, which opens the file with the table lock released (the comment at okm_syscall.c L397); openkal-musl 0.20.1 held the lock across the open, so a file open and a socket close could not interleave. Sockets were already exposed before that: kal_net_accept and kal_net_connect run outside the lock (okm_net.c L403, L467, the same in 0.20.1), so a connection accepted or connected in the window also gets a stale handle, and kal_net_stream turns that into stream 0 as well (net.cpp L205-L212).

What I saw, on a 4-vCPU x86_64 VM, with a small program in which one thread accepts loopback TCP connections, sends a few bytes on each and closes it, another thread connects and closes its end, and a third opens, reads and closes an 89-byte file:

• With descriptor 0 on /dev/null, about 150 reads per second returned 0, and fstat and lseek on those descriptors failed. With a listener but no connections there were none in about 950,000 reads.
• With descriptor 0 opened on a writable file, records that a fourth thread appended to another file ended up in the descriptor 0 file, and so did bytes the accepting thread sent on accepted sockets (the accept case above).
• Each stale file leaks its descriptor. With a limit of 1024 the process ran out within about 5 seconds, after which open failed with EIO rather than EMFILE.
• With descriptor 0 on a pipe, the read blocked instead of returning.

A socket that is created and closed without connecting does not trigger it: in openkal-musl, socket makes no kernel descriptor, so its close never reaches kal_net_close.

The loops, reduced:

/* thread A, repeated: accept a connection on a loopback listener, send a few bytes, close it */
/* thread B, repeated: connect a TCP socket to that listener, close it */
/* thread C, repeated: */
int fd = open("data.txt", O_RDONLY);
ssize_t n = read(fd, buf, sizeof buf);   /* sometimes 0, although data.txt is not empty */
close(fd);

I can attach the full program if that helps.

Retiring before closing in the three socket close functions, as kal_fs_close_file does, should close the window for files and for accepted and connected sockets alike. The generation array is also read and incremented without atomics, which remains a data race between threads after the reordering. Separately, it may be worth having kal_fs_stream and kal_net_stream refuse a handle that does not unpack instead of mapping it to stream 0: stream 0 is the standard input, so a mistake there goes unnoticed.