0.22.0: a node has one identity however it was reached; a closed attenuation gives its place back

.github/workflows/ci.yml

@@ -39,13 +39,18 @@ jobs:
   # On the system itself.
   conformance:
     name: conformance (${{ matrix.name }})
-    runs-on: windows-2022
+    runs-on: ${{ matrix.os || 'windows-2022' }}
     timeout-minutes: 60
     strategy:
       fail-fast: false
       matrix:
         include:
           - { name: 'llvm, msvc ABI', toolchain: 'llvm@20.1.7',  target: '' }
+          # Windows 11 24H2's build (26100): the system that has the record a
+          # name is answered by in one request, and the only one on which a
+          # name's identity could differ from its open file's
+          # (tests/one_identity_for_a_node.cpp).
+          - { name: 'llvm, msvc ABI, build 26100', toolchain: 'llvm@20.1.7',  target: '', os: 'windows-2025' }
           - { name: 'msvc',           toolchain: 'msvc@system',  target: '' }
           - { name: 'gcc, gnu CRT',   toolchain: 'gcc@16.1.0',   target: 'x86_64-windows-gnu' }
     defaults:

README.md

@@ -7,14 +7,42 @@ An implementation of [openkal](https://github.com/mcpplibs/openkal) for Windows.
 openkal = "0.25.0"
 
 [target.'cfg(windows)'.dependencies]
-openkal-windows = "0.21.0"
+openkal-windows = "0.22.0"
 ```
 
 Its purpose is as much to test the specification as to be used. openkal was
 drawn against systems that share an ancestry; a third implementation on a system
 that shares none of it is what turns "this interface is not Unix's shape" from a
 claim into an observation.
 
+## What 0.22 corrects
+
+**A node has one identity, however it was reached.** The identity `kal_fs_info`
+and `kal_fs_file_info` report is the volume's serial number and the file's index.
+A handle's volume is asked `FileFsVolumeInformation`, whose serial number is 32
+bits; a name, where the system has Windows 11 24H2's `FileStatBasicInformation`,
+was answered with that record's serial number, the 64 bits the file system keeps.
+On builds 26100 and later a file that was open and the same file asked about by
+name were therefore two nodes, and a program that keys a table of files by
+identity held one file twice. The handle's word is the identity's, as it is on
+every earlier system and in a listing; the record's word is translated to it
+through what is learned by opening one node of each volume, so an enquiry by
+name stays one request after the first of a volume. Nothing is assumed about how
+the two numbers relate. (Found as mcpp-language-server issue 86: clangd as an ok
+image ended in its include analysis on Windows 11 24H2 alone.)
+
+**A closed attenuation gives its place back.** `kal_fs_dir_attenuate` keeps 64
+trees for the life of the program. A tree's place was taken by every attenuation
+and returned by none, so the 65th was refused with `kal_err_no_memory` however
+many had been closed --- a program granting three read-only directories to each
+program it starts was refused its 22nd start. A tree that keeps no record gives
+its place back when its last handle closes; one that keeps a record holds it
+until the program ends, which is when its record is ended. The 64 now bound the
+trees held at once.
+
+CI gains a row on `windows-2025` (build 26100), the system the first of these
+could be seen on.
+
 ## What 0.21 adds (openkal 0.25)
 
 **A second name for a file** (`kal_fs_hard_link`) is the link record of

mcpp.toml

@@ -1,7 +1,7 @@
 [package]
 namespace   = "mcpplibs"
 name        = "openkal-windows"
-version     = "0.21.0"
+version     = "0.22.0"
 description = "An implementation of openkal for Windows, written on the Win32 interfaces and the object manager beneath them, using no C runtime symbol."
 license     = "Apache-2.0"
 

src/fs.cpp

@@ -890,10 +890,64 @@ namespace {
 volatile long long g_by_name = -1;   // -1 not asked; 0 none; 1 the stat record; 2 the basic one too
 pfn_NtQueryInformationByName_ g_query_by_name = nullptr;
 
+// ONE IDENTITY FOR A NODE, HOWEVER IT WAS REACHED (0.22.0).
+//
+// The identity's first word is the volume's serial number, and this system
+// states that number twice. A handle is asked FileFsVolumeInformation, whose
+// serial number is 32 bits; the record of Windows 11 24H2 (`volume_serial_number'
+// above) is the 64 bits the file system keeps. Written into the identity as
+// each came, the two differed for every file on such a system: a file opened
+// and the same file asked about by name were two nodes. A caller that keys a
+// table by identity --- a compiler's table of the files it has entered --- then
+// held one file twice, and a lookup by name of a file it had opened missed
+// (mcpp-language-server issue 86: clangd ended in its include analysis, on
+// builds 26100 and 26200 and on no earlier one, which has not the record).
+//
+// The word a handle gives is the identity's, because it is what every system
+// before 24H2 gave for a name too, and what a listing gives. How the 64 bits
+// relate to the 32 is the file system's own affair and is not assumed here: a
+// volume's two numbers are learned together, once, by opening a node of it ---
+// the enquiry that was to be answered by name is answered by opening instead,
+// as it is wherever the record is refused --- and every later enquiry of that
+// volume is one request again.
+struct volume_words { okw_u64 by_name; okw_u64 by_handle; };
+constexpr int kVolumes = 32;
+volume_words g_volumes[kVolumes];
+int g_volume_count = 0;
+SRWLOCK_ g_volume_lock{};
+
+bool volume_of(okw_u64 by_name, okw_u64& by_handle) {
+    bool found = false;
+    AcquireSRWLockExclusive(&g_volume_lock);
+    for (int i = 0; i < g_volume_count; ++i)
+        if (g_volumes[i].by_name == by_name) { by_handle = g_volumes[i].by_handle; found = true; break; }
+    ReleaseSRWLockExclusive(&g_volume_lock);
+    return found;
+}
+
+// A node of the volume was opened: its handle's word is the one the record's
+// stands for. A table that is full learns nothing more, and the volumes beyond
+// it are answered by opening every time, which is correct and slower.
+void learn_volume(okw_u64 by_name, void* h) {
+    struct { okw::file_fs_volume_information info; wchar_t label_tail[128]; } volume{};
+    okw::io_status_block s{};
+    const long rv = okw::NtQueryVolumeInformationFile(h, &s, &volume, sizeof volume,
+                                                      okw::fs_volume_information_class);
+    if (!(okw::ok(rv) || rv == okw::status_buffer_overflow)) return;
+    AcquireSRWLockExclusive(&g_volume_lock);
+    bool known = false;
+    for (int i = 0; i < g_volume_count; ++i) if (g_volumes[i].by_name == by_name) known = true;
+    if (!known && g_volume_count < kVolumes)
+        g_volumes[g_volume_count++] = { by_name, static_cast<okw_u64>(volume.info.serial_number) };
+    ReleaseSRWLockExclusive(&g_volume_lock);
+}
+
 
 // kal_ok or a kal_err when answered; 1 when the node is to be opened instead.
+// `learn' is set, with the record's word for the volume, when the node is to
+// be opened because that word's counterpart is not yet known.
 int info_by_name(void* root, const char* name, kal_uintptr len, kal_uintptr flags, kal_u32 wanted,
-                 kal_node_info* out, bool& absent) {
+                 kal_node_info* out, bool& absent, bool& learn, okw_u64& volume_by_name) {
     long long have = okw::atomic::load(&g_by_name);
     if (have < 0) {
         g_query_by_name = reinterpret_cast<pfn_NtQueryInformationByName_>(
@@ -942,8 +996,13 @@ int info_by_name(void* root, const char* name, kal_uintptr len, kal_uintptr flag
     // The volume's serial number, where the record did not give it.
     bool one_component = true;
     for (kal_uintptr i = 0; i < len; ++i) if (name[i] == '/' || name[i] == '\\') one_component = false;
-    okw_u64 serial = basic ? static_cast<okw_u64>(b.volume_serial_number) : 0;
-    bool have_serial = basic;
+    okw_u64 serial = 0;
+    bool have_serial = false;
+    if (basic && (wanted & KAL_INFO_IDENTITY)) {
+        volume_by_name = static_cast<okw_u64>(b.volume_serial_number);
+        if (!volume_of(volume_by_name, serial)) { learn = true; return 1; }
+        have_serial = true;
+    }
     if (!basic && (wanted & KAL_INFO_IDENTITY)) {
         if (!one_component) return 1;
         struct { okw::file_fs_volume_information info; wchar_t label_tail[128]; } volume{};
@@ -981,9 +1040,11 @@ int kal_fs_info(kal_dir base, const char* name, kal_uintptr len,
         fill_absent(out);
         return kal_ok;
     }
+    bool learn = false;
+    okw_u64 volume_by_name = 0;
     {
         bool absent = false;
-        const int e = info_by_name(root, name, len, flags, wanted, out, absent);
+        const int e = info_by_name(root, name, len, flags, wanted, out, absent, learn, volume_by_name);
         if (e != 1) {
             if (absent) fill_absent(out);
             if (e == kal_ok && rights) asked(root, !absent, name, len);
@@ -1013,6 +1074,9 @@ int kal_fs_info(kal_dir base, const char* name, kal_uintptr len,
         return e;
     }
     if (rights) asked(root, true, name, len);
+    // The node the name ends at, opened as the record described it (a link was
+    // sent here before its volume was looked at): its volume is the record's.
+    if (learn) learn_volume(volume_by_name, h);
     const int e = fill(h, wanted, out);
     okw::NtClose(h);
     return e;

src/rights.cpp

@@ -398,12 +398,27 @@ void forget(void* h) {
     const int ri = e->root;
     drop_entry(e);
     if (ri >= 0 && --g_roots[ri].refs <= 0) {
-        // The last handle of the tree: what it recorded is written now. The
-        // root itself stays until the program ends, which writes its `e'.
-        flush(g_roots[ri]);
-        g_roots[ri].names.clear();
-        g_roots[ri].listed.clear();
-        if (g_roots[ri].base) { okw::NtClose(g_roots[ri].base); g_roots[ri].base = nullptr; }
+        // The last handle of the tree: what it recorded is written now. A root
+        // that keeps a record stays until the program ends, which writes its
+        // `e'.
+        root& r = g_roots[ri];
+        flush(r);
+        r.names.clear();
+        r.listed.clear();
+        if (r.base) { okw::NtClose(r.base); r.base = nullptr; }
+        // A ROOT THAT KEEPS NO RECORD HAS NOTHING LEFT TO WRITE, AND ITS PLACE
+        // IS GIVEN BACK (0.22.0). The table has kRoots places for the life of
+        // the program, and a place was taken by every attenuation and returned
+        // by none: a program that grants the same three directories to each
+        // program it starts was refused its 22nd start (kal_err_no_memory,
+        // "out of memory" to its caller), with every earlier grant long closed
+        // (mcpp-language-server issue 86).
+        if (!r.record) {
+            give_back(r.rprefix, r.rplen + 1);
+            give_back(r.batch, kBatch);
+            r.seen.clear();
+            r = root{};
+        }
     }
 }
 

tests/attenuation_returns_its_place.cpp

@@ -0,0 +1,94 @@
+// A regression test: a directory attenuated and closed leaves nothing behind.
+//
+// `kal_fs_dir_attenuate` keeps a table of the trees it has made, 64 places for
+// the life of the program. A place was taken by every attenuation and given
+// back by none: closing the handle released what the tree had listed and kept
+// the place. A program that grants the same directories, read-only, to each
+// program it starts --- a language server and the engine it restarts --- was
+// refused once 64 grants had been made in all, with every one of them closed:
+// `kal_err_no_memory`, which its caller printed as "out of memory" on a machine
+// with memory to spare (mcpp-language-server issue 86: the 22nd engine start of
+// a session with three read-only grants, the 17th with four).
+//
+// A tree that keeps a record still holds its place until the program ends,
+// which is when its record is ended; this test attenuates without one.
+#include <cstdio>
+#include <string>
+#include <openkal/fs.h>
+#include <openkal/types.h>
+import mcpplibs.mtest;
+
+using namespace mcpplibs::mtest;
+
+namespace {
+
+void check(bool ok, const char* what) { expect(ok) << what; }
+
+} // namespace
+
+UTest attenuation_returns_its_place_test{"attenuation_returns_its_place", [] {
+    kal_dir cwd{};
+    bool haveDir = false;
+    {
+        const kal_uintptr n = kal_fs_preopen_count();
+        for (kal_uintptr i = 0; i < n && !haveDir; ++i) {
+            kal_dir d{};
+            char nm[1024];
+            kal_uintptr len = 0;
+            if (kal_fs_preopen(i, &d, nm, sizeof nm, &len) != kal_ok) continue;
+            cwd = d;
+            haveDir = true;
+        }
+    }
+    if (!haveDir) {
+        fail("a preopened directory to test in");
+        return;
+    }
+
+    // Four times the table, each attenuation closed before the next.
+    int refused = 0, first_refused = -1;
+    for (int i = 0; i < 256; ++i) {
+        kal_dir_rights rights{};
+        rights.self_size = sizeof rights;
+        rights.flags = (i % 2) ? (KAL_DIR_READ_ONLY | KAL_DIR_UNCHANGING) : KAL_DIR_READ_ONLY;
+        kal_dir held{};
+        const int rc = kal_fs_dir_attenuate(cwd, &rights, &held);
+        if (rc != kal_ok) {
+            if (first_refused < 0) { first_refused = i; std::printf("  attenuation %d -> %d\n", i + 1, rc); }
+            ++refused;
+            continue;
+        }
+        // The handle is of use while it is held: a name beneath it answers.
+        kal_node_info info = {};
+        info.self_size = sizeof info;
+        check(kal_fs_info(held, "okw-attenuation-probe.tmp", 25, 0, KAL_INFO_KIND, &info) == kal_ok, "the attenuated directory answers");
+        kal_fs_close_dir(held);
+    }
+    check(refused == 0, "256 attenuations, each closed before the next, are all made");
+
+    // And the table still bounds what is HELD at once: 64 trees, and the 65th is
+    // refused while they are; closing them gives every place back.
+    kal_dir many[80] = {};
+    int made = 0;
+    for (int i = 0; i < 80; ++i) {
+        kal_dir_rights rights{};
+        rights.self_size = sizeof rights;
+        rights.flags = KAL_DIR_READ_ONLY | KAL_DIR_UNCHANGING;
+        if (kal_fs_dir_attenuate(cwd, &rights, &many[made]) != kal_ok) break;
+        ++made;
+    }
+    std::printf("  held at once: %d\n", made);
+    check(made >= 1 && made < 80, "the trees held at once are bounded");
+    for (int i = 0; i < made; ++i) kal_fs_close_dir(many[i]);
+    {
+        kal_dir_rights rights{};
+        rights.self_size = sizeof rights;
+        rights.flags = KAL_DIR_READ_ONLY | KAL_DIR_UNCHANGING;
+        kal_dir again{};
+        const int rc = kal_fs_dir_attenuate(cwd, &rights, &again);
+        check(rc == kal_ok, "a directory is attenuated again once the held ones are closed");
+        if (rc == kal_ok) kal_fs_close_dir(again);
+    }
+
+    std::printf("openkal-windows: a closed attenuation gives its place back\n");
+}};