Add Windows support for the engine pipeline

Summary

src/pipeline/run-souffle.sh cannot run on Windows outside WSL/Cygwin. Soufflé itself is not the blocker, it builds natively with MSVC and is CI-tested on Windows. The gaps are in our pipeline: Unix-only header discovery, symlinked fact staging, and bash.

Soufflé already supports Windows

souffle-lang/souffle VS-CI-Tests.yml builds on windows-2025 with MSVC via vcpkg + chocolatey, currently targeting Visual Studio 18 2026.
souffle-lang/souffle#670 closed by a maintainer with "the code is now compatible with MSVC."
We already carry the Cygwin workaround from that same thread, run-souffle.sh:268:
```
case "$(uname -s)" in CYGWIN*) CXX_PLATFORM="-Wa,-mbig-obj";; esac
```

So partial Windows awareness exists; it's just incomplete.

What actually blocks it

1. Soufflé header discovery is Unix-prefix only, run-souffle.sh:56-59

for p in "$(brew --prefix souffle 2>/dev/null)" /usr/local /usr /opt/homebrew; do
  [ -d "$p/include/souffle" ] && { echo "$p/include/souffle"; return; }

A vcpkg install lands at $VCPKG_ROOT/installed/x64-windows/include/souffle, which is never searched. AXIOM_SOUFFLE_INCLUDE overrides it, so it works with one env var, it just isn't discovered, and the failure message reads like a broken install.

2. Library fact staging symlinks, the cached library signatures are ln -s'd into the facts dir. Native Windows needs Developer Mode or elevation for symlinks; Cygwin/MSYS are fine. A copy fallback when ln -s fails would be enough.

3. The pipeline is bash, fine under WSL2, Cygwin, MSYS2, Git Bash; not under cmd/PowerShell.

Proposed scope

Two options, worth deciding before implementing:

(a) conditional blocks in run-souffle.sh, add vcpkg/MSVC prefixes to the header search, fall back to cp when ln -s fails, extend the existing uname -s case. Keeps one script; the divergence is genuinely small.
(b) a separate run-souffle-windows.ps1, cleaner separation, but duplicates the staging/caching logic, which is the part most likely to drift.

Leaning (a), the only real differences are prefix search and symlink-vs-copy, and the Cygwin branch is already there.

Acceptance

test/python/run-tests.sh passes on Windows (WSL2 and at least one of Cygwin/MSYS2)
Soufflé headers discovered from a vcpkg install without setting AXIOM_SOUFFLE_INCLUDE
library staging works without symlink privileges
README states supported platforms and prerequisites (currently unstated, contributors discover them by hitting errors)

Notes

Nothing in the test suites is a blocker: verified no bash-4 features, no BSD/GNU-specific commands (md5 vs md5sum, sed -i '', stat -f), no absolute paths, and the suite passes from an arbitrary CWD and from a path containing spaces.

This needs to be true as the support for languages increases.

Re-surveyed this against main after #245 merged. The framing here holds up — soufflé is not the
blocker, our pipeline is — but three things have changed and one blocker is missing from the list.

Blocker 1 shrank, and its line refs are stale

Header discovery moved out of run-souffle.sh into src/pipeline/souffle-include.sh in #245, and
the Cygwin branch is now run-souffle.sh:282, not :268.

More usefully, the vcpkg case is now nearly free. #245 replaced the [ -d "$p/include/souffle" ]
test with a probe for the header the compiler actually opens, because Homebrew installs the headers
twice and a directory test cannot tell the layouts apart. vcpkg is the single-copy layout —
installed/x64-windows/include/souffle/*.h — which is exactly the layout the old test got wrong.
Verified locally on soufflé 2.5 that on a single-copy tree -I <prefix>/include/souffle fails with
fatal error: 'souffle/CompiledSouffle.h' file not found while -I <prefix>/include compiles, and
that souffle_include_under picks the latter. So this is no longer "add probing logic", it is one
entry in the fallback list:

"${VCPKG_ROOT:+$VCPKG_ROOT/installed/${VCPKG_DEFAULT_TRIPLET:-x64-windows}}"

Worth noting that before #245 this issue was unachievable on any single-copy layout, Windows
included — the engine could not be compiled at all. #245 is a prerequisite for this issue, not
merely adjacent to it.

The missing blocker: the compile invocation itself

run-souffle.sh:283 is a GCC/Clang driver line, and it is the largest piece of work here:

c++ -std=c++17 -O3 -march=native -w $CXX_PLATFORM -I "$INNER" "$INT/souffle-program.cpp" -o "$BIN.tmp.$$"

On a native MSVC toolchain every token differs — cl.exe, /std:c++17, /O2, /I, /Fe:, /w
— and -march=native has no MSVC equivalent at all (nearest is an explicit /arch:AVX2, which is a
different decision, not a translation). -Wa,-mbig-obj is a GNU assembler flag, so the existing
Cygwin branch only means anything under Cygwin's gcc; the MSVC form of that same 32768-section cap
is /bigobj, which is what soufflé's own CI passes (-DCMAKE_CXX_FLAGS=/bigobj), and its Windows
job carries the note "Visual Studio must be in the environment because cl.exe is required for
compiled Souffle" — it runs the synthesised suite, not just the interpreter, so souffle -g →
compile → run is genuinely supported upstream.

This means proposed scope (a) needs rewording. "Extend the existing uname -s case" undersells it:
you cannot extend a flag into a different compiler driver. What's needed is a small abstraction —
resolve a compiler and emit its flag vector — with two implementations. Still consistent with
preferring (a) over a separate .ps1, and I'd still prefer (a); it is just not a one-line case arm.

Two smaller items in the same area:

Path translation. Under Git Bash / MSYS2, -I "$INNER" and -o "$BIN.tmp.$$" hand cl.exe
MSYS-style paths (/c/vcpkg/...) that it cannot open. Needs cygpath -w at the boundary.
uname -s matching. Git Bash reports MINGW64_NT-* and MSYS2 MSYS_NT-*. The existing
CYGWIN* arm matches neither, so today the section-cap workaround does not apply on the two
shells people are most likely to use.

The note about spaced paths is no longer true

The closing note says the suite passes from a path containing spaces. That was true when written;
the gate added by #245 is not. test/tools/souffle-include-test.sh:26 word-splits the resolver it
bootstraps from, so with soufflé under a spaced prefix it prints SKIP and exits 0 — green, having
asserted nothing. Filed as #254 with the reproduction. It is a one-line quoting fix, but it matters
disproportionately here: C:\Program Files\... and C:\Users\First Last\... arrive as
/c/Program Files/..., so on Windows that gate would be permanently self-disabled while looking
green, on the platform whose include layout is least exercised. Worth taking before this issue, not
after.

Two additions to prerequisites and acceptance

No soufflé Windows binary exists. The 2.5 release ships only .deb and .rpm (checked the
release assets). So the Windows prerequisite is build-from-source via vcpkg + chocolatey, matching
what upstream CI does — a heavier ask than the README acceptance line implies, and worth stating
explicitly since it is the first thing a contributor hits.

The Parser is an unlisted prerequisite. Acceptance says test/python/run-tests.sh passes on
Windows, but the harness needs AXIOM_PARSER → Parser/dist/index.js, and the Parser depends on
tree-sitter, tree-sitter-java, tree-sitter-python and tree-sitter-groovy — node-gyp native
modules that need VS Build Tools on Windows, with no os field in its package.json. No blocking
logic found on that side (no process.platform branching in parser source), but the native build is
a real gate and the acceptance criterion cannot be met without it. It may deserve its own issue on
the Parser repo.

Shell layer is clear

Swept all 24 tracked .sh files for constructs that break under a Windows POSIX shell. Nothing
found: 12 process substitutions, 113 /dev/null, plus /dev/stderr, /dev/zero, readlink -f,
mktemp -d and shasum are all fine under Git Bash / MSYS2 / Cygwin. run-souffle.sh:282 is the
only uname branch in the tree. So the bash-ness of the pipeline is not the problem it looks like —
under WSL2 it should work today, post-#245, with no code change. That split is worth making explicit
in the scope: WSL2 is verification and documentation; native MSVC is the compiler abstraction
above. They are very different amounts of work and only the second needs any of it.

Ran the TypeScript front end end to end on Git Bash / MSYS2 (MINGW64_NT-10.0, uname -r 3.6.5) against a statically-linked Soufflé 2.x Windows build with MinGW-w64 as the souffle -g compiler. It works, and the 28-case suite is fully green on both passes with --oracle. Getting there needed five changes; three of them are not in this issue's list, and the two that are behave slightly differently from the description. Everything below is at engine 33b2958, parser 2de08cf (rebuilt, dist/index.js newer than HEAD).

Confirmed as described

uname -s matching — exactly right, and it is the first thing that bites. Git Bash reports MINGW64_NT-*, so the CYGWIN* arm matches nothing. Worth noting the arm turns out not to be needed on MinGW: the generated translation unit for the TypeScript rule set compiles without -mbig-obj (8.1 MB binary, 24 rule files / 482 distinct relations). So the section cap is not currently hit here — but the arm still needs to match for the other cases it guards.

Symlink privileges — real, and the failure is worse than "needs Developer Mode". Without SeCreateSymbolicLinkPrivilege, MSYS ln -s does not fail: it silently deep-copies the directory. So a copy fallback keyed on ln -s returning non-zero would never fire. Two of the TypeScript preflights (self-staging-test.sh, mirror-selflink-test.sh) build workspace self-link shapes with ln -s and then assert a predicate about them, so they fail on a shape the test never actually created — a false red about is_project_itself, which is correct. A Windows directory junction needs no privilege and MSYS reports it as a symlink with a correct realpath, which is what I used.

Not in the list

1. The generated #include paths, which is a different boundary from -I and -o.

The comment above has -I "$INNER" and -o "$BIN.tmp.$$" covered. But SRC is also interpolated into the generated program text:

SRC="$(cd "$(dirname "$0")/.." && pwd)"        # -> /c/…
echo "#include \"$DL/decls_base.dl\""          # -> #include "/c/…/decls_base.dl"

and Soufflé's own preprocessor is a native gcc.exe, which reads /c/… as <current drive>\c\…:

Pre-processor command failed with code 1: 'gcc.exe -x c -E -I "." -DRAM_DOMAIN_SIZE=64 "…/souffle-program.dl" 2> nul'
Uncaught exception: Failed to read input

Side effect worth knowing because it is confusing to debug: this leaves a stray C:\c\<…> directory tree on disk, created by whichever native tool got the path next. cygpath -m "$SRC" right after the assignment fixes every derived path at once ($DL, $ENG, $ENG2, $TPL), and the mixed form is accepted by both MSYS bash and the native tools, so no other line needs to change.

2. The cached engine binary has no executable suffix, and the cache-hit test disagrees with the exec.

BIN="$CACHE_DIR/souffle-engine-$NEW"
if [ ! -x "$BIN" ]; then … fi
"$BIN" -F "$FACTS" -D "$OUT"

MSYS execve resolves a bare name by appending .exe, so a PE stored under an extension-less name is ENOENT and bash reports rc=127. Meanwhile [ -x "$BIN" ] is true, because stat() does append .exe. So on a warm cache the script prints ▶ reusing cached binary and then cannot run it. Verified directly: the same bytes exit 0 when the file is named *.exe and 127 when it is not, on the same PATH.

An EXE=".exe" under the same uname -s case arm covers it; note -o "$BIN.tmp.$$" does not need changing, because MinGW gcc only appends .exe when the output name has no extension and .tmp.<pid> counts as one.

3. A transient file lock fails a solve that already finished.

rm -rf "$FACTS" "$INT/souffle-program.cpp"

runs after every output relation is written. On Windows a just-written multi-megabyte .facts file is still briefly held, so:

rm: cannot remove '…/lib_ts_type_reference.facts': Device or resource busy

and set -e turns a complete solve into solve failed. Measured: 9,748 rows already written to out/call-chain-edges.csv, then exit 1. Retrying the identical rm a few seconds later succeeds every time. Since this is per-run scratch deleted after all measurement is on disk, it should not be able to fail the run at all — a retry loop plus || true is enough, and that is true on every platform.

One correction to the Notes

Nothing in the test suites is a blocker

Not true for test/typescript/, in three separate places, all in the ground-truth stack rather than the pipeline. I have filed them separately rather than expanding this issue, since they are test-harness mechanisms and not pipeline portability:

the call-site join compares a raw relative-path string while target identity goes through realpath, so nothing joins and the dispatch-envelope block reports 0 / precision 0.000 instead of refusing;
.source-root is written by the shell and resolved by the Python readers, so every staged declaration lands under a nonexistent root and the client's own files are reported as TARGET NOT STAGED;
tools/signature-impl-test.sh asserts POSIX path syntax about a platform-native emitter, so it is a false red that aborts the whole suite before any case runs, while the mechanism it guards is working.

Two smaller ones, not filed:

run-evaluation.sh needs rsync, which ships with neither Git for Windows nor MSYS2. Only one invocation shape is used (-a plus --exclude), and tar has the same --exclude semantics for those patterns, so a fallback is cheap if that is wanted.
core.autocrlf is true in Git for Windows' system config, so a default clone materialises every committed LF golden as CRLF and check_golden's diff -q then fails on line endings alone, for every case, in all three languages. There is no .gitattributes in the repo. I worked around it with git -c core.autocrlf=false worktree add, but a one-line .gitattributes would mean no Windows contributor ever sees it.

Acceptance criteria, as they stand today

Against this issue's list, with the five changes above applied locally (nothing committed):

the TypeScript suite passes: 28/28 cases, both passes, --oracle, every committed golden byte-for-byte
library staging works without symlink privileges — via junctions rather than a copy fallback
souffle -g → compile → run works end to end under MinGW
headers discovered without AXIOM_SOUFFLE_INCLUDE — not tested; I set the variable
test/python/run-tests.sh — not run

Happy to hand over the exact diff for the three pipeline items if that is useful; it is 12 added lines in src/pipeline/run-souffle.sh and touches no rule, golden or template.

Merge pull request #16 from AxiomCodeAI/ts/parser-fixes-round-2

Fresh Windows 11 dev setup via `npm link`: two gaps not listed above

Verified at f60e4d4f on Windows 11, Node 24.11.1, npm 11.6.2, Git for Windows (default install), with no Soufflé, no C++ compiler and no WSL.

What works: npm install (including the prepare build of parser and dist; the unpublished @axiomcode/engine-* optionals are skipped without error), npm link, axiomcode --help, and parse + staging from any directory in Git Bash. axiomcode mcp completes initialize and tools/list. The solve then stops at the expected ❌ no engine for java@687b027f…, which is the gap already tracked in #454/#904/#1225.

1. The linked command fails in PowerShell and cmd with the default Git for Windows PATH

Now tracked in #1229, with a tested fix (a Node launcher as the bin entry). The notes below are kept for context.

npm's generated shims (%APPDATA%\npm\axiomcode.ps1 and .cmd) run bash.exe. The default Git for Windows install puts only Git\cmd on PATH, not Git\bin, so:

PS> axiomcode --help
& : The term 'bash.exe' is not recognized as the name of a cmdlet, function, script file, or operable program.
At C:\Users\<user>\AppData\Roaming\npm\axiomcode.ps1:24 char:7

The same command works in Git Bash. The README does require "a POSIX shell (Git Bash on Windows)", but a user who runs npm link or npm i -g and then opens PowerShell gets a message that points at neither axiomcode nor the fix. Possible fixes: document "run from Git Bash, or add C:\Program Files\Git\bin to PATH", or ship a small Node bin launcher that locates Git's bash (the MCP config path npx -y @axiomcode/code-graph mcp will hit the same thing).

Also note that on a machine that has WSL, bash.exe resolves to C:\Windows\System32\bash.exe, which is WSL bash and not Git Bash, so the shim then runs the pipeline under a different environment. This part is unverified here because WSL isn't installed.

2. The "no engine" message has no Windows route

graph/pipeline/run-souffle.sh:450:

• install souffle 2.5 to compile locally (macOS: brew install souffle; Ubuntu: the .deb from souffle-lang/souffle releases).

On Windows neither hint applies, and there is no packaged Soufflé 2.5 for native Windows. So until the engine packages are published, the message leaves a Windows developer with no actionable step. A Windows line (WSL2, or the MSYS2/MinGW route described in the comment above) would close that.

Navigation Menu

Repository navigation

Add Windows support for the engine pipeline #16

Summary

Soufflé already supports Windows

What actually blocks it

Proposed scope

Acceptance

Notes

Blocker 1 shrank, and its line refs are stale

The missing blocker: the compile invocation itself

The note about spaced paths is no longer true

Two additions to prerequisites and acceptance

Shell layer is clear

Confirmed as described

Not in the list

One correction to the Notes

Acceptance criteria, as they stand today

Fresh Windows 11 dev setup via `npm link`: two gaps not listed above

1. The linked command fails in PowerShell and cmd with the default Git for Windows PATH

2. The "no engine" message has no Windows route

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add Windows support for the engine pipeline #16

Description

Summary

Soufflé already supports Windows

What actually blocks it

Proposed scope

Acceptance

Notes

Activity

swapnilpaliwal-sd commented on Aug 30, 2026

swapnilpaliwal-sd commented on Sep 7, 2026

Blocker 1 shrank, and its line refs are stale

The missing blocker: the compile invocation itself

The note about spaced paths is no longer true

Two additions to prerequisites and acceptance

Shell layer is clear

Whua689 commented on Sep 9, 2026

Confirmed as described

Not in the list

One correction to the Notes

Acceptance criteria, as they stand today

Whua689 commented on Sep 24, 2026

Fresh Windows 11 dev setup via npm link: two gaps not listed above

1. The linked command fails in PowerShell and cmd with the default Git for Windows PATH

2. The "no engine" message has no Windows route

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Fresh Windows 11 dev setup via `npm link`: two gaps not listed above