Add WASM support: --wasm flag with Node.js and Deno runtimes#4176
Add WASM support: --wasm flag with Node.js and Deno runtimes#4176lostflydev wants to merge 6 commits into
Conversation
|
@lostflydev I will go over the review in the coming days (this is a hefty one, might take a bit), but in the meantime - it seems the reference doc hasn't been generated. |
| * Deno is first looked up on the system PATH. If not found, it is downloaded from GitHub releases | ||
| * and cached via Coursier's ArchiveCache. | ||
| */ | ||
| object WasmRuntimeDownloader { |
There was a problem hiding this comment.
what is the current behavior for scala-cli run --js foo.scala? i could be wrong but i would think it does not download a runtime
There was a problem hiding this comment.
Yeah, I also wondering the current behavior for that command.
I'm not sure about the policy on downloading the runtime through scala-cli (I personally don't like it though), but I believe downloading runtime stuff should be in different PR, because it's not very relevant to "supporing wasm".
There was a problem hiding this comment.
I'd treat WASM runtimes similar to how we treat node with Scala.js - the user needs to download it themselves, rather than Scala CLI doing it for them.
There was a problem hiding this comment.
Scala-cli has --jvm flag and downloads jvm by itself. This behaviour with wasm runtime downloading looks similar.
There was a problem hiding this comment.
@dos65 JVM is key to Scala CLI's core functionalities, and thus it's treated as a major dependency.
Node is considered necessary setup for Scala.js, similar to how clang is for Scala Native.
I understand this is blurry and vague, but I think we will expect the user to install WASM runtimes on their own, rather than do it out of the box here (certainly out of scope for this PR, but feel free to start a dedicated discussion or issue on this as a follow-up).
There was a problem hiding this comment.
There was a problem hiding this comment.
You got me there. 😅
Not sure why did I define the requirements this way when creating the issue.
Will amend it there.
My bad.
| * Deno is first looked up on the system PATH. If not found, it is downloaded from GitHub releases | ||
| * and cached via Coursier's ArchiveCache. | ||
| */ | ||
| object WasmRuntimeDownloader { |
There was a problem hiding this comment.
Yeah, I also wondering the current behavior for that command.
I'm not sure about the policy on downloading the runtime through scala-cli (I personally don't like it though), but I believe downloading runtime stuff should be in different PR, because it's not very relevant to "supporing wasm".
| // Standalone runtimes (future - requires upstream Scala.js standalone WASM support) | ||
| case object Wasmtime extends WasmRuntime("wasmtime") | ||
| case object WasmEdge extends WasmRuntime("wasmedge") | ||
| case object Wasmer extends WasmRuntime("wasmer") |
There was a problem hiding this comment.
If they don't work, I feel we don't need to add those options yet
|
(converted to a draft, as this clearly needs more work; feel free to change it back when it's ready to review) |
|
(I can see the post-review changes, but a rebase will be necessary to re-run the CI) |
80fe137 to
40df564
Compare
Hi @Gedochao ! I`ve rebased onto actual main, could you pls re-run CI/CD |
40df564 to
f138e4c
Compare
3358e24 to
7f43349
Compare
|
why not use bun as a runtime |
7f43349 to
0794656
Compare
0794656 to
721bbcf
Compare
@He-Pin Thanks for comment, added bun as a runtime |
721bbcf to
714e11a
Compare
714e11a to
077bf00
Compare
|
Hi @Gedochao, could you please re-run the failed native-windows-tests-default (https://github.com/VirtusLab/scala-cli/actions/runs/25658534589/job/75315398257?pr=4176#logs) job? I think it might be flaky |
|
Yep, almost certainly flaky. I restarted it. |
|
Hi @tanishiking @sjrd @Florian3k @lbialy @dos65 👋 Gentle ping — the PR is ready for another look. Addressed previous feedback: dropped runtime auto-download and unsupported standalone runtimes, added Bun support, rebased, fixed style and docs. Whenever you have a spare moment, I'd really appreciate your feedback |
| final case class WasmOptions( | ||
| @Group(HelpGroup.Wasm.toString) | ||
| @Tag(tags.experimental) | ||
| @HelpMessage("Enable WebAssembly output (Scala.js WASM backend). Uses Node.js by default. To show more options for WASM pass `--help-wasm`") |
There was a problem hiding this comment.
Let's use "Wasm" instead of "WASM" ;)
A contraction of “WebAssembly”, not an acronym, hence not using all-caps.
https://webassembly.github.io/spec/core/intro/introduction.html#wasm
There was a problem hiding this comment.
Thx, fixed it, in all @helpmessage annotations, scaladocs, error messages, and code comments
| } | ||
| else if (emitWasm) { | ||
| // For WASM mode with ES modules, run node directly instead of NodeJSEnv. | ||
| // NodeJSEnv's stdin piping with "-" doesn't work with Input.ESModule. |
There was a problem hiding this comment.
I believe it works with ESModule, and we don't need this else if (emitWasm) branch. What didn't work here?
There was a problem hiding this comment.
The branch is necessary, though the original comment was unclear about the reason. Here's what breaks without it:
The else path (NodeJSEnv) constructs nodeArgs as: nodeFlags ++ (if args.isEmpty then Nil else "-" :: args.toList) The "-" is the stdin-pipe signal for Input.Script mode. When NodeJSEnv runs an Input.ESModule with non-empty args, the final Node command becomes: node --experimental-wasm-exnref - foo bar baz /tmp/scalajs-runner.cjs
Node sees "-" and reads from stdin, ignoring /tmp/scalajs-runner.cjs. Wasm always uses ESModule, so the program never runs when user args are present
The direct ProcessBuilder path produces the correct command: node --experimental-wasm-exnref /tmp/main.mjs foo bar baz
Updated the comment to accurately reflect this. This branch is covered by the "Wasm passes arguments to program" integration test
There was a problem hiding this comment.
When NodeJSEnv runs an Input.ESModule with non-empty args, the final Node command becomes: node --experimental-wasm-exnref - foo bar baz /tmp/scalajs-runner.cjs
I don't think this is true, as commented below, NodeJSEnv "runs apps by piping JS to node" (write Input content to tmp file and piped JS dynamically imports that). So there shouldn't be something like /tmp/scalajs-runner.cjs on the final command line. (If that's true, scala-cli's ESModule Input should be broken).
This branch is covered by the "Wasm passes arguments to program" integration test
I see "Wasm passes arguments to program", and it passes without this branch.
|
|
||
| // Detects the major version of Node.js on PATH; cached for the JVM lifetime (lazy val). | ||
| // Returns None if node is not found or version cannot be parsed. | ||
| private lazy val nodeMajorVersion: Option[Int] = |
There was a problem hiding this comment.
I'm not sure about scala-cli policy, but I feel like detecting node version if it supports wasm or not is too much.
my 2 cents: scala-cli should loosely couple with the runtime environment, just try to run and let them fail if it's too old.
There was a problem hiding this comment.
Removed it, so scala cli lets runtime fail if it is too old
There was a problem hiding this comment.
I don't see it's removed, did you forget to push some commits?
| nodeMajorVersion.foreach { v => | ||
| if (v < 22) value(Left(new NodeVersionTooOldForWasmError(v))) | ||
| } | ||
| val nodeFlags = if (emitWasm && nodeNeedsWasmFlag) List("--experimental-wasm-exnref") else Nil |
There was a problem hiding this comment.
I don't think tools like scala-cli to hardcode Node options, and instead, let users explicitly specify Node options by themselves something like like: --node-args=--experimental-wasm-exnref,--experimental-wasm-imported-strings ?
in Node 26, options like --experimental-wasm-imported-strings is removed, and --experimental-wasm-exnref is now enabled by default (and may eventually be removed as well). If scala-cli hardcode options, we'll be in trouble when underlying runtime (node) removes options.
There was a problem hiding this comment.
Fixed. nodeNeedsWasmFlag is now version-aware: private def nodeNeedsWasmFlag: Boolean = nodeMajorVersion.forall(_ < 25)
There was a problem hiding this comment.
Ah sorry, --experimental-wasm-exnref being unnecessary in Node 26+ was just an example. The point was that options passed to Node shouldn't be hardcoded on the scala-cli side. (current implementation is fine since it doesn't add invalid options to node though)
Whether to detect the Node version and automatically add options should follow scala-cli's implementation policy. :) FYI @Gedochao
I was thinking about passing Node options from the scala-cli side like --node-args=--experimental-wasm-exnref,--experimental-wasm-imported-strings, but there's NODE_OPTIONS environment variable. Nevermind! 😄
| * @param runtime | ||
| * The WASM runtime to use for execution (node, deno) | ||
| */ | ||
| final case class WasmOptions( |
There was a problem hiding this comment.
Making wasm a standalone option sounds like it's an another platform like JS/JVM/Native/Wasm, but in reality everything is passed as scala-js options.
It makes more sense to register it as one of the scala-js options, like --js-wasm under ScalaJSOptions. WDYT?
There was a problem hiding this comment.
Good point, moved wasm settings into ScalaJsOptions at both build and CLI layers.
Directive syntax unchanged — //> using wasm and //> using wasmRuntime work as before, imho it is the most user-friendly approach
|
Just left some drive by comments. (Also, I think adding |
Implements scala-cli issue VirtusLab#3316: integrate WebAssembly with Scala CLI. - `--wasm` CLI flag and `//> using wasm` directive to enable WASM output - `--wasm-runtime <runtime>` option and `//> using wasmRuntime` directive Supported values: node (default), deno - `--deno-version`, `--wasmtime-version`, `--wasmer-version` options and corresponding directives for pinning runtime versions - **Node.js** (default): runs Scala.js WASM output with `--experimental-wasm-exnref` flag, requires Node.js >= 22 - **Deno**: runs Scala.js WASM output
…imes - Move --wasm flag to dedicated Wasm help group with --help-wasm option - Simplify wasmOptions parsing with fold/toRight pattern - Add runtime validation with UnrecognizedWasmRuntimeError in directives - Auto-enable WASM when wasmRuntime directive is set - Update reference documentation Code style: simplify denoNeedsWasmFlag, explicit runtime match cases, clean type annotation, scalfmt
…smRuntime bun)
- Add BunNotFoundError with install hint
- Add integration test for Bun (conditional on bun being on PATH)
- Add actions/setup-node@v6 node-version:24 to all Linux integration test
jobs: the default Node.js on ubuntu-24.04 runners is too old for Scala.js
WASM GC (which requires Node.js >= 22). Matches docs-tests job which
already pins node-version: 24
Node 24 still ships V8 12.x where wasm-exnref is gated behind --experimental-wasm-exnref; the flag only flips to default in V8 13.x (Node 25+). The previous nodeMajorVersion < 24 guard therefore left Node 24 (the version pinned in CI) without the flag, which made any Scala.js WASM code using exception bytecodes, runtime throws, JS interop or Scala 3 @main fail at runtime. Same reasoning applies to Deno (Deno 2.x = V8 12.x). Until V8 13.x is the default everywhere, just always set the flag, there is no any overhead
…efactor into ScalaJsOptions - Replace "WASM" with "Wasm" per WebAssembly spec: contraction, not acronym - Fix nodeNeedsWasmFlag to be version-aware: only pass --experimental-wasm-exnref for Node < 25 (V8 12.x); Node 25+ has it enabled by default, Node 26+ may remove it - Remove Node/Bun pre-flight version checks; let runtime fail naturally on old versions - Update else-if-emitWasm comment to more accurately explain why ProcessBuilder is used - Refactor WasmOptions into ScalaJsOptions: jsEmitWasm and wasmRuntime are now fields of ScalaJsOptions at both build and CLI layers; CLI flags are now --js-emit-wasm and --js-wasm-runtime under the Wasm help group; WasmOptions classes removed - linkerConfig() now forces ESModule when jsEmitWasm=true - Update all integration test CLI flags to --js-emit-wasm / --js-wasm-runtime - Regenerate reference docs
2d8329a to
b37dae0
Compare
About the Deno/Bun scope: they share the exact same V8-based Wasm execution path as Node. Because of this, the implementation required minimal additions. Since supporting multiple runtimes was in the initial requirements, I included them here @Gedochao wdyt? |
|
@lostflydev at a glance, I think it's fine to leave them in this PR. |
Implements scala-cli issue #3316: integrate WebAssembly with Scala CLI.
--wasmCLI flag and//> using wasmdirective to enable WASM output--wasm-runtime <runtime>option and//> using wasmRuntimedirective Supported values: node (default), deno, wasmtime, wasmedge, wasmer--deno-version,--wasmtime-version,--wasmer-versionoptions and corresponding directives for pinning runtime versionsNode.js (default): runs Scala.js WASM output with
--experimental-wasm-exnrefflag, requires Node.js >= 22Deno: runs Scala.js WASM output; if not found on PATH, downloads from GitHub releases via Coursier cache
Wasmtime / WasmEdge / Wasmer: return UnsupportedWasmRuntimeError pending upstream Scala.js standalone WASM support (Make Scala.js Wasm backend suitable for standalone Wasm VMs (a.k.a. support "server-side Wasm") scala-js/scala-js#4991)