Prepare a standalone Rust application SDK and build helper

.github/workflows/rust-sdk-check.yml

@@ -0,0 +1,35 @@
+name: Check standalone Rust application SDK
+on:
+  push:
+    branches: [staging, main, master]
+    paths: ['rust-sdk/**', 'actions/setup-rust-sdk/**', '.github/workflows/rust-sdk-check.yml']
+  pull_request:
+    paths: ['rust-sdk/**', 'actions/setup-rust-sdk/**', '.github/workflows/rust-sdk-check.yml']
+permissions:
+  contents: read
+jobs:
+  check:
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
+        with:
+          persist-credentials: false
+      - run: python3 -m unittest discover -s rust-sdk/tests -v
+  signer:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 30
+    env:
+      CARGO_BUILD_JOBS: '4'
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
+        with:
+          persist-credentials: false
+      - id: sdk
+        uses: ./actions/setup-rust-sdk
+        with:
+          output: ${{ runner.temp }}/mochios-rust-sdk
+      - name: Test and build the trusted signer without application sources or credentials
+        env:
+          SDK: ${{ steps.sdk.outputs.sdk }}
+          HELPER: ${{ steps.sdk.outputs.helper }}
+        run: python3 "$(dirname "$HELPER")/host-tool.py" --sdk "$SDK" --tool msign --output "$RUNNER_TEMP/host-tools" --test

actions/setup-rust-sdk/action.yml

@@ -0,0 +1,23 @@
+name: Set up mochiOS Rust application SDK
+description: Install a pinned C SDK through komeup and prepare Rust, AppCore and ViewKit without an OS checkout.
+inputs:
+  output:
+    description: Fresh directory for the isolated SDK (prefer runner.temp).
+    required: true
+outputs:
+  sdk:
+    description: Local app-build.json, for this runner only; do not publish it.
+    value: ${{ steps.setup.outputs.sdk }}
+  helper:
+    description: Path to the generic Rust app build/test/staging helper.
+    value: ${{ steps.setup.outputs.helper }}
+runs:
+  using: composite
+  steps:
+    - id: setup
+      shell: bash
+      env:
+        SDK_OUTPUT: ${{ inputs.output }}
+        SDK_ACTION: ${{ github.action_path }}
+        SDK_ACTION_REF: ${{ github.action_ref }}
+      run: python3 "$SDK_ACTION/../../rust-sdk/setup.py" --output "$SDK_OUTPUT"

rust-sdk/README.md

@@ -0,0 +1,84 @@
+# Standalone Rust application builds
+
+`actions/setup-rust-sdk` prepares a native Rust application build environment on
+Ubuntu 24.04. It installs the explicit released C/C++ SDK through komeup, verifies
+the reviewed archive SHA-256, and fetches the fixed Rust std, ViewKit, font and
+protocol sources from `sources.lock.json`. AppCore comes from the same DevKit
+revision as the Action. No OS checkout, image build, SDK release, credential or
+organization membership is required.
+
+This initial source SDK prepares one application at a time. It uses the existing
+mochiOS Rust std fork and pinned nightly compiler. It is not a newly published
+binary Rust SDK. The runner-local `app-build.json` contains local paths and must
+not be uploaded or shipped. Immutable source pins and font hashes are public;
+the installation does not change the developer's default komeup/rustup choices.
+
+Example caller (replace `staging` with a reviewed full commit SHA):
+
+```yaml
+- id: sdk
+  uses: mochiOS/devkit/actions/setup-rust-sdk@staging
+  with:
+    output: ${{ runner.temp }}/mochios-rust-sdk
+- name: Test, compile and stage
+  env:
+    HELPER: ${{ steps.sdk.outputs.helper }}
+    SDK: ${{ steps.sdk.outputs.sdk }}
+  run: python3 "$HELPER" --source "$GITHUB_WORKSPACE" --sdk "$SDK" --output "$RUNNER_TEMP/app-build" --test
+- id: package
+  uses: mochiOS/devkit/actions/build-mpkg@20d3fb71b952ae7a4ff583dafe8cc3d00e24d3a3
+  with:
+    manifest: ${{ runner.temp }}/app-build/stage/manifest.toml
+    payload: ${{ runner.temp }}/app-build/stage/payload
+    output: dist/Example.mpkg
+```
+
+Compilation and native host unit tests run in an isolated staged Cargo project.
+The helper maps the existing AppCore/protocol source dependencies into the SDK,
+preserves the application's lockfile as input, and exports its actual resulting
+resolution as `resolved.Cargo.lock` for review. It never edits or rolls back the
+source checkout's Cargo.lock. SDK changes can require adopting this resolution
+into the application repository before production release.
+
+The staging helper supports a single-entry Rust application with a completed
+MPKG manifest. It preserves document associations and requested capabilities,
+copies declared resources, binds real file hashes/modes, removes source-tree
+built-in provenance, and changes the binary destination to the normal
+`/applications/<name>.app` package path. ELF and host-path checks do not execute
+target programs and do not establish native runtime success. The public MPKG
+Action separately creates the unsigned container. Signing, approval and release
+publication remain caller workflow responsibilities.
+
+## Approval-protected publication
+
+The generic `release.py` helper consumes the unsigned artifact from the same
+successful workflow run. It checks the source commit, package ID, actual MPKG
+manifest, SDK source lock and archive checksum before accessing signing inputs.
+It does not compile or execute application code. `host-tool.py` builds msign from
+the same trusted SDK source revision, with developer-CA networking disabled.
+
+The caller must place its publication job behind a protected GitHub Environment.
+Configure the repository's chosen reviewer/team and restrict the environment to
+its default branch. Pass `--environment`, `--reviewer-team` (numeric Team ID),
+and `--branch` to the release helper. It requires exactly that Team as reviewer,
+admin bypass disabled, and exactly that branch allowed; unreadable or changed
+policy aborts publication. This verifies configuration, while GitHub's protected
+job enforces the approval itself. The public packaging/SDK Actions have no organization or
+team requirement of their own.
+
+Official app callers use an `app-release` Environment with
+`application-maintainers` approval. Releases stay disabled until the repository
+variable `APP_RELEASE_ENABLED` is explicitly set to `true` and these Environment
+values have been configured:
+
+* Secret `MSIGN_CERTIFICATE_B64`: base64 of the issued Developer Certificate file.
+* Secret `MSIGN_PRIVATE_KEY_B64`: base64 of its corresponding private-key file.
+* Variable `MSIGN_ROOT_PUBLIC_KEY_B64`: base64 of the trusted issuer public-key file.
+
+No root private key, fixture key, generated substitute identity or unsigned
+publication is accepted. Credentials are decoded into a private temporary
+directory, not propagated as subprocess environment values, and removed on
+success or failure. The signed package is verified against the trusted root
+before publication. Existing release tags/assets are never replaced. The initial
+release helper publishes prereleases as `v<version>-r<revision>`; update the
+manifest revision before making another release of the same version.

rust-sdk/build.py

@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""Build/test a Rust app and stage an unsigned package for the public MPKG Action."""
+import argparse
+import copy
+import hashlib
+import json
+import os
+from pathlib import Path, PurePosixPath
+import re
+import shutil
+import struct
+import subprocess
+import tomllib
+
+
+def run(args, **kwargs):
+    """Preserve real compiler and test failures with streaming build output."""
+    subprocess.run([str(a) for a in args], check=True, **kwargs)
+
+
+def dumps(data):
+    """Write ordinary Cargo/MPKG tables, preserving nested metadata and array tables."""
+    lines = []
+    def table(mapping, path=(), header=None):
+        if header:
+            lines.append(header)
+        children = []
+        for key, value in mapping.items():
+            if isinstance(value, dict) or (isinstance(value, list) and value and isinstance(value[0], dict)):
+                children.append((key, value))
+            else:
+                lines.append(json.dumps(key) + ' = ' + json.dumps(value, ensure_ascii=True, allow_nan=False))
+        for key, value in children:
+            child = (*path, key)
+            name = '.'.join(json.dumps(p) for p in child)
+            for item in value if isinstance(value, list) else [value]:
+                lines.append('')
+                table(item, child, '[[' + name + ']]' if isinstance(value, list) else '[' + name + ']')
+    table(data)
+    return '\n'.join(lines) + '\n'
+
+
+def resource(source, name):
+    """Find a declared bundle-local resource without following links outside source."""
+    path = PurePosixPath(name)
+    if not name or path.is_absolute() or '..' in path.parts or '\\' in name:
+        raise ValueError('Invalid resource path: ' + name)
+    for candidate in (source / name, source / 'resources' / name):
+        if candidate.is_file() and candidate.resolve().is_relative_to(source.resolve()):
+            return candidate
+    raise ValueError('Missing or external resource: ' + name)
+
+
+def stage(source, binary, output, licenses=None):
+    """Preserve capabilities/associations and bind actual files without claiming built-in trust."""
+    with binary.open('rb') as stream:
+        header = stream.read(64)
+    if len(header) != 64 or header[:6] != b'\x7fELF\x02\x01' or struct.unpack_from('<HH', header, 16) != (2, 62):
+        raise ValueError('Expected a native x86-64 ELF executable')
+    manifest = copy.deepcopy(tomllib.loads((source / 'manifest.toml').read_text()))
+    package, app = manifest['package'], manifest['application']
+    package.pop('provenance', None)
+    name = package['name']
+    if not re.fullmatch('[A-Za-z][A-Za-z0-9_-]*', name) or app['entry'] != 'entry.elf' or len(manifest.get('binary', [])) != 1:
+        raise ValueError('Expected a single-entry native application')
+    if output.exists():
+        raise ValueError('Use a fresh package stage')
+    bundle = output / 'payload/bundle'
+    bundle.mkdir(parents=True)
+    shutil.copyfile(binary, bundle / 'entry.elf')
+    (bundle / 'entry.elf').chmod(0o755)
+    for name in dict.fromkeys([app['icon'], *app.get('resources', [])]):
+        target = bundle / name
+        target.parent.mkdir(parents=True, exist_ok=True)
+        shutil.copyfile(resource(source, name), target)
+        target.chmod(0o644)
+    if licenses and licenses.is_dir():
+        shutil.copytree(licenses, bundle / 'licenses/fonts')
+    manifest['file'] = []
+    for index, path in enumerate(sorted(bundle.rglob('*'))):
+        if path.is_file():
+            relative = path.relative_to(bundle).as_posix()
+            path.chmod(0o755 if relative == 'entry.elf' else 0o644)
+            manifest['file'].append(dict(id='f' + str(index), path='$/' + relative,
+                digest='sha256:' + hashlib.sha256(path.read_bytes()).hexdigest(), size=path.stat().st_size,
+                mode='0755' if relative == 'entry.elf' else '0644'))
+    entry = next(f for f in manifest['file'] if f['path'] == '$/entry.elf')
+    manifest['binary'][0].update(path='/applications/' + package['name'] + '.app/entry.elf', file=entry['id'])
+    (output / 'manifest.toml').write_text(dumps(manifest))
+    return manifest
+
+
+def build(source, output, sdk, tests):
+    """Build independently of the caller's checkout layout; never change its lockfile."""
+    project = output / 'project'
+    project.mkdir(parents=True)
+    manifest = tomllib.loads((source / 'Cargo.toml').read_text())
+    for group in [manifest.get('dependencies', {})] + [v.get('dependencies', {}) for v in manifest.get('target', {}).values()]:
+        for dependency in group.values():
+            if not isinstance(dependency, dict) or 'path' not in dependency:
+                continue
+            path = dependency['path']
+            if path == '../../tools/devkit/crates/appcore':
+                dependency['path'] = sdk['appcore']
+            elif path.startswith('../../user/crates/'):
+                dependency['path'] = str(Path(sdk['user']) / 'crates' / path.removeprefix('../../user/crates/'))
+            elif (source / path).resolve().is_relative_to(source):
+                dependency['path'] = str((source / path).resolve())
+            else:
+                raise ValueError('External path dependency needs an explicit SDK mapping: ' + path)
+    for child in source.iterdir():
+        if child.name not in ('.git', '.github', 'out', 'target', 'Cargo.toml', 'Cargo.lock'):
+            (project / child.name).symlink_to(child.resolve(), target_is_directory=child.is_dir())
+    (project / 'Cargo.toml').write_text(dumps(manifest))
+    shutil.copyfile(source / 'Cargo.lock', project / 'Cargo.lock')
+    options = []
+    for config in sdk['cargo_config']:
+        options += ['--config', config]
+    host_env = dict(os.environ, **sdk['fonts'], CARGO_PROFILE_TEST_DEBUG='0')
+    for key in ('RUSTFLAGS', 'CARGO_ENCODED_RUSTFLAGS', 'RUSTC', 'RUSTDOC'):
+        host_env.pop(key, None)
+    host_env.update(RUSTC=sdk['rustc'], RUSTDOC=sdk['rustdoc'])
+    if tests:
+        run([sdk['cargo'], 'test', '--manifest-path', project / 'Cargo.toml', '--target-dir', output / 'host-target', *options],
+            env=host_env, cwd=project)
+    env = dict(host_env, RUSTC=sdk['rustc'], RUSTDOC=sdk['rustdoc'],
+               CARGO_ENCODED_RUSTFLAGS='\x1f'.join(sdk['rustflags'] + ['--remap-path-prefix=' + str(source) + '=/src/app',
+                 '--remap-path-prefix=' + str(output) + '=/build/app']))
+    run([sdk['cargo'], 'build', '--release', '-Z', 'build-std=std,panic_abort,compiler_builtins', '-Z', 'json-target-spec',
+         '--manifest-path', project / 'Cargo.toml', '--target', sdk['target'], '--target-dir', sdk['target_dir'],
+         '--bin', manifest['package']['name'], *options], env=env, cwd=project)
+    shutil.copyfile(project / 'Cargo.lock', output / 'resolved.Cargo.lock')
+    binary = Path(sdk['target_dir']) / Path(sdk['target']).stem / 'release' / manifest['package']['name']
+    for option in ('-l', '-d', '--version-info'):
+        text = subprocess.check_output(['readelf', option, str(binary)], text=True)
+        if any(value in text for value in ('GLIBC_', 'libc.so.6', '/ld-linux', '(RUNPATH)', '(RPATH)')):
+            raise ValueError('Host linkage or runtime search path in native executable')
+    data = binary.read_bytes()
+    for prefix in (str(source), str(output), str(Path.home())):
+        if prefix.encode() in data:
+            raise ValueError('Build-host path in executable')
+    package = stage(source, binary, output / 'stage', Path(sdk['font_licenses']))
+    revision = subprocess.check_output(['git', '-C', str(source), 'rev-parse', 'HEAD'], text=True).strip()
+    (output / 'build-info.json').write_text(json.dumps(dict(format=1, source_commit=revision,
+        package=package['package'], sources=sdk['sources'], devkit_ref=sdk['devkit_ref'], runtime_tested=False, signed=False), indent=2) + '\n')
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--source', type=Path, required=True)
+    parser.add_argument('--sdk', type=Path, required=True)
+    parser.add_argument('--output', type=Path, required=True)
+    parser.add_argument('--test', action='store_true')
+    args = parser.parse_args()
+    build(args.source.resolve(), args.output.resolve(), json.loads(args.sdk.read_text()), args.test)

rust-sdk/host-tool.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+"""Build a trusted SDK host tool without compiling or executing application code."""
+import argparse
+import json
+import os
+from pathlib import Path
+import subprocess
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--sdk', type=Path, required=True)
+    parser.add_argument('--output', type=Path, required=True)
+    parser.add_argument('--tool', choices=['msign'], required=True)
+    parser.add_argument('--test', action='store_true')
+    args = parser.parse_args()
+    sdk = json.loads(args.sdk.read_text())
+    devkit = Path(sdk['appcore']).parents[1]
+    env = os.environ.copy()
+    for key in ('RUSTFLAGS', 'CARGO_ENCODED_RUSTFLAGS', 'CARGO_BUILD_TARGET'):
+        env.pop(key, None)
+    env.update(RUSTC=sdk['rustc'], RUSTDOC=sdk['rustdoc'])
+    command = [sdk['cargo'], 'build', '--release', '--manifest-path', str(devkit / 'Cargo.toml'),
+               '--package', args.tool, '--no-default-features', '--target-dir', str(args.output.resolve())]
+    for config in sdk['cargo_config']:
+        command += ['--config', config]
+    if args.test:
+        test = command.copy()
+        test[1] = 'test'
+        test.remove('--release')
+        env['CARGO_PROFILE_TEST_DEBUG'] = '0'
+        subprocess.run(test, env=env, cwd=devkit, check=True)
+    subprocess.run(command, env=env, cwd=devkit, check=True)
+
+
+if __name__ == '__main__':
+    main()