Skip to content
hyper-lightPublic

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

kiln

Self-hosted GitOps CI/CD for the hyper-light stack. One static binary contains the server, the agent, a process supervisor, the installer, the CLI and the web UI. It runs the same way on bare metal and on Kubernetes.

  • GitOps: kiln polls GitHub with git ls-remote (no token, no rate limit) and builds what .kiln/*.yaml at each commit declares: branches, tags, PR heads, cron schedules, manual runs. Pipelines are multi-step DAGs with matrices, conditions, retries, timeouts, artifacts and concurrency groups. See docs/pipelines.md.
  • Runs anywhere: runs-on: [k8s] gets a fresh pod per job, with build caches on a PVC. runs-on: [baremetal, bench] runs on real hardware; exclusive: true keeps benchmarks quiet.
  • hyper-raft transport: agents talk to the server over hyper-transport QUIC with mutual TLS from kiln's own CA. Agents enroll by CSR, so private keys never travel. Control, terminal and log traffic use separate prioritised lanes.
  • Full machine control: real PTY terminals in the browser on the server and on every agent. Resize, alt-screen and TUIs (k9s, htop, vim) all work.
  • Observability: Prometheus /metrics (host stats read straight from /proc on Linux, from the OS on macOS and Windows, sampled every 5 s), Grafana dashboards, alert rules, live UI charts. No sidecars, nothing paid.
  • Never down for long: no unwrap/expect/panic/indexing (enforced by clippy). Recovery happens at several levels:
    • subsystems restart in-process, Erlang style;
    • kiln supervise restarts the process with backoff, reaps zombies (works as PID 1) and hot-swaps a new binary;
    • jobs are requeued when an agent or the server dies;
    • orphaned build processes are reaped on restart.

Quick start

cargo build --release              # after building the UI: npm --prefix ui ci && npm --prefix ui run build
./target/release/kiln setup        # a guide: kiln on this machine, its own signature on a Mac, HTTPS for your devices and tailnet, a password

kiln setup looks at the machine, asks a few questions, shows everything it will do, and does it once you say so:

  • Where kiln runs: this machine by default, as a service started at boot, whatever kubeconfig is here; or Docker on it, Kubernetes, or a kiln server it finds on the network, to join.
  • Upgrades: on a Mac, kiln's own code signature (kiln signing setup). macOS lets programs onto the local network by their signature, and kiln's builds are signed ad hoc: without one, every upgrade is cut off from the local network, and from the kiln server, until you allow it again.
  • The front door: DNS and HTTPS for kiln under a name of your own (kiln.internal unless you choose), with a certificate from kiln's own root, on your tailnet too where Tailscale runs (docs/front-door.md, docs/tailscale.md). Phones need HTTPS for voice input. On Linux it shows the one-time steps that takes root for, and runs them with sudo once you say so.
  • Signing in: a password, for any device.

It ends with a sign-in link, and what's left for you (trusting kiln's root on your devices, your router's DNS). Run it again any time: what's done stays done. --dry-run says what it would do; --unattended asks nothing (--on, --server, --domain and --sudo say what it would have asked).

Without the guide, kiln install installs on the target you name:

./target/release/kiln install      # the cluster of your kubeconfig's current context (the default; without one, this machine)
./target/release/kiln install --on docker    # a container on this machine
./target/release/kiln install --on machine   # this machine: ~/.local/bin, ~/.config/kiln/kiln.toml, start on boot, start now
kiln login                         # opens the UI with a one-time link; no password to type

Each installs a whole kiln (server, UI, stores and a runner of its own) and ends only once it answers, its UI loads and its runner is connected, saying where to open it; a failure says what failed and takes back what the install made. In Kubernetes it goes in namespace kiln (--namespace, --context), reached at a node's address and a NodePort (or, when no node can be reached, through kubectl port-forward); Docker's is on port 7070 (--port), QUIC on the next. Both run an image built for the cluster's (or Docker's) CPU: from the binary that installs them when it's a Linux kiln for it, else (a Mac's, Windows', or another CPU) from the static Linux build of the kiln server this machine's runner is enrolled with (--server names another; --image runs an image of your own). minikube, kind and k3d clusters are loaded with it, and Docker Desktop's cluster (one node or more) and OrbStack's share it; any other cluster pulls it from a registry: --registry, else the one the cluster names (kube-public's local-registry-hosting).

kiln install --unattended never prompts. kiln install --system installs system-wide on this machine (asks for sudo; add --unattended to use sudo -n only). The server runs a local agent by default, so runs-on: [baremetal] jobs work immediately.

On Linux with systemd the server comes first on its machine (docs/priority.md, "What kiln install grants"): its kiln.slice at CPU and I/O weight 1000, and its shards in Linux's real-time class wherever its unit can grant it, written into a new server's config ([server] realtime): every system install, and a user's whose service manager allows RLIMIT_RTPRIO 1. The shards take it each on a CPU of its own, the server's other threads kept off them, or not at all, and the server says which. [server] cores gives it cores of its own, which a system install also makes a cpuset partition nothing else runs on. A user's install can't get, without root: a slice ahead of the system's, the real-time class past their service manager's limit, AllowedCPUs= without the cpuset controller delegated to it, or a partition; kiln install says, line by line, what it got and what it would take. On macOS and Windows the server keeps its own ways (an Interactive launchd job; above normal priority), its legs and agents below it.

To add another machine with a join token, such as a benchmark box, copy the command from Agents → Add agent in the UI:

curl -fsSL http://<server>:7070/install.sh | sh -s -- --token <join-token> --labels baremetal,bench

At a terminal the script hands over to kiln setup, which says what it will do first (and on a Mac gives kiln its own signature); without one it asks nothing.

Platforms

kiln builds and runs on Linux and Windows, on x86_64 and arm64, and on Apple silicon Macs (not Intel ones). Every tagged release (.github/workflows/release.yml) publishes kiln-<version>-<target>.tar.gz / .zip, the bare binaries and SHA256SUMS for x86_64/aarch64-unknown-linux-musl (fully static), aarch64-apple-darwin and x86_64/aarch64-pc-windows-msvc.

Linux macOS Windows
kiln install --on machine starts on boot with systemd (else cron @reboot, else XDG autostart) launchd (LaunchAgent; LaunchDaemon as root) a Scheduled Task at boot that runs as you, whether or not anyone signs in (administrator rights, asked for once; without them, at sign-in)
binary ~/.local/bin (/usr/local/bin as root), put on your PATH (a line in your shell's startup file, when it isn't) same %LOCALAPPDATA%\Programs\kiln, put on your PATH
config / data ~/.config/kiln, ~/.local/share/kiln (/etc/kiln, /var/lib/kiln as root) same %APPDATA%\kiln, %LOCALAPPDATA%\kiln
job steps (no shell:) bash -eo pipefail, else sh -e same PowerShell 7 (pwsh), else Windows PowerShell
job process trees process groups process groups job objects (they die with the agent)

A server hands out agents for every platform: GET /dl/kiln/<target> serves its own binary when it was built for that target and otherwise redirects to the matching release asset of the same version. On Windows, add an agent from PowerShell:

& ([scriptblock]::Create((irm http://<server>:7070/install.ps1))) -Token <join-token> -Labels windows

Windows asks once for administrator rights, to register the task that starts kiln at boot (an administrator PowerShell has them already). That task runs kiln as you through Windows' S4U logon, so no password is kept; as in an SSH session with a key, kiln there has none of your network credentials or Credential Manager secrets, and no desktop to show windows on. Say no (or install with --boot logon) and kiln starts when you sign in, in your session, instead.

Give Windows and macOS agents their own labels (windows, macos) so runs-on: [baremetal] jobs written for Linux do not land on them.

Add another computer

On a home network kiln servers announce themselves (multicast DNS, _kiln._tcp.local.), so the other computers need no token and no address. With the kiln binary on the new computer:

kiln join          # finds the server, asks to join, prints a verification code, waits

The request shows up in the UI under Runners → Add machines. Approve it there if the UI shows the same verification code as the terminal. The computer then installs its agent, which starts on boot, and appears in the fleet with its terminal, metrics and jobs. kiln join --server http://<server>:7070 skips the search. --labels, --name and --slots work as for kiln install --agent. A computer without kiln can fetch it from the server and join in one step: curl -fsSL http://<server>:7070/install.sh | sh -s -- --join. Agents → On this network lists the kiln machines the server hears that have not joined yet, with the command to run on each.

Firewalls: discovery needs UDP 5353 open to the LAN on both machines, for example sudo ufw allow from 192.168.1.0/24 to any port 5353 proto udp. Joining needs only the server's HTTP port (TCP 7070) reachable from the new computer: its QUIC port (UDP 7071) is used when it is reachable, and agents tunnel through the HTTP port when it is not, through a company proxy and TLS inspection too (networks that block UDP). Discovery can be turned off with [discovery] enabled = false; see docs/architecture.md.

Kubernetes (multi-node, ArgoCD, Prometheus, Grafana)

deploy/cluster/bootstrap.sh --unattended   # 3-node minikube + ArgoCD + kube-prometheus-stack + kiln
deploy/cluster/port-forward.sh             # kiln :7070 · ArgoCD :8080 · Grafana :3000 · Prometheus :9090

See deploy/README.md. Bare-metal agents join the in-cluster server through the kiln-external service (UDP 7071 + TCP 7070).

Layout

Path What
crates/kiln The binary: server, agent, supervise, install, join, login, validate, …
ui/ React + Tailwind dashboard in the hyperlight-site design language, embedded at build time
deploy/ Dockerfile support, Helm chart, ArgoCD apps, Grafana dashboards, alert rules, cluster scripts
docs/ architecture, API, pipelines, CI spec (GitHub Actions compatible CI, in progress)
.kiln/ci.yaml kiln's own pipeline

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages