From 3c76867d996b7bd546c35c17be4751efcdea60d0 Mon Sep 17 00:00:00 2001 From: Ben Gittins <5601410+ihavespoons@users.noreply.github.com> Date: Sun, 8 Nov 2026 15:45:10 +1000 Subject: [PATCH] Open container ports to the LAN with firewall drop-ins sibyl.nft includes /etc/nftables/sibyl.d/*.nft in its input chain. An administrator opens a published container port with one rule file there, and the image ships none, so a local file does not stop bootc from updating sibyl.nft. At boot, nft-load falls back to the base rules if a drop-in fails, so one bad file cannot leave the host without a firewall. The reload now loads only the sibyl table instead of flushing the whole ruleset, which kept other programs' rules, such as tailscaled's, from being dropped. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 1 + docs/design/overview.md | 2 +- docs/guides/host.md | 20 +++++++++++++++++++ docs/guides/troubleshooting.md | 2 +- os/rootfs/etc/nftables/sibyl.d/README | 19 ++++++++++++++++++ os/rootfs/etc/nftables/sibyl.nft | 7 +++++++ .../system/nftables.service.d/sibyl.conf | 9 +++++++++ os/rootfs/usr/libexec/sibyl/nft-load | 14 +++++++++++++ 8 files changed, 72 insertions(+), 2 deletions(-) create mode 100644 os/rootfs/etc/nftables/sibyl.d/README create mode 100644 os/rootfs/usr/lib/systemd/system/nftables.service.d/sibyl.conf create mode 100755 os/rootfs/usr/libexec/sibyl/nft-load diff --git a/AGENTS.md b/AGENTS.md index 44d6329..3af6c70 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,6 +27,7 @@ Sibyl is a Fedora bootc host image plus the container images that it runs. Read - Fix linter findings. Do not add exclusions. Ask the maintainer if a finding cannot be fixed. - `bootc container lint` must pass with no warnings. - Every container drops all capabilities and sets `NoNewPrivileges=true`. Only Arcane runs with `SecurityLabelDisable=true`, because its Podman socket and D-Bus mounts need it. +- The image ships no rule files in `os/rootfs/etc/nftables/sibyl.d/`. That directory is for the administrator's LAN ports, and a file that the image also ships stops bootc from updating it. - State under `/var` needs a `tmpfiles.d` entry, because bootc does not update `/var` after install. - Users and groups come from `sysusers.d`. The `render` and `video` groups are copied into `/etc/group` at build time, so that sysusers can add members. diff --git a/docs/design/overview.md b/docs/design/overview.md index 78050ac..67cd2c4 100644 --- a/docs/design/overview.md +++ b/docs/design/overview.md @@ -548,7 +548,7 @@ A bundle is a directory: - SELinux stays enforcing. Only Arcane runs with `SecurityLabelDisable=true`. - Secure Boot uses the Fedora shim and kernel. The NVIDIA variant adds the project MOK key. -- nftables default-deny. Only 443, 8443, 9443 and rate-limited 22 are open on the LAN. When the owner joins a tailnet, every port is open on `tailscale0`, and the tailnet's access rules decide who gets in. UDP 41641 is open for direct Tailscale connections. +- nftables default-deny. Only 443, 8443, 9443 and rate-limited 22 are open on the LAN. When the owner joins a tailnet, every port is open on `tailscale0`, and the tailnet's access rules decide who gets in. UDP 41641 is open for direct Tailscale connections. An administrator opens more LAN ports with rule files in `/etc/nftables/sibyl.d/`, which `sibyl.nft` includes in its input chain. The image ships no files there, so a local file does not block updates to `sibyl.nft`. At boot, `/usr/libexec/sibyl/nft-load` loads the base rules alone if a rule file fails. The reload loads only the `sibyl` table, not a flush of the whole ruleset, so the rules of other programs such as tailscaled stay. - Tailscale comes from its own Fedora repository, `os/repos/tailscale.repo`, because Fedora's package lags the security releases. The repository key is in `os/repos/tailscale.gpg`, so the build does not trust a downloaded key. - A Tailscale auth key goes from the System API to `sibyl-hostd` through `/run/sibyl-secret`, in memory. The request in the disk queue names only the key file. `sibyl-hostd` reads it without following a symlink, removes it, and gives `tailscale` the key in a file in its private `/tmp`, not on the command line. - SSH accepts keys only. Root login is off. diff --git a/docs/guides/host.md b/docs/guides/host.md index e87e6f1..9bda951 100644 --- a/docs/guides/host.md +++ b/docs/guides/host.md @@ -127,6 +127,26 @@ The host accepts images from these public registries: An administrator can allow another registry in `/etc/containers/policy.json`. +### Open a container's port to the LAN + +The host firewall drops every port on the LAN that it does not list. A port that a container publishes is open on the tailnet, but not on the LAN, until you open it. Each port that you open needs a file in `/etc/nftables/sibyl.d/`. + +1. In the container's Compose file, publish the port on a host port of 1024 or higher, such as `"9000:9000"`. Rootless Podman cannot publish a lower port. +2. Connect to the host over SSH. +3. Create a file for the port, such as `/etc/nftables/sibyl.d/whisper.nft`, with one rule: + + ``` + tcp dport 9000 accept + ``` + + To open the port to one subnet only, write `ip saddr 192.168.1.0/24 tcp dport 9000 accept`. +4. Make sure that the rules are correct: `sudo nft -c -f /etc/sysconfig/nftables.conf`. +5. Load the rules: `sudo systemctl reload nftables`. + +To close the port, delete its file and reload the rules again. + +If a file has an error, the reload fails and the old rules stay. If a file has an error at boot, the host loads its own rules without the files in `sibyl.d`, and the journal of `nftables.service` names the file. Host updates do not change the files in `sibyl.d`. + ## Related - [System API](../reference/system-api.md): the `/host` and `/pools` routes. diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index adada6b..1d658ee 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -55,7 +55,7 @@ The web console shows "Wrong user or password. The user must be in the wheel gro 1. Make sure that you use `https://`. 2. Make sure that you use the right port. The table below lists the ports. -3. The host firewall allows only these ports and SSH. Make sure that no firewall between your computer and the host blocks them. +3. The host firewall allows only these ports, SSH, and the ports in `/etc/nftables/sibyl.d/`. Make sure that no firewall between your computer and the host blocks them. [Host](host.md#open-a-containers-port-to-the-lan) shows how to open a container's port. | Service | Port | |---|---| diff --git a/os/rootfs/etc/nftables/sibyl.d/README b/os/rootfs/etc/nftables/sibyl.d/README new file mode 100644 index 0000000..57e2ef9 --- /dev/null +++ b/os/rootfs/etc/nftables/sibyl.d/README @@ -0,0 +1,19 @@ +Ports that the administrator opens to the LAN. + +nftables reads every *.nft file in this directory into the input chain of +the sibyl table (/etc/nftables/sibyl.nft). A file holds rules only, with no +table or chain around them. For example, /etc/nftables/sibyl.d/whisper.nft: + + # Whisper, published by Arcane on host port 9000 + tcp dport 9000 accept + +To limit a port to the LAN subnet: + + ip saddr 192.168.1.0/24 tcp dport 9000 accept + +Check the rules, then load them: + + sudo nft -c -f /etc/sysconfig/nftables.conf + sudo systemctl reload nftables + +Rootless Podman publishes host ports of 1024 and higher only. diff --git a/os/rootfs/etc/nftables/sibyl.nft b/os/rootfs/etc/nftables/sibyl.nft index 1ad619c..5269413 100644 --- a/os/rootfs/etc/nftables/sibyl.nft +++ b/os/rootfs/etc/nftables/sibyl.nft @@ -5,6 +5,7 @@ # The high ports accept only connections that arrived through a redirect. # The tailnet reaches every port through tailscale0: the tailnet's access # rules decide who gets in. +# Extra LAN ports go in /etc/nftables/sibyl.d/*.nft, not in this file. destroy table inet sibyl @@ -46,6 +47,12 @@ table inet sibyl { # mDNS, so sibyl.local resolves on the LAN udp dport mdns accept + # Ports that the administrator opens to the LAN. Each file in + # sibyl.d holds rules for this chain, such as + # "tcp dport 8000 accept". The image ships no rule files, so a + # local file there does not stop updates to this file. + include "/etc/nftables/sibyl.d/*.nft" + # Tailscale: direct connections from other tailnet devices, and # every port from the tailnet, such as the containers that Arcane # runs. tailscaled refuses packets on tailscale0 that do not come diff --git a/os/rootfs/usr/lib/systemd/system/nftables.service.d/sibyl.conf b/os/rootfs/usr/lib/systemd/system/nftables.service.d/sibyl.conf new file mode 100644 index 0000000..c9941c7 --- /dev/null +++ b/os/rootfs/usr/lib/systemd/system/nftables.service.d/sibyl.conf @@ -0,0 +1,9 @@ +# Sibyl loads only its own table (sibyl.nft starts with "destroy table"). +# The stock reload flushes the whole ruleset, which also drops the rules of +# other programs, such as tailscaled. A reload with a bad file in +# /etc/nftables/sibyl.d fails as one transaction and keeps the old rules. +[Service] +ExecStart= +ExecStart=/usr/libexec/sibyl/nft-load +ExecReload= +ExecReload=/usr/sbin/nft -f /etc/sysconfig/nftables.conf diff --git a/os/rootfs/usr/libexec/sibyl/nft-load b/os/rootfs/usr/libexec/sibyl/nft-load new file mode 100755 index 0000000..ec31cf2 --- /dev/null +++ b/os/rootfs/usr/libexec/sibyl/nft-load @@ -0,0 +1,14 @@ +#!/bin/bash +# Load the host firewall at boot. If the rules fail to load, a file in +# /etc/nftables/sibyl.d is the likely cause, so load the base rules without +# those files. Without this, one bad file leaves the host with no firewall. +set -uo pipefail + +CONF=${SIBYL_NFT_CONF:-/etc/sysconfig/nftables.conf} +BASE=${SIBYL_NFT_BASE:-/etc/nftables/sibyl.nft} + +if nft -f "$CONF"; then + exit 0 +fi +echo "<3>sibyl: the firewall rules failed to load; loading $BASE without the files in /etc/nftables/sibyl.d" >&2 +sed '\|^[[:space:]]*include "/etc/nftables/sibyl\.d/|d' "$BASE" | nft -f -