Open container ports to the LAN with firewall drop-ins

AGENTS.md

@@ -27,6 +27,7 @@ Sibyl is a Fedora bootc host image plus the container images that it runs. Read
 - Fix linter findings. Do not add exclusions. Ask the maintainer if a finding cannot be fixed.
 - `bootc container lint` must pass with no warnings.
 - Every container drops all capabilities and sets `NoNewPrivileges=true`. Only Arcane runs with `SecurityLabelDisable=true`, because its Podman socket and D-Bus mounts need it.
+- The image ships no rule files in `os/rootfs/etc/nftables/sibyl.d/`. That directory is for the administrator's LAN ports, and a file that the image also ships stops bootc from updating it.
 - State under `/var` needs a `tmpfiles.d` entry, because bootc does not update `/var` after install.
 - Users and groups come from `sysusers.d`. The `render` and `video` groups are copied into `/etc/group` at build time, so that sysusers can add members.
 

docs/design/overview.md

@@ -548,7 +548,7 @@ A bundle is a directory:
 
 - SELinux stays enforcing. Only Arcane runs with `SecurityLabelDisable=true`.
 - Secure Boot uses the Fedora shim and kernel. The NVIDIA variant adds the project MOK key.
-- nftables default-deny. Only 443, 8443, 9443 and rate-limited 22 are open on the LAN. When the owner joins a tailnet, every port is open on `tailscale0`, and the tailnet's access rules decide who gets in. UDP 41641 is open for direct Tailscale connections.
+- nftables default-deny. Only 443, 8443, 9443 and rate-limited 22 are open on the LAN. When the owner joins a tailnet, every port is open on `tailscale0`, and the tailnet's access rules decide who gets in. UDP 41641 is open for direct Tailscale connections. An administrator opens more LAN ports with rule files in `/etc/nftables/sibyl.d/`, which `sibyl.nft` includes in its input chain. The image ships no files there, so a local file does not block updates to `sibyl.nft`. At boot, `/usr/libexec/sibyl/nft-load` loads the base rules alone if a rule file fails. The reload loads only the `sibyl` table, not a flush of the whole ruleset, so the rules of other programs such as tailscaled stay.
 - Tailscale comes from its own Fedora repository, `os/repos/tailscale.repo`, because Fedora's package lags the security releases. The repository key is in `os/repos/tailscale.gpg`, so the build does not trust a downloaded key.
 - A Tailscale auth key goes from the System API to `sibyl-hostd` through `/run/sibyl-secret`, in memory. The request in the disk queue names only the key file. `sibyl-hostd` reads it without following a symlink, removes it, and gives `tailscale` the key in a file in its private `/tmp`, not on the command line.
 - SSH accepts keys only. Root login is off.

docs/guides/host.md

@@ -127,6 +127,26 @@ The host accepts images from these public registries:
 
 An administrator can allow another registry in `/etc/containers/policy.json`.
 
+### Open a container's port to the LAN
+
+The host firewall drops every port on the LAN that it does not list. A port that a container publishes is open on the tailnet, but not on the LAN, until you open it. Each port that you open needs a file in `/etc/nftables/sibyl.d/`.
+
+1. In the container's Compose file, publish the port on a host port of 1024 or higher, such as `"9000:9000"`. Rootless Podman cannot publish a lower port.
+2. Connect to the host over SSH.
+3. Create a file for the port, such as `/etc/nftables/sibyl.d/whisper.nft`, with one rule:
+
+   ```
+   tcp dport 9000 accept
+   ```
+
+   To open the port to one subnet only, write `ip saddr 192.168.1.0/24 tcp dport 9000 accept`.
+4. Make sure that the rules are correct: `sudo nft -c -f /etc/sysconfig/nftables.conf`.
+5. Load the rules: `sudo systemctl reload nftables`.
+
+To close the port, delete its file and reload the rules again.
+
+If a file has an error, the reload fails and the old rules stay. If a file has an error at boot, the host loads its own rules without the files in `sibyl.d`, and the journal of `nftables.service` names the file. Host updates do not change the files in `sibyl.d`.
+
 ## Related
 
 - [System API](../reference/system-api.md): the `/host` and `/pools` routes.

docs/guides/troubleshooting.md

@@ -55,7 +55,7 @@ The web console shows "Wrong user or password. The user must be in the wheel gro
 
 1. Make sure that you use `https://`.
 2. Make sure that you use the right port. The table below lists the ports.
-3. The host firewall allows only these ports and SSH. Make sure that no firewall between your computer and the host blocks them.
+3. The host firewall allows only these ports, SSH, and the ports in `/etc/nftables/sibyl.d/`. Make sure that no firewall between your computer and the host blocks them. [Host](host.md#open-a-containers-port-to-the-lan) shows how to open a container's port.
 
 | Service | Port |
 |---|---|

os/rootfs/etc/nftables/sibyl.d/README

@@ -0,0 +1,19 @@
+Ports that the administrator opens to the LAN.
+
+nftables reads every *.nft file in this directory into the input chain of
+the sibyl table (/etc/nftables/sibyl.nft). A file holds rules only, with no
+table or chain around them. For example, /etc/nftables/sibyl.d/whisper.nft:
+
+    # Whisper, published by Arcane on host port 9000
+    tcp dport 9000 accept
+
+To limit a port to the LAN subnet:
+
+    ip saddr 192.168.1.0/24 tcp dport 9000 accept
+
+Check the rules, then load them:
+
+    sudo nft -c -f /etc/sysconfig/nftables.conf
+    sudo systemctl reload nftables
+
+Rootless Podman publishes host ports of 1024 and higher only.

os/rootfs/etc/nftables/sibyl.nft

@@ -5,6 +5,7 @@
 # The high ports accept only connections that arrived through a redirect.
 # The tailnet reaches every port through tailscale0: the tailnet's access
 # rules decide who gets in.
+# Extra LAN ports go in /etc/nftables/sibyl.d/*.nft, not in this file.
 
 destroy table inet sibyl
 
@@ -46,6 +47,12 @@ table inet sibyl {
         # mDNS, so sibyl.local resolves on the LAN
         udp dport mdns accept
 
+        # Ports that the administrator opens to the LAN. Each file in
+        # sibyl.d holds rules for this chain, such as
+        # "tcp dport 8000 accept". The image ships no rule files, so a
+        # local file there does not stop updates to this file.
+        include "/etc/nftables/sibyl.d/*.nft"
+
         # Tailscale: direct connections from other tailnet devices, and
         # every port from the tailnet, such as the containers that Arcane
         # runs. tailscaled refuses packets on tailscale0 that do not come

os/rootfs/usr/lib/systemd/system/nftables.service.d/sibyl.conf

@@ -0,0 +1,9 @@
+# Sibyl loads only its own table (sibyl.nft starts with "destroy table").
+# The stock reload flushes the whole ruleset, which also drops the rules of
+# other programs, such as tailscaled. A reload with a bad file in
+# /etc/nftables/sibyl.d fails as one transaction and keeps the old rules.
+[Service]
+ExecStart=
+ExecStart=/usr/libexec/sibyl/nft-load
+ExecReload=
+ExecReload=/usr/sbin/nft -f /etc/sysconfig/nftables.conf

os/rootfs/usr/libexec/sibyl/nft-load

@@ -0,0 +1,14 @@
+#!/bin/bash
+# Load the host firewall at boot. If the rules fail to load, a file in
+# /etc/nftables/sibyl.d is the likely cause, so load the base rules without
+# those files. Without this, one bad file leaves the host with no firewall.
+set -uo pipefail
+
+CONF=${SIBYL_NFT_CONF:-/etc/sysconfig/nftables.conf}
+BASE=${SIBYL_NFT_BASE:-/etc/nftables/sibyl.nft}
+
+if nft -f "$CONF"; then
+    exit 0
+fi
+echo "<3>sibyl: the firewall rules failed to load; loading $BASE without the files in /etc/nftables/sibyl.d" >&2
+sed '\|^[[:space:]]*include "/etc/nftables/sibyl\.d/|d' "$BASE" | nft -f -