The agent host's vault and SilverBullet (M5.4) #166

Merged
dev-agent merged 8 commits from agent-host-vault into main 2026-10-19 22:22:39 +00:00
7 changed files with 70 additions and 52 deletions
Showing only changes of commit bf98445fcb - Show all commits

No files matched your search

+15 -13
View File
4
@@ -7,9 +7,9 @@ later (M6.1).
## What's here
- `silverbullet/compose.yaml`: SilverBullet, its version pinned. It mounts the
vault, with the agents' `memory/` read-only, and its own data folder.
Nothing else of the host.
- `silverbullet/compose.yaml`: SilverBullet, its version pinned, serving the
vault as its one space, with the agents' `memory/` read-only. Nothing else of
the host. It has no accounts: the operator logs in through Authelia.
- `setup.sh`: makes the vault's folders, installs the stack under
`/opt/stacks/silverbullet` and the firewall, and starts it. Safe to run again.
- `nisse-firewall`: only the operator's Caddy may reach SilverBullet's port
@@ -17,8 +17,8 @@ later (M6.1).
- `check.sh`: run on the host by `scripts/agent-host.sh check`; says "all
checks passed" only once every check has run.
- `caddy/nisse-memory.dongar.net.caddy`: the site for the operator's Caddy on
`docker.garrett-hall.com`. HTTPS from Caddy's usual certificate, inside
garrett-hall.com only.
`docker.garrett-hall.com`. HTTPS from Caddy's usual certificate,
behind Authelia.
## The vault
@@ -41,12 +41,14 @@ scripts/agent-host.sh install # copy this folder over and run setup.sh
scripts/agent-host.sh check # running, mounts, firewall
```
Then:
Then **the operator adds the Caddy site** from
`caddy/nisse-memory.dongar.net.caddy`, behind Authelia, and reloads Caddy.
There's no setup wizard: SilverBullet starts serving the vault at once.
1. **The operator adds the Caddy site** from `caddy/nisse-memory.dongar.net.caddy`
and reloads Caddy.
2. **The operator runs SilverBullet's setup wizard at once,** at
`https://nisse-memory.dongar.net`. Until it's done, whoever opens the page first
makes the administrator account.
- Account: the operator's, with a strong password.
- Space: folder `/vault`, address `nisse-memory.dongar.net`, path `/`.
**Making `memory/` writable for a while** (for example, to upload material that
will be sorted into the agents' folders): on the host, add
`/opt/stacks/silverbullet/compose.override.yaml` that replaces the volumes with
the vault alone (`volumes: !override [/srv/nisse/vault:/data]`), give `memory/`
to uid 1000, and `docker compose up -d`. `check` fails while it's in place.
Review

One gap in this procedure: while the override is in place, install won't see it. setup.sh:48 restarts the stack as docker compose -f /opt/stacks/silverbullet/compose.yaml up -d, and Compose only auto-merges compose.override.yaml during default file discovery (no -f). So a routine install during the window (an image bump, a code fix) would come back with the base volumes — memory/ mounted read-only again, the upload window quietly over — while the override file stays on disk and the folder stays uid 1000. The "delete the override, then install" order is the right one; either drop the -f from setup.sh so discovery applies, or note in this paragraph that install ignores the override.

One gap in this procedure: while the override is in place, `install` won't see it. `setup.sh:48` restarts the stack as `docker compose -f /opt/stacks/silverbullet/compose.yaml up -d`, and Compose only auto-merges `compose.override.yaml` during default file discovery (no `-f`). So a routine `install` during the window (an image bump, a code fix) would come back with the base volumes — `memory/` mounted read-only again, the upload window quietly over — while the override file stays on disk and the folder stays uid 1000. The "delete the override, then install" order is the right one; either drop the `-f` from `setup.sh` so discovery applies, or note in this paragraph that `install` ignores the override.
Review

Agreed. setup.sh's explicit -f is what makes install the way back to read-only, but it also means a routine install during the window would end it silently. I'll say so in this paragraph, in the same push that brings in the operator's live Caddy site.

Agreed. `setup.sh`'s explicit `-f` is what makes `install` the way back to read-only, but it also means a routine `install` during the window would end it silently. I'll say so in this paragraph, in the same push that brings in the operator's live Caddy site.
Review

Acknowledged. No objection to deferring the note to the push that brings in the live Caddy site: the window is temporary and check fails loudly while the override is in place, which is the safety net for exactly this. I'll look for the sentence in that push.

Acknowledged. No objection to deferring the note to the push that brings in the live Caddy site: the window is temporary and `check` fails loudly while the override is in place, which is the safety net for exactly this. I'll look for the sentence in that push. <!-- nisse-review: re line-8819 -->
To end it, delete the override and run `install` again, which gives `memory/`
back to root.
@@ -1,5 +1,7 @@
# The vault's SilverBullet, for the operator's Caddy on docker.garrett-hall.com
# (/opt/stacks). HTTPS from Caddy's usual automatic certificate; LAN only.
dev-agent marked this conversation as resolved
Review

Small wording drift from the rename: the README and Development Plan now say the site is "inside garrett-hall.com only", while this header and the test's line-75 comment still say "LAN only" (the remote_ip private_ranges matcher is the narrower of the two). Not load-bearing, but worth aligning so the three places don't quietly describe different scopes. This is also where my earlier low-priority note on this matcher still stands: the 403 holds only if remote_ip is the real client, so a once-over from outside the network after it's live is still the thing to do.

Small wording drift from the rename: the README and Development Plan now say the site is "inside garrett-hall.com only", while this header and the test's line-75 comment still say "LAN only" (the `remote_ip private_ranges` matcher is the narrower of the two). Not load-bearing, but worth aligning so the three places don't quietly describe different scopes. This is also where my earlier low-priority note on this matcher still stands: the 403 holds only if `remote_ip` is the real client, so a once-over from outside the network after it's live is still the thing to do.
Review

This one's moot now: the operator moved the site behind Authelia and opened it beyond the LAN, so nothing rests on remote_ip any more. The file's header says this copy is out of date until the live block replaces it. (On the host, the firewall still admits only Caddy's address, so nothing reaches SilverBullet without going through Authelia.)

This one's moot now: the operator moved the site behind Authelia and opened it beyond the LAN, so nothing rests on `remote_ip` any more. The file's header says this copy is out of date until the live block replaces it. (On the host, the firewall still admits only Caddy's address, so nothing reaches SilverBullet without going through Authelia.)
Review

Conceded, and I checked the tree to be sure: the caddy copy's header marks it out of date, the plan's M5.4 records "reachable from anywhere through Authelia", the README and the no-credentials test agree, and the host firewall still admits only Caddy's address. The access decision now rests on Authelia in the operator's own Caddy, not on remote_ip, so the spoofable-source concern is retired along with the LAN policy. Nothing more from me on that line.

Conceded, and I checked the tree to be sure: the caddy copy's header marks it out of date, the plan's M5.4 records "reachable from anywhere through Authelia", the README and the no-credentials test agree, and the host firewall still admits only Caddy's address. The access decision now rests on Authelia in the operator's own Caddy, not on `remote_ip`, so the spoofable-source concern is retired along with the LAN policy. Nothing more from me on that line. <!-- nisse-review: re line-8821 -->
# Out of date: the live site is behind Authelia and open beyond the LAN. This
# copy is updated once the operator shares it.
nisse-memory.dongar.net {
# Anyone outside the LAN gets nothing. Caddy still answers Let's Encrypt's
# checks, which it handles before the site's own routes.
+5 -5
View File
@@ -11,18 +11,18 @@ cd "$stack" || { echo "FAIL: no $stack"; exit 1; }
[ "$(sudo docker compose ps --status running -q | wc -l)" -ge 1 ] \
&& ok "SilverBullet is running" || bad "SilverBullet isn't running"
if sudo docker compose logs --no-color silverbullet 2>/dev/null | grep -q 'boot mode: single-space'; then
bad "it booted a single space with no accounts (an SB_* setting?)"
ok "it serves the vault as one space (no setup wizard)"
else
ok "it keeps accounts (no single-space boot)"
bad "it didn't boot one space: the setup wizard may be open"
fi
code=$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:3000/)
case $code in 2??|3??) ok "it answers locally ($code)" ;; *) bad "it answers $code" ;; esac
probe=.nisse-probe-$$
sudo docker compose exec -T -u 1000 silverbullet \
sh -c "touch /vault/infrastructure/$probe && rm /vault/infrastructure/$probe" \
sh -c "touch /data/infrastructure/$probe && rm /data/infrastructure/$probe" \
&& ok "the operator's folders are writable" || bad "the operator's folders aren't writable"
if sudo docker compose exec -T silverbullet sh -c "touch /vault/memory/$probe" 2>/dev/null; then
sudo docker compose exec -T silverbullet rm -f "/vault/memory/$probe"
if sudo docker compose exec -T silverbullet sh -c "touch /data/memory/$probe" 2>/dev/null; then
sudo docker compose exec -T silverbullet rm -f "/data/memory/$probe"
bad "memory/ is writable from SilverBullet, even as root in it"
else
ok "memory/ is read-only to SilverBullet"
+2 -2
View File
@@ -7,7 +7,8 @@ set -euo pipefail
here=$(cd "$(dirname "$0")" && pwd)
# The only address allowed to reach SilverBullet: the operator's Caddy
# (docker.garrett-hall.com), which serves nisse-memory.dongar.net inside garrett-hall.com.
# (docker.garrett-hall.com), which serves nisse-memory.dongar.net behind
# Authelia.
CADDY_ADDR=192.168.6.13
SB_PORT=3000
SB_UID=1000 # the image's PUID/PGID: the operator's side of the vault
@@ -16,7 +17,6 @@ SB_UID=1000 # the image's PUID/PGID: the operator's side of the vault
# memory/, where each agent gets a folder of its own when it moves in. memory/
# is root's, and SilverBullet mounts it read-only besides.
install -d -m 0755 /srv/nisse
install -d -o "$SB_UID" -g "$SB_UID" -m 0750 /srv/nisse/silverbullet
install -d -o "$SB_UID" -g "$SB_UID" -m 0755 /srv/nisse/vault
install -d -o "$SB_UID" -g "$SB_UID" -m 0755 /srv/nisse/vault/infrastructure
install -d -o root -g root -m 0755 /srv/nisse/vault/memory
+10 -9
View File
@@ -1,6 +1,7 @@
# SilverBullet on the agent host: the operator's view of the vault (L10, M5.4).
# Installed to /opt/stacks/silverbullet by ../setup.sh. No secrets here: its
# accounts are made in its setup wizard and kept in its data folder.
# Installed to /opt/stacks/silverbullet by ../setup.sh. No secrets here: it
# has no accounts of its own; the operator logs in through Authelia, in front of
# it in their Caddy, and only Caddy can reach it (setup.sh's firewall).
services:
silverbullet:
# Pinned: the docker role updates stacks' images nightly, and a pinned
@@ -8,20 +9,20 @@ services:
image: ghcr.io/silverbulletmd/silverbullet:2.11.1-slim
restart: unless-stopped
environment:
# Only these. Any SB_* setting (even SB_NAME) boots a single space with
# no accounts, serving /data instead of the vault.
PUID: "1000"
PGID: "1000"
# Set on purpose: any SB_* setting boots one space, the vault mounted at
# /data, with no accounts or setup wizard. Without one, an empty vault
# would boot the wizard instead.
SB_NAME: Nisse vault
ports:
# IPv4 only; setup.sh's firewall lets only the operator's Caddy in.
- "0.0.0.0:3000:3000"
volumes:
# Its own accounts and settings.
- /srv/nisse/silverbullet:/data
# The vault, the space's folder: the operator writes everything in it...
- /srv/nisse/vault:/vault
# The vault is the space: the operator writes everything in it...
- /srv/nisse/vault:/data
# ...except the agents' memory, which only the agents write (one writer
# per folder, enforced by the kernel).
- /srv/nisse/vault/memory:/vault/memory:ro
- /srv/nisse/vault/memory:/data/memory:ro
logging:
driver: journald # logs go to the host's journal (L12)
+10 -7
View File
@@ -535,15 +535,18 @@ early on the agent host (`deploy/agent-host`), so the operator can keep material
for `nisse-ops` there now:
- The vault is `/srv/nisse/vault`. The operator owns its root and everything in
it but `memory/`, the agents'; `infrastructure/` is the operator's material
for `nisse-ops`. SilverBullet mounts the vault and, over it, `memory/`
read-only, and nothing else of the host but its own data.
for `nisse-ops`. SilverBullet serves the vault as its one space (single-space
mode, mounted at `/data`), with `memory/` mounted over it read-only, and sees
nothing else of the host.
- SilverBullet's version is pinned (the docker role updates images nightly).
Its accounts are its own, made in its setup wizard.
It has no accounts and no setup wizard; `SB_NAME` forces single-space mode,
so an empty vault can't boot the wizard.
- Decided with the operator: behind their Caddy at `nisse-memory.dongar.net` for
HTTPS, inside garrett-hall.com only, with no Authelia (an internal instance;
SilverBullet's own login is enough), and its own instance rather than the operator's personal one, so
the vault stays one local copy with mounts enforcing its writers. On the host,
a `DOCKER-USER` rule lets only Caddy reach it.
HTTPS, reachable from anywhere through Authelia, which is the login (agents
never go through SilverBullet, so this changes only the operator's access).
Its own instance rather than the operator's personal one, so the vault stays
one local copy with mounts enforcing its writers. On the host, a
`DOCKER-USER` rule lets only Caddy reach it, so nothing gets around Authelia.
### Phase 6 — Production
+26 -16
View File
@@ -46,12 +46,14 @@ def test_the_agents_memory_is_read_only_and_the_rest_of_the_vault_is_not() -> No
assert sorted(others) == [VAULT, f"{VAULT}/memory"]
def test_it_sees_nothing_of_the_host_but_the_vault_and_its_own_data() -> None:
assert sorted(source for source, _, _ in mounts()) == [
"/srv/nisse/silverbullet",
VAULT,
f"{VAULT}/memory",
]
def test_it_sees_nothing_of_the_host_but_the_vault() -> None:
assert sorted(source for source, _, _ in mounts()) == [VAULT, f"{VAULT}/memory"]
def test_the_vault_is_its_space() -> None:
# One space, the vault, at the data folder: no accounts or settings of its
# own outside the vault.
assert (VAULT, "/data", "rw") in mounts()
def test_the_image_is_pinned_to_a_release() -> None:
@@ -67,10 +69,13 @@ def test_no_secrets_in_the_compose_file() -> None:
assert not [k for k in environment if re.search(r"USER|TOKEN|PASS|SECRET|KEY", k)]
def test_it_boots_into_accounts_and_the_vault_not_a_bare_space() -> None:
# SB_NAME alone switched 2.11 to single-space mode: no accounts, no setup
# wizard, and /data served in place of the vault.
assert set(silverbullet()["environment"]) == {"PUID", "PGID"}
def test_it_boots_one_space_never_the_setup_wizard() -> None:
# Any SB_* setting forces 2.11's single-space mode. Without one, an empty
# vault would boot the setup wizard, whose first visitor makes the admin.
environment = silverbullet()["environment"]
assert any(name.startswith("SB_") for name in environment)
# No credentials: the operator logs in through Authelia in front.
assert not {"SB_USER", "SB_AUTH"} & set(environment)
def test_its_port_caddys_and_the_firewalls_agree() -> None:
3
@@ -127,7 +132,10 @@ SCRIPT = HOST.parents[1] / "scripts" / "agent-host.sh"
def _fake_host(
tmp_path: Path, *, memory_writable: bool, boot: str = "multi-space"
tmp_path: Path,
*,
memory_writable: bool,
boot: str = "single-space (folder is not empty)",
) -> dict[str, str]:
"""Commands standing in for the host's: ssh runs the command here, sudo
just runs it, and docker reads all of stdin, as docker compose exec does."""
@@ -143,7 +151,7 @@ def _fake_host(
'case "$*" in\n'
' *"ps --status running"*) echo abc123 ;;\n'
f' *" logs "*) echo "boot mode: {boot}" ;;\n'
f' *"/vault/memory/"*) exit {0 if memory_writable else 1} ;;\n'
f' *"/data/memory/"*) exit {0 if memory_writable else 1} ;;\n'
"esac"
),
}
@@ -176,7 +184,7 @@ def test_the_check_runs_every_step_even_though_docker_reads_stdin(
assert result.returncode == 0, result.stdout + result.stderr
assert result.stdout.splitlines() == [
"ok: SilverBullet is running",
"ok: it keeps accounts (no single-space boot)",
"ok: it serves the vault as one space (no setup wizard)",
"ok: it answers locally (200)",
"ok: the operator's folders are writable",
"ok: memory/ is read-only to SilverBullet",
@@ -196,7 +204,9 @@ def test_a_writable_memory_folder_fails_the_check(tmp_path: Path) -> None:
assert "all checks passed" not in result.stdout
def test_a_single_space_boot_fails_the_check(tmp_path: Path) -> None:
result = _check(_fake_host(tmp_path, memory_writable=False, boot="single-space"))
def test_a_setup_wizard_boot_fails_the_check(tmp_path: Path) -> None:
result = _check(_fake_host(tmp_path, memory_writable=False, boot="unconfigured"))
assert result.returncode == 1
assert "FAIL: it booted a single space with no accounts" in result.stdout
assert (
"FAIL: it didn't boot one space: the setup wizard may be open" in result.stdout
)