A cluster-first container engine for FreeBSD.
SatL runs OCI containers as FreeBSD jails, with the orchestration inside the daemon rather than beside it — an embedded Raft store, a scheduler, VXLAN overlays, an ingress routing mesh and mTLS everywhere. It speaks the Docker Engine API, so the tooling you already have works against it unchanged.
A single node is a cluster of one, and every container is a task of a service. There is nothing to init.
satl service create --name web --replicas 3 -p 8080:80 freebsd-nginx:latestsatl stack deploy -c compose.yaml shopsatl service update --limit-memory 512m web
Three replicas behind a published port, a stack from a compose file, and a memory limit rewritten on the live jails — no restart, same task IDs.
Results
What it does on a three-node cluster
Measured on FreeBSD 15.1, with the conditions for each figure stated underneath.
0of 6612
requests lost during a rolling update
A six-replica service updated across three nodes while traffic ran through the ingress routing mesh. 6612 requests sampled, none dropped.
9.97s
from failed probe to out of the traffic pool
Health probe failure to the task address leaving the pf redirect pool. Two runs: 9.967 s and 9.971 s. Docker’s defaults take roughly ninety.
6s
for a lost manager to rejoin
End to end on a cluster that still had quorum, with no backup involved. The cluster kept committing writes throughout.
7s
warm rebuild, against 51 s cold
The same image rebuilt through the content-addressed incremental build cache with nothing changed.
1.4MB
smallest FROM scratch image
A statically linked C binary, assembled in a multi-stage build from a FreeBSD toolchain stage.
The full set, with the runs that produced them, is in the project’s roadmap and across the status page. Measured on a small cluster — your hardware and workload will differ.
Quick install
Two commands, and six things to do first
The order matters. The daemon will start with several of these missing and degrade quietly, which is why host preparation comes before the package rather than after it.
Install the package
A self-contained FreeBSD package — it needs no repository of its own, and it installs four files. Two steps rather than one so you can see the file land, check it, and copy it to the other nodes of a cluster instead of downloading it three times.
fetch https://satl.cc/download/satl-freebsd.pkgpkg add ./satl-freebsd.pkg
One package, every node. A cluster wants the same build everywhere. Fetch once,scp the file around, pkg add on each machine — mixing versions across nodes is not a configuration SatL is tested in.
Or build it from source
This is also how you get an unreleased fix, and it is the only path that needs a Rust toolchain — Rust 1.96, edition 2024.
git clone https://github.com/fredericalix/satl satlcd satlmake install
Check it came up
satl versionsatl node lsdocker -H unix:///var/run/satl.sock version
The third line only applies if you have the docker CLI installed. It is the fastest proof that the Engine API is answering.
Prepare the host first
In this order — each one depends on the one before it. Every step here, with the symptom you get for skipping it, is on the requirements page; the package’s post-install message recalls them too.
Create the ZFS dataset
zfs create -o mountpoint=/var/db/satl zroot/satlZFS is mandatory, not one storage driver among several — a layer *is* a dataset.
satldrefuses to start without this. Substitute your pool name and setzfs_rootin the config if it is notzroot.Turn on IP forwarding
sysrc gateway_enable=YESsysctl net.inet.ip.forwarding=1Container traffic is routed between the bridge and the egress interface. Skip it and you get the most misleading symptom in SatL: published ports answer, and containers cannot reach anything.
Declare the pf anchors
nat-anchor "satl/*"rdr-anchor "satl/*"anchor "satl/*"pass allIn
/etc/pf.conf, translation anchors before any filter rule. SatL owns thesatl/*anchors and never writes a rule outside them. An anchor not declared inpf.confis never evaluated —satldwill load rules into it, report success, and change nothing about how packets move.Set the boot tunables
kern.racct.enable=1if_vxlan_load="YES"In
/boot/loader.conf, then reboot when convenient. The first enables enforced--memoryand--cpusthroughrctl; the second is only needed for overlay networks.Write satld.toml — do not skip this
pf_mode = "enforce"The package installs
satld.toml.sample, notsatld.toml. A missing config is legal and the daemon runs on defaults — and the defaultpf_modeischeck, which generates the pf rules, syntax-checks them, and never loads one. A published port is then allocated, recorded and shown bysatl psexactly as if it worked, with no redirect behind it and nothing logged as an error. This catches essentially every first install.Enable and start the daemon
sysrc satld_enable=YESservice satld startThen read the startup lines:
grep -a satld /var/log/messages | tail -40. The-ais not optional.
What is built
Eight things worth knowing
Not a roadmap. Every item below is in the release, and every one has been run on FreeBSD.
Containers are jails
One VNET jail per task, with its own network stack, driven through the ocijail runtime. SatL implements no container runtime of its own and never will — it pulls the image, applies the layers, generates an OCI runtime spec, and hands it over.
ZFS-native image store
A layer is a dataset. Applying a layer is a snapshot and a clone; a container’s writable layer is a clone. There is no graph driver to choose and nothing to tune, because there is no alternative to tune it against.
The surface is Docker’s
satld serves the Docker Engine REST API v1.43 on /var/run/satl.sock, negotiable down to 1.24, and the satl CLI mirrors the verbs you already know.
docker -H unix:///var/run/satl.sock versionworks, and so does your existing tooling.Nothing to initialise
A fresh daemon mints a node identity, a cluster identity and a root CA, initialises a one-member Raft cluster, and starts scheduling. There is no swarm init, no standalone mode, and no etcd beside it — one daemon is the whole control plane.
VXLAN overlays, DNS discovery
One VNI per network, unicast, with a Raft-distributed forwarding table and learning turned off. A service name resolves to its running tasks, shuffled per query, through a DNS responder on every node.
An ingress mesh in pf
Ports are allocated cluster-wide and every manager answers every published port, relaying to a live task with return-path SNAT. Opt into PROXY protocol v2 per service when a task needs the real client address.
mTLS everywhere, by default
An ECDSA P-256 cluster CA issues 90-day node certificates, renewed live and swapped into the running TLS config without a restart. The root rotates on a live cluster. Every internal connection is mutually authenticated, with the role carried in the certificate.
Secrets that never touch a disk
Encrypted at rest in the Raft log, delivered over the mTLS dispatcher stream, materialised on a per-task tmpfs. A worker never writes a secret to its own storage.
How it works
One daemon, and nothing beside it
satld holds the API, the store, the scheduler, the orchestration loops and the agent. Adding a machine adds another copy of the same daemon — there is no second component to install and no separate control plane to operate.
- The agent talks to the dispatcher over a socket even on a single node, using the same session protocol a remote worker uses. There is no shortcut path — which is why one node and a cluster behave identically.
- The store is the only place cluster state lives. Every write goes through the leader; followers forward once.
- Workers dial managers, never the reverse. The firewall rule is one-directional.
What happens when you create a service
satl service create --replicas 3 --publish 8080:80 web, end to end.
The spec is committed
The CLI posts to the Engine API. The leader commits the Service through Raft, and the allocators claim a published port, a subnet, a VNI and the task addresses in the same transaction.
The orchestrator creates tasks
The replicated orchestrator commits three Tasks in NEW. A task is one attempt at one replica on one node as one jail — immutable and one-shot. The slot is what carries the replica’s identity across replacements.
The scheduler places them
Filters first — readiness, resources, constraints, platform, host ports, replica caps — then spread ranking. Tasks move to ASSIGNED. Constraints are enforced continuously, so editing a node label moves running tasks.
The agent builds the jail
The dispatcher session delivers the assignment. The agent pulls the image, clones the layers, generates the OCI spec, creates the epair and the jail, and drives PREPARING → READY → STARTING → RUNNING. A periodic pass then re-derives the whole satl/rdr pf anchor from what is actually running locally.
What the model buys you
One orchestrator, or none, is not a choice you have to make
No development mode that behaves differently from production. No compose file on one host and something else on three. No moment where you rebuild your mental model because you added a machine.
Desired state is the only state you set
You do not start containers. You declare four replicas, and level-triggered loops make it true and keep it true. Nothing is triggered by an event alone.
Every lifecycle transition has a name
Each one is a state-machine step, logged with its task, service and node identifiers. Diagnosis is grep-by-identity rather than inference.
The API stays honest
Options SatL cannot honour are refused with a 400 and the reason named, not accepted and quietly ignored. A half-honoured isolation flag is a security trap.
Networking
Overlays in VXLAN, the data path in pf
SatL owns the satl/* pf anchors and never writes a rule outside them. Everything a packet does — egress NAT, a published port, the mesh relay, the encrypted guard — is one of those anchors, re-derived from what is actually running.
Overlay MTU is measured, not assumed
1450 on a 1500-byte underlay, because VXLAN costs 50 bytes; 1416 once a network is encrypted, because ESP adds a measured 34 on top. SatL derives it from the underlay it actually finds rather than from a constant.
The forwarding table is distributed, not learned
One VNI per network, unicast, with static entries distributed through Raft and learning turned off. That is also why the usual 2000-endpoint ceiling does not apply — 2500 entries install cleanly at vxlanmaxaddr 2000.
Discovery is DNS round-robin
FreeBSD has no IPVS, so there is no service virtual IP. A per-node DNS responder resolves a service name to its running tasks, shuffled per query. Clients that cache a single A record for a long time will notice.
Encryption is per network, and rotates itself
Opt in with --opt encrypted: IPsec ESP with AES-128-GCM on the VXLAN data plane, keys delivered only to participating nodes, rotated automatically every twelve hours. A pf guard anchor drops cleartext on the encrypted ports.
Docker compatibility
The first hour is mostly familiar
The satl CLI speaks the verbs you already know — twenty-six of them — and satld serves the Docker Engine REST API v1.43 on a unix socket, negotiable down to 1.24. The right-hand column below is not a translation layer; it is the same command.
docker run -d -p 8080:80 nginxsatl run -d -p 8080:80 nginxdocker pssatl psdocker logs -f websatl logs -f webdocker build -t app .satl build -t app .docker service scale web=5satl service scale web=5docker stack deploy -c compose.yaml shopsatl stack deploy -c compose.yaml shopOr keep using the docker CLI
There is no shim and no wrapper. satld answers the Engine API, so point DOCKER_HOST at its socket and your existing tooling works unchanged.
export DOCKER_HOST=unix:///var/run/satl.sockdocker versiondocker compose up -d
There is no TCP listener for the API, deliberately — the socket is the only surface, and it is group-owned rather than world-writable.
Where it differs, it says so
An option SatL cannot honour comes back as a 400 with the reason named, rather than being accepted and quietly ignored — Privileged, CapAdd, Devices, Sysctls, IPv6 subnets and the rest. A half-honoured isolation flag is a security trap, so SatL would rather fail your docker run than pretend.
Why FreeBSD
Every piece already existed
SatL invents no isolation primitive, no filesystem layer and no packet path. Jails, ZFS, if_bridge, pf, rctl and if_vxlan have been in the base system for years — SatL is the part that was missing: what drives them from a desired state.
| A container needs | FreeBSD gives | SatL uses it as |
|---|---|---|
| Isolation | jail(8) | One jail per task, with vnet for its own network stack |
| A layered filesystem | ZFS | A layer is a dataset; applying one is a snapshot and a clone |
| Networking | if_bridge, epair, vnet | One bridge per network, one epair per task |
| The data path | pf(4) | NAT for egress, rdr for published ports, inside SatL’s own anchors |
| Resource limits | rctl(8) / racct | One rule per container, added and removed with it |
| Cross-node networking | if_vxlan(4) | One VNI per overlay, unicast, Raft-distributed forwarding table |
Before you start
What the host has to be
Tested on FreeBSD 15.1 and CURRENT, amd64. The first five are hard requirements — the daemon either refuses to start without them or misbehaves in a way that looks like something else. The last four are things you will want before you trust it.
FreeBSD 15.1 or CURRENTrequiredamd64. Tested on both. IPv4 only.
A ZFS pool and root datasetrequiredMandatory. satld refuses to start without it. Defaults to zroot/satl.
ocijailrequiredpkg install ocijail — declared as a package dependency.
rootrequiredThe daemon creates jails, datasets, interfaces and pf anchors.
pf, loaded and enabledrequiredpf is the data path, with the satl/* anchors declared in pf.conf.
kern.racct.enable=1A boot tunable, so it needs a reboot. Enforces --memory and --cpus; degrades gracefully without it.
gateway_enable=YESContainer egress. Without it, published ports answer and containers reach nothing.
2377/tcp, 2378/tcp, 4789/udpClustering only. 2377 is mTLS, 2378 is CA bootstrap, 4789 is the VXLAN data plane.
Rust 1.96, edition 2024Only to build from source. pkg install rust gives exactly 1.96 — there is no margin.
A FreeBSD port and pkg install satl from the official repositories do not exist yet — the package on this site is self-contained and installs with no repository configured. Version0.1.0-beta.
Compared
Against Docker and Swarm
Several rows here are losses, and they stay in. A comparison that only lists wins tells you nothing you can plan against — and every one of these differences is already documented in full, numbered, on the docs site.
| Docker / Swarm | SatL | |
|---|---|---|
| Getting started | docker swarm init required | Nothing to initialise — a cluster of one from first boot — a difference in SatL’s favour |
| Control plane | dockerd plus swarmkit, or etcd | One daemon, embedded Raft, nothing beside it — a difference in SatL’s favour |
| Standalone containers | A separate mode with its own semantics | None — every container is a task of a service — a difference in SatL’s favour |
| Storage | overlay2, plus a graph-driver choice | ZFS, mandatory — no driver to pick, nothing to tune — a trade-off |
| Service load balancing | An IPVS virtual IP | DNS round-robin — FreeBSD has no IPVS, so there is no VIP — a trade-off |
| Routing mesh | Every node answers a published port | Every manager answers, through pf — a trade-off |
| Data path | iptables | pf, confined to SatL-owned satl/* anchors — a difference in SatL’s favour |
| Memory limit | cgroup throttle, then an OOM kill | rctl memoryuse:sigkill — a kill, with no throttling or reclaim — a trade-off |
| Resource resize | Rolls the service | Hot resize — live rctl rewrite, same task IDs, no restart — a difference in SatL’s favour |
| Unhealthy container | Left running, marked unhealthy | Stopped and replaced — health gates the RUNNING state — a difference in SatL’s favour |
| Unsupported options | Often accepted and silently ignored | Refused with a 400 and the reason named — a difference in SatL’s favour |
| Unsupported compose keys | “Ignoring unsupported options: …” | The whole file is refused, key and line named — a difference in SatL’s favour |
| Join token | SWMTKN-… | SATL-1-<digest>-<secret> — the digest pins the whole root CA bundle — a difference in SatL’s favour |
| Platform | Linux, Windows, macOS | FreeBSD 15.1 and CURRENT on amd64, IPv4 only — a trade-off |
A tick marks a difference in SatL’s favour; a dash marks a trade you are accepting. Both are differences, and the docs explain the reasoning for each.
Clustering
Adding the second machine is a join, not a migration
There is no standalone mode to grow out of. The first node is already a cluster — of one — running the same Raft store, the same scheduler and the same dispatcher session protocol it will run with five. Nothing about it changes shape when the second one arrives.
One node
Install the package, write the config, start the daemon. It mints its identity and root CA, initialises a one-member Raft cluster, and begins scheduling. It is already a cluster — of one.
service satld startsatl node lsA join token
Ask the first node for one. The digest in the token pins the whole root CA bundle, so a joiner cannot be talked onto the wrong cluster.
satl swarm join-token managerThe second node
A join, not a migration. Nothing about the first machine changes shape, and nothing you deployed on it needs rewriting.
satl swarm join --token SATL-1-… node1:2377
SATL-1-… token pins the whole root CA bundle, so a joiner cannot be talked onto the wrong cluster.Managers run tasks too
A manager holds a Raft replica and elects a leader, and it also schedules and runs containers. Three managers is the recommended shape for a small cluster — it survives one loss, and quorum is what you pay for that.
Workers dial managers, never the reverse
Sessions are outbound from the worker, so the firewall rule is one-directional. The control plane is 2377 over mutual TLS; 2378 exists unauthenticated because a first-time joiner has no certificate yet.
Certificates renew themselves
Ninety-day node certificates, renewed at a random point inside a 50–80% window and swapped into the running TLS config with no restart. The root rotates on a live cluster through a cross-signed intermediate.
Questions
The ones people actually ask
Including the two that are not bugs.
Is this production-ready?
No, and it does not claim to be. This is a proof of concept at 0.1.0-beta: the feature set is complete enough to run real workloads — the documentation site’s Node.js and MariaDB tutorial runs end to end on a three-node cluster — but there has been no independent security audit, there is no compatibility promise between pre-1.0 versions, and no upgrade path across them. Run it where losing it is survivable.
Do my existing Docker Compose files work?
Standard Compose files, yes, with stack semantics rather than single-host ones. A file that uses keys SatL cannot honour is refused whole, with the key and the line named, rather than half-deployed with a warning you might miss.
Can I point the docker CLI at it?
Yes. satld serves the Docker Engine REST API v1.43 on /var/run/satl.sock, negotiable down to 1.24. Set DOCKER_HOST=unix:///var/run/satl.sock and your existing tooling talks to it unchanged. There is no TCP listener for the API, by design.
Can I run Linux images?
Yes. linux/amd64 images run under the linuxulator as a first-class fallback when no FreeBSD image exists, and both satl ps and satl images carry a PLATFORM column so you always know which you got.
What about IPv6?
Not yet — SatL is IPv4 only, with no IPv6 path anywhere. It is named on the status page as missing rather than planned-and-silent.
Why is ZFS mandatory?
Because a layer is not stored in ZFS, it *is* a dataset. Applying a layer is a snapshot and a clone, which is why there is no graph driver to select and nothing to tune. satld refuses to start without its root dataset rather than degrading into something slower and less predictable.
Why can’t I curl my published port from the host?
Because pf applies rdr to packets *entering* an interface, and traffic you originate on the publishing host never enters one. The redirect is correct; test it from another machine. This is the single most reported non-bug in SatL.
How do I upgrade?
For now, you do not — there is no supported upgrade path between pre-1.0 versions. A cluster wants the same build on every node, so fetch the package once, copy it around, and pkg add on each machine. Mixing versions across nodes is not a configuration SatL is tested in.
How do I report something?
A log excerpt and the command that produced it, on the engine’s issue tracker. That pair is worth more than a paragraph of description, and the documentation has a page on what makes a report useful. Security vulnerabilities go privately to security@satl.cc instead — never to an issue or a pull request.
Where it stands
Early, and real
This is a first version — a proof of concept — and the results so far are genuinely encouraging. The feature set already runs real workloads: the documentation site’s Node.js and MariaDB tutorial goes end to end on a three-node cluster. Everything described on this page and in the documentation has been run on FreeBSD 15.1 and CURRENT, not inferred from how it ought to behave.
Which is also why the list below exists. “Beta” here is a statement about the edges, not the middle, and the honest version of that statement is a list of names rather than a reassurance. The project will keep evolving — everyone is welcome to be part of that.
- 18Rust crates
- ~184klines of Rust
- ~2300tests
- 26CLI verbs
What is not there
No independent security audit
The security model is written down in full, and the deliberate choices are named — port 2378 is unauthenticated by design, the daemon runs as root, /metrics is unauthenticated, there is no user-level authorisation in the API. None of it has been reviewed by anyone outside the project.
No upgrade path yet
No compatibility promise between pre-1.0 versions, and no migration across them. Treat a version bump as a rebuild.
FreeBSD 15.1 and CURRENT, amd64, IPv4
Nothing else is built or tested. No arm64 build, no cross-compilation, and no IPv6 anywhere.
Node-local volumes only
No shared storage, no CSI, no cluster volumes. A stateful task is pinned to the machine holding its data, and placing it is your job.
Logs are per-container and node-local
There is no cluster-wide satl service logs. You go to the node.
A permanently lost quorum cannot be repaired from inside
There is a measured backup and restore procedure for the Raft dataset, but no satl verb performs it, and a cluster that loses quorum for good is rebuilt rather than recovered.
No CI, and one maintainer
Opening a pull request runs nothing. The build and the tests need FreeBSD 15.1 with ZFS, jails, pf and ocijail, which no hosted runner offers, so make check on a real host is the only gate — and running it is the contributor’s job.
Not for hostile multi-tenant workloads
Stated plainly in the security policy. Isolation options SatL cannot honour are refused rather than faked, but that is not the same as being hardened against a tenant who is trying.
Get involved
Everyone is welcome
This is a first version — a proof of concept — and the results so far are genuinely encouraging. It will keep evolving, and anyone who wants to be part of that is welcome.
The most useful thing you can send back
Run it on FreeBSD 15.1 or CURRENT and tell us what happened. A log excerpt and the command that produced it is worth more than a paragraph of description — the documentation has a whole page on what makes a report useful, because a good report is the difference between a fix and a guess.
There is no CI, deliberately
Opening a pull request runs nothing — no Actions, no checks tab. The build and the tests need FreeBSD with ZFS, jails, pf and ocijail, which no hosted runner offers. So make check on a real host is the only gate, and pasting its output into the pull request is what makes the change reviewable. Networking, runtime and storage changes also want sudo make integration; cluster behaviour wants make cluster-test.
Where the work is
The roadmap names what is missing rather than hiding it — IPv6, shared storage, cluster-wide logs, a real FreeBSD port. Those are the honest starting points. The contributor guide lives in the repository.
Vulnerabilities go privately
Not to an issue and not to a pull request: security@satl.cc. Scope and expectations are in the repository’s security policy, including the choices that are deliberate and therefore not findings.
Try it on a machine you can afford to lose
A self-contained FreeBSD package. It needs no repository, installs four files, and pulls ocijail if you have a package repository configured. Then read the host preparation steps — the daemon starts without several of them and degrades quietly.
fetch https://satl.cc/download/satl-freebsd.pkgpkg add ./satl-freebsd.pkg
FreeBSD 15.1 and CURRENT, amd64. ZFS mandatory. IPv4 only. BSD-2-Clause.
