Skip to content

Unix sockets and fd inheritance

Two ways to bind that aren't a plain TCP port — AF_UNIX for local reverse proxies, and fd inheritance for systemd socket activation.

AF_UNIX — local domain socket

When BlackBull runs behind a local reverse proxy (nginx, Caddy), using an AF_UNIX socket eliminates TCP overhead and avoids exposing a port on 0.0.0.0:

# CLI
blackbull myapp:app --bind unix:/run/blackbull.sock

# nginx upstream
upstream blackbull { server unix:/run/blackbull.sock; }

# In Python
app.run(unix_path='/run/blackbull.sock')

The socket file is created with mode 0660 so a reverse proxy running in the same group can connect without a chmod. A leftover socket file at the path is removed automatically before bind; BlackBull refuses to unlink a regular file or directory at that path (safety check).

TCP-only socket options (SO_REUSEPORT, TCP_USER_TIMEOUT, IPV6_V6ONLY) are skipped for AF_UNIX sockets — they carry no meaning on a domain socket.

fd inheritance — systemd socket activation

Systemd can pre-bind the port as root and then start BlackBull unprivileged, handing the bound socket as an open file descriptor:

/etc/systemd/system/blackbull.socket
[Socket]
ListenStream = 443

[Install]
WantedBy = sockets.target
/etc/systemd/system/blackbull.service
[Service]
User      = www-data
ExecStart = blackbull myapp:app --bind fd://3

# In Python — equivalent shape
app.run(inherited_fd=3)

BlackBull validates the $LISTEN_PID / $LISTEN_FDS environment variables per the sd_listen_fds(3) protocol:

  • LISTEN_PID must equal the current process PID — if it points elsewhere BlackBull refuses to adopt the fd (prevents accidentally stealing another process's socket).
  • LISTEN_FDS defines the valid fd window [3, 3 + LISTEN_FDS). Fds outside that window are rejected.

When neither variable is set (non-systemd handoff, tests) BlackBull accepts the fd unconditionally.

With BB_SOCKET_REUSEPORT=1, several workers and no --reload, a creator-held IPv4-only, IPv6-only, or dual-stack socket would leave some or all connections unserved — startup refuses each shape instead (see Several workers behind an activated socket).

What systemd activation buys you

  • Bind privileged ports without running as root. systemd binds :443 while running as root; BlackBull starts as www-data and inherits the already-bound socket.
  • Zero-downtime restarts. systemd keeps the socket open across stop/start cycles — connections that arrive while the new BlackBull is launching queue in the kernel accept buffer instead of being refused.
  • Lazy activation. The socket is ready before BlackBull starts; the first connection wakes the service.

Several workers behind an activated socket

systemd-socket-activate execs the service in place, so the port is the workers' to re-bind; a systemd.socket unit keeps its own copy for the service's whole lifetime, and ReusePort=yes leaves that copy in the port's reuseport group — the per-worker bind then succeeds while the supervisor's copy keeps taking connections no worker accepts on. A socket bound in another network namespace refuses for the same reason: the workers would bind where its clients are not. The namespace comparison needs SO_NETNS_COOKIE (Linux 5.14+); a host that will not give it reads the socket as local, and that refusal is skipped silently. Where /proc/net/tcp* cannot be read — or reads empty — a released ReusePort=yes socket is refused too: there is nothing to tell it apart from a kept one.

The startup window

Accepts begin only after lifespan startup completes, so arrivals before then wait in the kernel's accept queue, bounded by the creator's Backlog= where the fd is adopted and by BB_SOCKET_BACKLOG where BlackBull binds it. The window lasts as long as the startup hook runs, with no deadline from the framework. Budget startup memory across every listener's backlog and the receive buffers of parked connections. Effective kernel values are platform-dependent; adopted sockets retain the creator's backlog. BB_SOCKET_RCVBUF affects listeners BlackBull binds and is inherited by their accepted connections. Long startup hooks can exceed clients' connect budgets.

What a client beyond the backlog observes

Until startup completes, backlog + 1 connections can wait. Beyond that:

family the client
TCP retransmits, and is served once startup completes
AF_UNIX, non-blocking (nginx, asyncio, a Python socket with a timeout) is refused at once (EAGAIN)
AF_UNIX, blocking waits, and connects once the queue drains

Set BB_SOCKET_BACKLOG above the number of connections you expect during startup, especially on AF_UNIX. An adopted fd (--bind fd://N) keeps the backlog it was created with until accepting opens, and BB_SOCKET_BACKLOG after: size the startup burst where the socket is created, such as systemd's Backlog=. net.core.somaxconn caps both. On shutdown an activated socket's file stays in place; a stale file left after --reload is replaced on the next start.

On Linux, a worker that finds an AF_UNIX listener's queue full as it opens accepting logs one socket_backlog warning on blackbull.caps, naming the listener, the connections waiting and the backlog. An adopted fd created in another network namespace (systemd PrivateNetwork=yes) cannot be read and is not checked.

Inspecting the bind

server.port returns the kernel-assigned port number after open_socket(port=0) — useful in tests where you want an ephemeral port:

from blackbull.server import ASGIServer

server = ASGIServer(app)
server.open_socket(port=0)
print(server.port)        # e.g. 39423

For AF_UNIX sockets server.port is the path string instead of a port number.

Next