Skip to content

Workers

By default app.run() runs one worker process with the standard asyncio event loop. This page covers how to scale to multiple workers, what uvloop buys you, and the abuse defences that protect each worker.

Pre-fork multi-worker

# Pre-fork N worker processes, each with its own event loop.
# 0 → os.cpu_count() workers.
BB_WORKERS=8 python app.py

# Equivalently via the CLI:
blackbull myapp:app --workers 8

# Or in code:
app.run(port=8000, workers=8)

Multi-worker uses pre-fork multiprocessing — each worker is a separate OS process, not a thread. The master process binds the socket, forks workers, and then sleeps until SIGTERM/SIGINT.

The master owns the complete listener set and every Process object until a successful handoff or shutdown. Worker creation is sequentially committed: if worker N cannot start, workers 0 through N−1 are terminated and joined before the startup error returns. The same rollback covers per-worker SO_REUSEPORT binds and watcher startup. Cleanup continues after an individual close or process-operation error, while the first startup error remains the reported failure. If cleanup itself has several independent failures and no startup error is active, they are reported together as an ExceptionGroup; a single cleanup failure is raised unchanged.

On Linux / modern BSDs each worker can bind its own listening socket via SO_REUSEPORT (BB_SOCKET_REUSEPORT=1), so the kernel hashes incoming connections across workers' accept queues — no thundering-herd, per-connection CPU affinity for free. It is off by default: with BB_SOCKET_REUSEPORT=0 (the default) all workers share the master's single socket, and every incoming connection wakes every worker's loop, with N−1 workers accept()-ing to EAGAIN — the thundering-herd accept race. See "Workers vs cores under connection churn" below for when turning it on helps and when it does not. An adopted fd (--bind fd://N) is not always the workers' to split — see Unix sockets and fd inheritance.

Shared-nothing model

Workers share nothing mutable — no shared dict, no shared cache, no shared lock. Anything that needs cross-worker coordination (session storage, response cache, rate-limit counters) belongs in an external store: Redis, Postgres, or a sticky-session reverse proxy.

The built-in Cache middleware (and the blackbull-session extension package) explicitly state their per-worker limits — see Middleware.

Which setting

Setting Use case
BB_WORKERS=1 (default) Development, tests, debugging. Easiest to reason about; everything in one process.
BB_WORKERS=N (N > 1) Production CPU-bound workload — each worker saturates one core.
BB_WORKERS=0 Production "use whatever the box has" — resolves to os.cpu_count() at start.

Socket options across bind paths

knob single shared + REUSEPORT adopted fd AF_UNIX
BB_SOCKET_BACKLOG bounds queue bounds queue per-worker queues creator's, until accepting bounds queue
BB_SOCKET_REUSEPORT n/a off per worker creator's socket blocks it not with > 1
BB_SOCKET_SNDBUF/RCVBUF set set set on each creator's value listener only
BB_TCP_USER_TIMEOUT_MS set set set on each creator's value unsupported
BB_MAX_CONNECTIONS per worker ×N ×N ×N ×N

Each per-worker listener repeats the address the master bound, so a listener on a named host stays on that interface instead of widening to every one; an adopted fd keeps the address its supervisor bound. Where the two columns meet — an adopted fd under + REUSEPORT — the workers re-bind with BlackBull's own options, so their queues carry BB_SOCKET_BACKLOG rather than the creator's Backlog=.

CPU pinning

Each worker pins its event loop to one core after forking, so the hot state it accumulates — the header line table, the HPACK dynamic tables — stays resident in that core's L1/L2 instead of following the thread around the machine. BB_CPU_PINNING controls it:

# Default: worker i takes the i-th CPU we are allowed to run on.
BB_CPU_PINNING=auto BB_WORKERS=8 python app.py

# Pin nothing — leave placement entirely to the operator.
BB_CPU_PINNING=off python app.py

# Confine workers to a specific set (taskset syntax). 0 is CPU 0,
# not the off switch.
BB_CPU_PINNING=8-11 BB_WORKERS=4 python app.py

Two things it deliberately does not do:

  • It never widens the mask it was given. Placement is drawn from the affinity mask the process already carries, so taskset -c 8-11, numactl, and a container's cpuset are inputs rather than obstacles. An explicit list is intersected with that mask, never added to it; a list with nothing in common with it pins nothing and logs a warning.
  • It never pins the thread pool. Linux threads inherit the creating thread's mask, so pinning the loop would otherwise put every run_in_executor compression and every asyncio.to_thread file read on the one core the loop is already saturating. Worker threads are handed the full mask back — offloading exists to get off that core.

Set off on a shared host, under an orchestrator that already does placement, or any time you would rather make this decision yourself. Pinning is Linux-only (sched_setaffinity) and multi-worker-only; a single-worker server is never pinned.

Workers vs cores under connection churn

Start at or below the available physical-core count and tune against your workload. Extra workers contend for CPU and accept queues, especially when clients reconnect frequently. Compare shared listeners and BB_SOCKET_REUSEPORT=1 under that workload before choosing either.

Warm-up before fork

@app.on_warmup runs once before binding listeners and forking workers. Only warm reusable code and data here; open per-worker resources at startup.

@app.on_warmup
async def warm(app):
    from blackbull import Connection, Headers
    conn = Connection(
        method='POST', path='/rpc', raw_path=b'/rpc',
        headers=Headers([(b'content-type', b'application/grpc')]))
    await app.warm_request(conn, body=req_bytes, n=2000)

The hook must therefore warm and nothing else. A DB pool or a socket opened here would be inherited by every worker as the same file descriptor, which is a bug, not a saving; per-worker resources belong in @app.on_startup. Warm-up is best-effort — an exception is logged and the master degrades to a cold start — and the whole budget is capped by BB_WARMUP_BUDGET_S (default 60 s).

In single-worker mode there is no fork, and the one process is warmed before it binds.

uvloop

uvloop is a drop-in libuv-based replacement for the standard asyncio event loop. BlackBull installs it as the loop policy when BB_UVLOOP=1 and the blackbull[speed] extra is present.

pip install 'blackbull[speed]'
BB_UVLOOP=1 python app.py

If BB_UVLOOP=1 is set but uvloop isn't installed, BlackBull logs a warning and falls back to the standard loop — your app keeps running.

Combining both for a production default:

BB_WORKERS=0 BB_UVLOOP=1 python app.py

See Configuration and Reference — Environment variables for the full list of runtime tunables.

Abuse defences

Three independent limits protect a worker from misbehaving or malicious clients. All three apply before the application is reached and are on by default — production deployments should leave them on and only tune the numbers.

Slowloris — partial-headers attack

A slowloris attack keeps a TCP connection open by sending the request header block one byte at a time, indefinitely. Without a deadline, every such connection holds one worker slot forever — a single attacker can exhaust the server's connection pool with no abnormal traffic volume.

BlackBull enforces a deadline on the HTTP/1.1 header read. When the client hasn't delivered the complete header block (request-line + headers + CRLFCRLF) within BB_HEADER_TIMEOUT seconds (default 10), the server answers 408 Request Timeout and closes the socket.

# Default — already on
python app.py

# Tighten to 3 seconds (recommended behind a reverse proxy that already buffers)
BB_HEADER_TIMEOUT=3 python app.py

# Disable (only safe on a trusted local socket)
BB_HEADER_TIMEOUT=0 python app.py

HTTP/2 uses the same BB_HEADER_TIMEOUT for an unfinished HEADERS/CONTINUATION block. A breach closes the connection with GOAWAY(ENHANCE_YOUR_CALM), because HPACK state is shared across streams. Frame lengths do not prevent a peer from stalling mid-frame or mid-block.

Oversized headers — memory exhaustion

Two limits stop a 1 GB X-Foo: <random bytes> header from accumulating in the connection's read buffer while the server waits for a line terminator that may never arrive. Both bound the head read itself, on every request of a connection rather than only the first:

Limit Default Triggered when
BB_HEADER_MAX_LINE 8192 A single header line (or the request line) exceeds this
BB_HEADER_MAX_TOTAL 65536 The full header block exceeds this

Oversized request lines receive 400; oversized fields or an oversized field block receive 431. Both close the HTTP/1.1 connection.

Either way the server discards a bounded amount of what the peer is still sending before it closes, because a close with unread bytes queued makes the kernel send RST and the peer would never see the rejection it was just given.

The defaults match Apache's LimitRequestLine / LimitRequestFieldsize and nginx's large_client_header_buffers.

Connection cap and per-request timeout

Limit Default Behaviour
BB_MAX_CONNECTIONS auto per worker Connections beyond the cap are refused at accept time. Combine with BB_SOCKET_BACKLOG for graceful overload. 0 = unlimited.
BB_REQUEST_TIMEOUT 0 (off) Handler deadline in seconds. HTTP/1.1 answers 408 and closes; HTTP/2 cancels the stream with RST_STREAM CANCEL. Set a positive budget compatible with long-lived requests.

Shutdown

On SIGTERM the master signals every worker and waits before SIGKILL. A worker that receives it stops accepting and lets the requests it already accepted finish, for BB_WORKER_DRAIN_TIMEOUT — which sits inside the master's wait, so the drain ends in the worker rather than in a kill. Raising it past the master's wait only moves the deadline.

Nothing in flight is cancelled while that budget lasts. A cancelled handler means a client holding a half-written response, which is worse than the wait. What is left in the last half second (at most half the budget) is cut: its connection is aborted, discarding output the peer has not read, and its handler is cancelled. A shutdown that must complete still completes, and leaves no connection open.

Watcher stop, worker joins and forced kills consume one master shutdown deadline rather than receiving a fresh timeout each. Listener descriptors are closed after the children are reaped; aliases of one descriptor are disarmed once, while descriptors created with dup() remain separate resources and are each closed.

This is what a rolling deploy or a docker stop depends on. Drain the node at the load balancer first if you want zero in-flight requests at all — the server's job here is to not truncate the ones already running, not to know that your deploy started.

--reload uses the same path, so a code change now recycles workers by draining them rather than by killing them.

SIGTERM ──▶ close listeners ──▶ drain accepted connections
                                        │
                        BB_WORKER_DRAIN_TIMEOUT reached ──▶ abort, cancel
                                        │
                          master's wait exhausted ──▶ SIGKILL

Scaling ceiling — plan against physical cores

Worker throughput scales roughly to the physical core count, not the logical (SMT-thread) count. Adding workers past the physical-core ceiling does not multiply throughput. Plan capacity against nproc / 2 on typical SMT-enabled hardware.

Workers alongside a raw protocol handler

A non-ASGI protocol registered via app.raw_handler() or app.register_protocol_handler() follows a different rule from HTTP, because a stateful broker must have exactly one owner.

The protocol has a single owner; HTTP still scales. The master binds the protocol port once and hands it to worker 0 only, so the broker lives there — while app.run(port=8000, workers=4) alongside, say, MQTTExtension(port=1883) still runs HTTP on all workers. If worker 0 crashes, the master respawns it and it re-inherits the still-open listening socket, so the broker resumes on the same port. Clients reconnect; in-memory broker state does not survive (see MQTT).

Auto-reload is the exception. --reload hands listening sockets across an exec via fd inheritance, and that handoff does not yet include the protocol listeners — so reload=True with a port-bound protocol still forces workers=1. Run without reload to scale HTTP alongside the broker.

SO_REUSEPORT applies to HTTP only. BB_SOCKET_REUSEPORT=1 gives each HTTP worker its own kernel accept queue (best load distribution); without it the workers share the master's listening socket. The protocol port is always bound without SO_REUSEPORT — a single owner is the point.

Production checklist

A reasonable production environment:

BB_WORKERS=0 \
BB_UVLOOP=1 \
BB_HEADER_TIMEOUT=3 \
BB_REQUEST_TIMEOUT=30 \
BB_MAX_CONNECTIONS=1000 \
BB_ACCESS_LOG=1 \
python app.py --cert /etc/ssl/site.pem --key /etc/ssl/site.key

What each line buys you:

  • BB_WORKERS=0 — fills the CPU budget without hand-counting.
  • BB_UVLOOP=1 — use uvloop when the [speed] extra is installed.
  • BB_HEADER_TIMEOUT=3 — slowloris defence; 3 s is plenty behind a buffering reverse proxy.
  • BB_REQUEST_TIMEOUT=30 — cancels stalled HTTP/1.1 and HTTP/2 handlers.
  • BB_MAX_CONNECTIONS=1000 — bounds concurrent connections per worker; size it together with per-connection and per-stream budgets.
  • BB_ACCESS_LOG=1 — leave on unless a separate log aggregator is consuming structured logs.

Equivalent TOML for a config file — see Configuration.

Next