SQLite’s WAL mode delivers reliable concurrent access across Docker containers sharing a volume on the same host. Two containers—one writing, the other reading—propagate changes in real time without corruption or locks. This works because containers leverage the host’s kernel and filesystem, sharing memory-mapped files for WAL coordination.
The test, run on Docker Desktop for macOS with a named volume, confirms what a Hacker News thread questioned: WAL’s shared memory (-shm file) functions across container boundaries. Skeptics worried about isolation breaking SQLite’s concurrency model. They were wrong here—everything syncs as expected.
The Experiment Setup
Researchers spun up two Alpine Linux containers using a shared named volume. Container A initializes a database and inserts records in a loop:
docker volume create sqlite-test
docker run -d --name writer -v sqlite-test:/data alpine sh -c '
apk add sqlite &&
sqlite3 /data/test.db "PRAGMA journal_mode=WAL;" &&
while true; do
sqlite3 /data/test.db "INSERT INTO t VALUES (datetime(), randomblob(100));";
sleep 1;
done'
Container B polls the count:
docker run -d --name reader -v sqlite-test:/data alpine sh -c '
apk add sqlite &&
while true; do
sqlite3 /data/test.db "SELECT count(*) FROM t;" &&
sleep 1;
done'
Logs from both show counts climbing together. No deadlocks, no rollbacks. They monitored the test.db-shm file: its size grew to 32KB (SQLite’s fixed shm size in WAL), and hex dumps revealed identical contents across containers, proving shared mapping.
Why WAL Succeeds Here
SQLite WAL decouples writers from readers. Writers append to .db-wal; a shared memory file (.db-shm) tracks commit points via memory-mapped pages. Containers on the same host share the Linux kernel’s VFS and tmpfs-like shm. Docker’s namespaces isolate processes but not the underlying filesystem or kernel memory mappings for volume files.
On Docker Desktop macOS, a Linux VM handles this transparently—shm maps propagate. Native Linux hosts behave identically. Key fact: WAL limits writers to one at a time anyway, checkpointing every 1000 pages or 1000KB WAL growth. Multiple readers scale fine, hitting 10k+ TPS in benchmarks.
This matters for microservices or edge computing. SQLite edges out heavier databases like PostgreSQL for low-latency, single-host clusters. A 2023 Fly.io benchmark showed WAL SQLite sustaining 50k reads/sec across threads—extend that to containers, and you get cheap concurrency without sharding.
Caveats and Real-World Implications
It fails across hosts. NFS or cloud volumes introduce latency; shm doesn’t share. Kubernetes PersistentVolumes work if pods colocate on one node, but node failover corrupts WAL—use READ replicas or Litestream for replication.
Test on bare Linux: confirmed identical results. Docker Compose yaml simplifies:
version: '3'
volumes:
db:
services:
writer:
image: alpine
volumes: [db:/data]
command: sh -c "apk add sqlite && sqlite3 /data/test.db 'PRAGMA journal_mode=WAL; CREATE TABLE IF NOT EXISTS t(id INTEGER PRIMARY KEY, data BLOB);' && while true; do sqlite3 /data/test.db 'INSERT INTO t (data) VALUES (randomblob(100));'; sleep 1; done"
reader:
image: alpine
volumes: [db:/data]
command: sh -c "apk add sqlite && while true; do echo $(sqlite3 /data/test.db 'SELECT count(*) FROM t'); sleep 1; done"
Why care? Developers waste cycles on “SQLite isn’t for servers.” This debunks it for containerized, single-host setups—think IoT gateways, serverless backends, or crypto nodes logging trades. Pair with Turso or EdgeDB for multi-host, but for local stacks, WAL + Docker volumes deliver production-grade reliability at zero cost.
Bottom line: Use it. Monitor WAL size (PRAGMA wal_checkpoint(FULL); periodically) and vacuum. SQLite powers 1T+ devices; containers just unlock more.
