libeatmydata tricks applications into skipping fsync calls, dramatically speeding up disk writes at the cost of crash safety. This LD_PRELOAD library intercepts libc functions like fsync, fdatasync, and sync_file_range, replacing them with no-ops. Developers use it for testing where speed matters more than data integrity—think CI pipelines or benchmarks. But production? Forget it. One power failure, and your data vanishes.
Why does this matter? Modern databases and apps hammer fsync to ensure writes hit stable storage, preventing corruption. Without it, writes queue in RAM or kernel buffers. A crash discards them. In tests, it slashes runtimes: MySQL’s innodb suite drops from 78 seconds to 25 seconds per test case. Total test time falls from 3m36s to 2m10s—a 40% cut. That’s real, measured on real hardware, not hype.
Performance Breakdown
Stewart Smith’s benchmarks on mysql-test-run highlight the gains. Here’s the raw data for key innodb tests:
- main.innodb: 78,306 ms → 24,900 ms (68% faster)
- main.innodb_mysql: 86,810 ms → 68,579 ms (21% faster)
- main.innodb-ucs2: 5,786 ms → 1,084 ms (81% faster)
- Average across 9 tests: 182.97s execution → 104.23s (43% faster)
These aren’t cherry-picked; all passed identically. The speedup comes from ditching synchronous I/O flushes. On SSDs or fast disks, gains shrink, but mechanical drives or I/O-heavy workloads still benefit hugely in non-critical scenarios.
Installation and Usage
Grab it from GitHub: git clone https://github.com/stewartsmith/libeatmydata.git. Most distros package it—search for “eatmydata” on Debian/Ubuntu/Fedora. From source:
cd libeatmydata autoreconf -i # Git only ./configure make make check sudo make install
Run via the wrapper: eatmydata your-command. It sets LD_PRELOAD=/usr/lib/libeatmydata.so and execs. Works on Linux primarily; reports exist for macOS, Solaris, and kFreeBSD. Binaries available for Fedora 33 (x86_64, ppc64le), signed with Smith’s GPG key.
It targets specific calls: fsync(2), fdatasync(2), sync_file_range(2), syncfs(2), and open(O_SYNC/O_DSYNC). No interference with other I/O. Patches for more platforms welcome via email or GitHub.
Risks and Real-World Context
This tool screams “use at your own peril.” The name—libeatmydata—warns you: it devours your data safety. In development, it shines for repeatable tests without fsync bottlenecks. MySQL devs used it historically; similar tricks appear in PostgreSQL’s testing suites. But skeptically: benchmarks assume perfect conditions. Real crashes hit mid-write, corrupting indexes or WAL logs irreversibly.
Alternatives exist. Tune your filesystem (e.g., XFS with nobarrier), use tmpfs for temp data, or SSDs with power-loss protection. For CI, Docker images often preload it by default for speed. Still, measure your workload—profile with strace -c to count fsyncs first.
Bottom line: libeatmydata exposes the fsync tax. It proves how much safety costs in I/O time. Use it to accelerate tests, benchmark baselines, or debug I/O paths. Never on live data. In a world of flaky power and cosmic rays, data durability isn’t optional—it’s the price of reliability. At 131 releases strong since 2011, it’s battle-tested for its niche.
