Docker build fails with “no space left on device” because BuildKit cache fills /var/lib/docker

2026-11-13 · 3 min read

Docker build fails with “no space left on device” because BuildKit cache fills /var/lib/docker

A CI build dies near the end with no space left on device; retries fail faster because the Docker host is still full. The immediate cost is a blocked pipeline, but the bigger cost is noisy disk alerts and cold rebuilds after someone clears cache blindly. If df -h /var/lib/docker is at 100%, treat it as a Docker BuildKit no space left on device incident, not a generic “clean up later” ticket.

Check the cache before deleting anything

Confirm the filesystem and the consumer:


df -h /var/lib/docker
docker system df -v
docker buildx du
sudo du -sh /var/lib/docker/buildkit 2>/dev/null

BuildKit cache normally lives under Docker’s data root, commonly /var/lib/docker/buildkit [link needed]. docker system df -v shows reclaimable space by category; docker buildx du shows BuildKit cache records. If the cache is small but images or volumes are large, pruning cache will not fix the incident.

Clear the cache with a controlled procedure

Run this on the affected Docker host. If multiple agents share a remote builder, run it on the builder host, not the CI worker.

1. Pause new builds or drain the CI runner. Pruning during active builds can remove cache layers another job is using [link needed].

2. Capture evidence: df -h /var/lib/docker, docker system df -v, and docker buildx du > /tmp/buildx-du.txt.

3. Prune stale cache first: docker builder prune --filter until=168h --force.

4. If the host is still full and no builds are running, clear all build cache: docker builder prune --all --force.

5. Recheck with df -h /var/lib/docker and docker buildx du. If space did not recover, inspect images, containers, and volumes with docker system df -v.

Avoid deleting /var/lib/docker/buildkit by hand unless the daemon is stopped and you have verified the path [link needed]. docker builder prune only removes build cache; it does not remove images or volumes [link needed].

Stop the same Docker BuildKit no space left on device failure recurring

Schedule retention instead of waiting for a full disk. A conservative cron entry for a busy builder:


15 3 * * * /usr/bin/docker builder prune --filter until=168h --force >/var/log/docker-builder-prune.log 2>&1

Use --keep-storage if you want a floor:


docker builder prune --keep-storage 40GB --force

Check the flag against your Docker version with docker builder prune --help [link needed]. Also monitor /var/lib/docker at 80% and 90%. If the volume is undersized, move Docker’s data root to a larger disk or add a dedicated builder; cache tuning only buys time.

When NOT to do this

Do not run --all during active builds on a shared builder unless you accept cold rebuilds. Do not prune on the CI worker if builds use a remote BuildKit endpoint. Do not clear cache first when the actual consumer is old images, stopped containers, or volumes. Do not put an aggressive prune on a low-traffic builder where warm cache saves most of the build time.

Checklist