← all writings

The disk was never the problem

How I spent an evening moving 12GB of Docker images around a Raspberry Pi to fix a problem that turned out to be RAM.

linuxraspberry-pidebugging

My Pi felt slow. I had just moved Docker’s data root onto an external USB drive to free up space on a full SD card, and everything got sluggish immediately afterward. Obvious conclusion: USB 2.0 on a Pi 3B is slow, the container images are on USB now, so the disk is the bottleneck. Move it back.

That conclusion was wrong, and the reason it was wrong is more interesting than the fix.

What I actually measured

Before undoing the move, I checked memory:

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           906Mi       486Mi        22Mi        66Mi       496Mi       420Mi
Swap:          1.0Gi       692Mi       331Mi

22MB free. 692MB of swap in use, out of 1GB. The box wasn’t waiting on the USB drive — it was thrashing, constantly paging memory out to disk and pulling it back in.

And here’s the part that makes the “move Docker back” plan actively counterproductive: the swapfile lived on the SD card. Every swap hit was already hammering the exact storage I was about to move 12GB of images onto.

The actual cause

I’d added one container too many. Stopping it:

$ docker stop ersatztv
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           906Mi       415Mi       133Mi       756Ki       374Mi       491Mi
Swap:          1.0Gi       472Mi       551Mi

Free memory went from 22MB to 133MB, swap dropped by 220MB. One .NET application was the difference between a responsive machine and a thrashing one.

Two things I took from it

Stale swap doesn’t clear itself. Freeing memory doesn’t automatically pull swapped pages back into RAM — they sit there being read slowly until something touches them. swapoff -a && swapon -a forces the issue, and it’s a ten-second fix versus an hour of moving files.

“It got slow right after I changed X” is a hypothesis, not a diagnosis. The timing was genuinely suspicious and the story was plausible. It was still wrong. Two commands would have told me that before I started planning a migration.

The disk move I’d made was fine. The machine was just out of memory, and no amount of rearranging files fixes being 500MB short.