The file existed. That was the problem.

An audio pipeline was generating long tracks, copying them between stages, and feeding them into another scanner. The scanner watched a directory and did what scanners do: it saw a new filename and assumed there was a new file to process.

Except the producer was still writing it.

The scanner opened a half-finished track, read whatever bytes happened to exist at that moment, and produced a failure that looked like corrupted audio. The source was not corrupt. The source was unfinished.

Existence is a weak signal

Filesystems make partial work visible unless we deliberately hide it.

A process can create a destination path, write for several minutes, flush buffers, update metadata, and close the handle. During most of that time, other programs can see the name. A naive watcher treats the first directory event as a completion event.

Those are not the same event.

The safe pattern is boring and effective:

  1. Write to a clearly temporary name such as chapter-08.mp3.partial.
  2. Flush and close it.
  3. Validate size, duration, checksum, or format.
  4. Rename it to chapter-08.mp3 on the same filesystem, where a rename is atomic.
  5. Let downstream scanners ignore every .partial file.
chapter-08.mp3.partial  -> invisible to consumers
          validate
          rename
chapter-08.mp3          -> finished contract

The rename is the receipt.

Stable size is helpful, not sufficient

Another common fix is to wait until a file’s size stops changing. That helps, but it is not proof.

A producer may pause while waiting on a network, encoder, or API. The file can remain the same size for thirty seconds and then continue. Conversely, buffered writes can make size updates arrive in bursts. “Stable for a moment” is a heuristic.

Stronger completion signals include:

  • producer-controlled atomic rename (same volume, not a cross-disk copy);
  • a companion manifest written after validation;
  • expected byte length and checksum;
  • a media probe that decodes cleanly through the expected duration;
  • a database status transition committed after the file closes.

The producer knows when it is done. Make it say so.

Consumers should still be suspicious

Even a clean handoff can encounter interrupted copies, disk-full conditions, or older software that ignores the convention. Downstream jobs should reject obviously partial input and record why.

For audio, that may mean probing duration and codec before import. For archives, it means listing and testing entries. For downloads, it means checking expected length and hash. For documents, it may mean opening the container rather than trusting the extension.

The rule scales far beyond media. Database dumps, camera exports, backups, reports, and generated websites all suffer when a path is mistaken for a promise.

The fix is to define what “finished” means in a way another process can test.

A file with a final-looking name is evidence that someone started. A validated atomic handoff is evidence that they finished.