Zero-Junk Hosting

Make every import job clean up after itself.

For each job that downloads or generates raw data ({{RAW_DATA}}):

1. Give the job its own working folder under one dedicated temporary root (outside the web root), named with the job name and start time. Each job holds an exclusive lock while it runs; when it starts, it first removes every earlier working folder of its own, whatever its age.
2. Stream large downloads to disk and process them in chunks; never load a whole large file into memory.
3. Keep only the final processed result the site reads. Raw files, extracted archives and intermediate files are deleted when the job ends.
4. Cleanup runs in a finally block (or shutdown handler), so it also happens after an error or a timeout.
5. If a job fails, keep at most ONE small sample of the failing input (first {{SAMPLE_KB — e.g. 64}} KB) for debugging, at _samples/<job>.sample under the temporary root, overwriting the previous sample.
6. Before starting, check free space. On shared hosting the server's free disk says nothing about my plan, so free space means my plan limit minus the use measured for my account. If less than {{MIN_FREE_MB — e.g. 500}} MB is free, do not start; record why.
7. If the site stores images: store one optimised size set per image, de-duplicate identical files by content hash, and delete images of items that no longer exist.

Add an orphan sweep at the start of each job: any working folder left by OTHER jobs under the temporary root and older than {{ORPHAN_HOURS — e.g. 6}} hours is removed. The sweep skips the _samples folder. The sweep must be unable to touch anything outside that temporary root: resolve the real path and refuse if it is not inside it.
…