Problem / impact
clawbench-reproduce --work-dir <existing-directory> recursively deletes the entire supplied directory after comparison unless --keep-cache is supplied. The directory need not have been created by this invocation, so unrelated user files are removed along with downloaded traces.
Source: reproduce.py. The code calls shutil.rmtree(args.work_dir, ignore_errors=True) without tracking ownership. mkdir(..., exist_ok=True) earlier accepts an existing directory.
Verified reproduction (offline, temporary directory only)
- Create a temporary
user-work/ with unrelated-user-file.txt and downloaded-traces/.
- Mock
download to return downloaded-traces/ and rescore to return a matching 129-task summary; invoke main with --model deepseek-v4-flash --work-dir user-work.
- Actual: exit 0; both the unrelated file and
user-work/ no longer exist.
No real data, downloads, or judge API calls were used for verification.
Acceptance criteria
Suggested priority P1: user data loss. Separate from #296 / #313 (rescore path defaults).
@Perry2004 please review the cleanup ownership contract and prioritize this fix.
Problem / impact
clawbench-reproduce --work-dir <existing-directory>recursively deletes the entire supplied directory after comparison unless--keep-cacheis supplied. The directory need not have been created by this invocation, so unrelated user files are removed along with downloaded traces.Source: reproduce.py. The code calls
shutil.rmtree(args.work_dir, ignore_errors=True)without tracking ownership.mkdir(..., exist_ok=True)earlier accepts an existing directory.Verified reproduction (offline, temporary directory only)
user-work/withunrelated-user-file.txtanddownloaded-traces/.downloadto returndownloaded-traces/andrescoreto return a matching 129-task summary; invokemainwith--model deepseek-v4-flash --work-dir user-work.user-work/no longer exist.No real data, downloads, or judge API calls were used for verification.
Acceptance criteria
--keep-cachebehavior explicit, and report the exact owned cache path.Suggested priority P1: user data loss. Separate from #296 / #313 (rescore path defaults).
@Perry2004 please review the cleanup ownership contract and prioritize this fix.