Skip to content

feat(dfget): skip existing output files by default when overwrite is disabled#1846

Open
NelZyhh wants to merge 2 commits into
dragonflyoss:mainfrom
NelZyhh:feature/add-skipIfExists
Open

feat(dfget): skip existing output files by default when overwrite is disabled#1846
NelZyhh wants to merge 2 commits into
dragonflyoss:mainfrom
NelZyhh:feature/add-skipIfExists

Conversation

@NelZyhh
Copy link
Copy Markdown
Contributor

@NelZyhh NelZyhh commented May 19, 2026

Background / Motivation

When running dfget repeatedly in batch jobs, retry workflows, or model synchronization tasks,
users often expect idempotent behavior when the output file already exists.

Previously, dfget had only two behaviors for existing output files:

  • overwrite the file when --overwrite is enabled
  • return an error when --overwrite is not enabled

Based on review feedback, there is no need to introduce a new CLI option for this case.
Instead, when --overwrite=false, dfget should skip existing files by default and return success.

What’s in this PR

Update dfget existing-file behavior without adding a new option:

  • --overwrite=true: overwrite the existing output file
  • --overwrite=false (default): skip the download and return success if the output file already exists

Additional changes:

  • remove the previously introduced --skip-if-exists option
  • update argument help text to reflect the new default behavior
  • apply the skip behavior before starting a single-file download
  • make recursive downloads skip existing files file-by-file through the same logic
  • update unit tests accordingly

Usage

Skip existing files by default:

dfget modelscope://Qwen/Qwen3-8B -O ./Qwen3-8B -r

Overwrite existing files explicitly:

dfget modelscope://Qwen/Qwen3-8B -O ./Qwen3-8B -r --overwrite

Compatibility

  • this change modifies the default behavior when the output file already exists
  • previously, --overwrite=false returned an error
  • now, --overwrite=false skips the existing file and returns success
  • --overwrite=true behavior remains unchanged

Testing

Local validation:

cargo fmt --all -- --check
cargo clippy --all --all-targets -- -D warnings
cargo test -p dragonfly-client --bin dfget

Runtime verification:

  • download to an existing output path without --overwrite
  • confirm the command exits successfully and does not overwrite the file
  • download to the same path with --overwrite
  • confirm the existing file is overwritten

@codecov
Copy link
Copy Markdown

codecov Bot commented May 19, 2026

Codecov Report

❌ Patch coverage is 83.33333% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.20%. Comparing base (667c327) to head (9c90d87).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
dragonfly-client/src/bin/dfget/main.rs 83.33% 7 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1846   +/-   ##
=======================================
  Coverage   44.20%   44.20%           
=======================================
  Files          93       93           
  Lines       27229    27256   +27     
=======================================
+ Hits        12036    12049   +13     
- Misses      15193    15207   +14     
Files with missing lines Coverage Δ
dragonfly-client/src/bin/dfget/main.rs 50.10% <83.33%> (+0.48%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: zhangyh66 <zhangyh66@chinatelecom.cn>
@NelZyhh NelZyhh force-pushed the feature/add-skipIfExists branch from f374806 to c70ce9c Compare May 19, 2026 09:50
@gaius-qi gaius-qi added the enhancement New feature or request label May 19, 2026
Copy link
Copy Markdown
Member

@gaius-qi gaius-qi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to add an option parameter. We can just make "skip if exists" the default behavior.

@NelZyhh NelZyhh changed the title feat(dfget): add --skip-if-exists option for existing output files feat(dfget): skip existing output files by default when overwrite is disabled May 19, 2026
@NelZyhh NelZyhh force-pushed the feature/add-skipIfExists branch from b0600e7 to 20250ac Compare May 19, 2026 11:36
…disabled

Signed-off-by: zhangyh66 <zhangyh66@chinatelecom.cn>
@NelZyhh NelZyhh force-pushed the feature/add-skipIfExists branch from 20250ac to 9c90d87 Compare May 19, 2026 11:58
@NelZyhh NelZyhh requested a review from gaius-qi May 19, 2026 12:01
@gaius-qi gaius-qi added this to the v2.5.0 milestone May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants