Skip to content

fix: preserve HDFS backward file lists - #641

Merged
njzjz merged 1 commit into
deepmodeling:masterfrom
njzjz:fix/issue-615-hdfs-download-lists
Aug 29, 2026
Merged

fix: preserve HDFS backward file lists#641
njzjz merged 1 commit into
deepmodeling:masterfrom
njzjz:fix/issue-615-hdfs-download-lists

Conversation

@njzjz

@njzjz njzjz commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

  • copy task and submission backward-file lists before adding discovered error artifacts
  • keep generated error paths relative to their task or common download roots
  • prevent repeated downloads from mutating Submission configuration or targeting stale absolute paths

Closes #615

Validation

  • full unit suite: 164 passed, 42 skipped
  • focused HDFS regression tests passed
  • pre-commit and Ruff checks passed
  • exact CI ty==0.0.17 check passed
  • CLI smoke checks passed

Repository-wide Pyright retains unrelated baseline diagnostics.

Coding agent: Codex
Codex version: codex-cli 0.149.0
Model: gpt-5.6-sol
Reasoning effort: xhigh

Copy backward file lists and keep discovered error artifacts relative to their download roots.

Coding-Agent: Codex
Codex-Version: codex-cli 0.144.6
Model: gpt-5.6-sol
Reasoning-Effort: xhigh
@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. bug Something isn't working labels Aug 23, 2026
@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 18 seconds.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 38db7492-6714-4527-8683-3cd9b78fca61

📥 Commits

Reviewing files that changed from the base of the PR and between 34ddb4b and d212a7d.

📒 Files selected for processing (2)
  • dpdispatcher/contexts/hdfs_context.py
  • tests/test_hdfs_context.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.31%. Comparing base (f547911) to head (d212a7d).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #641      +/-   ##
==========================================
+ Coverage   48.38%   49.31%   +0.93%     
==========================================
  Files          40       40              
  Lines        3960     3960              
==========================================
+ Hits         1916     1953      +37     
+ Misses       2044     2007      -37     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@njzjz-bot njzjz-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings after reviewing the full diff, related code, and CI checks.

Coding agent: Codex
Codex version: codex-cli 0.151.0
Model: gpt-5.6-sol
Reasoning effort: xhigh

@njzjz-bot njzjz-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete diff and relevant surrounding code. No blocking findings. The relevant CI checks are passing, and targeted tests were run additionally for higher-risk changes where warranted.

Coding agent: Codex
Codex version: codex-cli 0.151.0
Model: gpt-5.6-sol
Reasoning effort: xhigh

@njzjz-bot njzjz-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review completed. I found no blocking issues in the changed behavior, compatibility, or test coverage.

Coding agent: Codex
Codex version: codex-cli 0.151.0
Model: gpt-5.6-sol
Reasoning effort: xhigh

@njzjz
njzjz merged commit c744908 into deepmodeling:master Aug 29, 2026
31 checks passed
@njzjz
njzjz deleted the fix/issue-615-hdfs-download-lists branch August 29, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Code scan] Avoid mutating HDFS backward file lists during download

2 participants