fix(security): avoid shell interpolation in HDFS and SGE

[njzjz-bot]

Summary

• run every Hadoop filesystem operation with an argv list and shell=False
• preserve Hadoop-side wildcard expansion while keeping spaces and shell metacharacters inside single arguments
• reject empty, NUL-containing, non-string, and option-like HDFS operands
• quote SGE remote paths and prefix relative paths so cd and qsub cannot treat them as options
• add isolated HDFS and SGE command-construction tests

Validation

• python -m coverage run -p --source=./dpdispatcher -m unittest: 167 passed, 42 skipped
• ruff check .: passed
• ruff format --check .: passed
• focused HDFS/SGE tests: 4 passed
• CLI smoke tests: passed
• ty check: reports four pre-existing allow_ref/dargs diagnostics in unchanged files
• full pre-commit initialization was attempted but the external Prettier/Node download stalled; the documented Ruff fallback passed

Closes #605

Coding agent: Codex
Codex version: codex-cli 0.144.4
Model: gpt-5.6-sol
Reasoning effort: xhigh

[coderabbitai]

Warning

Review limit reached

@njzjz-bot, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 39 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bd6afba9-771b-4652-9084-a85241cdb572

📥 Commits

Reviewing files that changed from the base of the PR and between e23bb8c and da5ad88.

📒 Files selected for processing (3)

• dpdispatcher/machines/pbs.py
• dpdispatcher/utils/hdfs_cli.py
• tests/test_hdfs_cli.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 78.48101% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.26%. Comparing base (e23bb8c) to head (da5ad88).

Files with missing lines	Patch %	Lines
dpdispatcher/utils/hdfs_cli.py	77.63%	17 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #621      +/-   ##
==========================================
+ Coverage   48.38%   50.26%   +1.87%     
==========================================
  Files          40       40              
  Lines        3960     3997      +37     
==========================================
+ Hits         1916     2009      +93     
+ Misses       2044     1988      -56     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[njzjz-bot]

Reviewed the complete HDFS argv migration and SGE submission quoting. Hadoop receives literal argv with shell=False while retaining its own HDFS glob handling; SGE paths are quoted and protected from option parsing. The full Python/platform, pyright, pre-commit, docs, and backend-build matrix is green. I found no blocking issue.

This PR is authored by the reviewing account, so this is a comment review rather than a self-approval.

Review context: Codex's quota is close to resetting, so I'm deliberately spending the remaining tokens on reviewing the open PR queue now.

Coding agent: Codex
Codex version: codex-cli 0.144.6
Model: gpt-5.6-sol
Reasoning effort: xhigh

[njzjz-bot]

No blocking findings after reviewing the full diff, related code, and CI checks. Intended decision: APPROVE. GitHub does not permit njzjz-bot to approve a pull request authored by the same account, so this formal review is submitted as COMMENT.

Coding agent: Codex
Codex version: codex-cli 0.151.0
Model: gpt-5.6-sol
Reasoning effort: xhigh

[njzjz-bot]

Reviewed the complete diff and relevant surrounding code. No blocking findings. GitHub does not permit the njzjz-bot account to approve a pull request authored by njzjz-bot, so this review is submitted with the COMMENT event only.

Coding agent: Codex
Codex version: codex-cli 0.151.0
Model: gpt-5.6-sol
Reasoning effort: xhigh

[njzjz-bot]

Independent review result: no blocking issues found. GitHub prevents njzjz-bot from approving a pull request authored by the same account, so this formal review is submitted as COMMENT rather than APPROVE.

Coding agent: Codex
Codex version: codex-cli 0.151.0
Model: gpt-5.6-sol
Reasoning effort: xhigh