test: add fuzzing for parsers and the numeric kernel#110
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Prepared current head What I changed on top of the fuzzing branch:
Local verification on this head:
Local tool limits:
The previous OpenCode job failed by timing/context timeout, not by a review finding. This push reduces diff size and removes the binary PDF; the new current-head OpenCode and required checks are queued due the active GitHub Actions delay. |
OpenCode Review Overview
Pull request overviewOpenCode reviewed the current-head bounded evidence and found no blocking issues. FindingsNo blocking findings. SummaryApproval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: ci.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: ci.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Test: test_fuzz_properties.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_fuzz_properties.py"]
R3 --> V3["targeted test run"]
|
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/ci.yml, Cargo.lock, crates/mlsirm-core/Cargo.toml, crates/mlsirm-core/tests/proptest_neg_loglik.rs, fuzz/README.md, and 9 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/ci.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: All tests pass, coverage is adequate, and no unresolved review threads exist.
- Head SHA:
60371f95bdbb8836b8c02ba9919a2f220ef25e7a - Workflow run: 29082501771
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: ci.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: ci.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Changed file (17 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (17 files)"]
R2 --> V2["required checks"]
Evidence --> S3["Test: test_fuzz_properties.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_fuzz_properties.py"]
R3 --> V3["targeted test run"]
|
Current-head update for PR #110:
Remaining blocker, if GitHub still shows blocked: org/repo native branch ruleset approvals are still required ( |
|
@copilot resolve the merge conflicts on this branch. |
Head branch was pushed to by a user without write access
Resolved by merging |
Add coverage-guided and property-based fuzzing for the untrusted-input surfaces surfaced with CodeGraph blast-radius exploration: - load_factor_csv (io.py) — CSV file parser - render_diagnostics_report (report.py) — arbitrary JSON -> HTML renderer - MLS2PLMConfig / FitConfig .validate() — request/DTO validators - neg_loglik_and_grad (mlsirm-core) — core numeric kernel Harnesses (all permissive tools): - Atheris (Apache-2.0) coverage-guided targets in fuzz/atheris/ with seed corpora; contract = no crash / only benign exceptions, plus an HTML-escaping invariant on the report renderer. - Hypothesis (MPL-2.0) property tests in tests/test_fuzz_properties.py that run in the existing pytest suite. - proptest (MIT/Apache-2.0) harness in crates/mlsirm-core/tests/ that runs under cargo test --workspace; asserts gradient shapes + finiteness invariants. Wire a bounded CI `fuzz` job (~60s/target) so PR cost stays low; add the `fuzz` extra and Hypothesis to the dev extra. Attach a coverage-guided fuzzing survey paper under docs/papers/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
Resolve OpenSSF Scorecard pinned-dependencies code-scanning alerts on the CI workflow: - Pin actions/checkout to v4.3.1 (34e114876b0b) by commit SHA. - Pin actions/setup-python to v5.6.0 (a26af69be951) by commit SHA. - Pin pip to an exact version (pip==26.1.2) before resolving the fuzz extras so the install toolchain is reproducible. The editable install (pip install -e .[fuzz]) is the repo-under-test and cannot be hash-pinned (pip forbids --require-hashes with editable/local installs); its extras are permissive dev-only tooling (atheris, hypothesis). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RTAMs4bpSZS77Xe3RQjv9P
seonghobae
force-pushed
the
feat/add-fuzzing
branch
from
a72bc93 to
14b5989
Compare
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .Jules/palette.md, .github/workflows/ci.yml, Cargo.lock, crates/mlsirm-core/Cargo.toml, crates/mlsirm-core/tests/proptest_neg_loglik.rs, and 10 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .Jules/palette.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: No blocking issues found in the PR adds fuzzing for parsers and the numeric kernel with comprehensive test coverage.
- Head SHA:
14b5989cb376936c0650c3867eab2819e55ad5ab - Workflow run: 29102352023
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: ci.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: ci.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Test: test_fuzz_properties.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_fuzz_properties.py"]
R3 --> V3["targeted test run"]
Summary
Adds fuzzing across the highest-value untrusted-input surfaces of fast-mlsirm. Targets were selected with CodeGraph by ranking blast radius on parser / loader / validator nodes:
load_factor_csvpython/fast_mlsirm/io.pyrender_diagnostics_reportpython/fast_mlsirm/report.pyMLS2PLMConfig/FitConfig.validate()python/fast_mlsirm/config.pyneg_loglik_and_gradcrates/mlsirm-core/src/lib.rsWhat changed
fuzz/atheris/with seed corpora infuzz/corpus/.tests/test_fuzz_properties.pythat run in the normal pytest suite when the dev extra is installed.crates/mlsirm-core/tests/proptest_neg_loglik.rs, run bycargo test --workspace.fuzzjob with short per-target budgets.License notes
Verification
pytestpreviously passed in GitHub CI.cargo test --workspacepreviously passed in GitHub CI.py -m pytest tests/test_fuzz_properties.py-> 4 passedpy -m pytest tests-> 132 passed, 3 skippedNot merged; waiting behind lower-numbered PRs and current-head OpenCode approval.