Skip to content

feat(php): consolidated PHP tooling (php, artisan, phpunit, phpstan, pest, paratest, ecs, pint)#1649

Merged
KuSh merged 20 commits into
rtk-ai:developfrom
iliaal:feat/php-tooling
Jul 1, 2026
Merged

feat(php): consolidated PHP tooling (php, artisan, phpunit, phpstan, pest, paratest, ecs, pint)#1649
KuSh merged 20 commits into
rtk-ai:developfrom
iliaal:feat/php-tooling

Conversation

@iliaal

@iliaal iliaal commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Consolidates the open and closed PHP-tooling work into a single PR against develop, omitting .phpt (which has its own PR #1503).

This PR adds eight rtk subcommands for PHP test runners and code tooling: php, artisan, phpunit, phpstan, pest, paratest, ecs, pint. Output compression in the 60% to 95% range per tool, structured parsers where possible (PHPUnit state machine, PHPStan typed JSON, Pint per-file rule counts).

Credits to original authors

  • @aaronflorey (rtk-ai/rtk#1246, self-closed 2026-04-26): php, artisan, ecs, pest, paratest, test_output, utils, composer_bin_dirs, registry normalization for Composer custom-bin-dir layouts.
  • @Beninho (rtk-ai/rtk#874, open): phpunit state-machine parser. I ported it into the consolidated module, switched it to runner::run_filtered, and stripped emoji from output.
  • @LucianoVandi (rtk-ai/rtk#1110, open): phpstan typed serde::Deserialize parser for --error-format=json. Groups errors by file, sorts by count descending; passes utility commands (--version, list, clear-result-cache) through unchanged. I stripped emoji from the success output.
  • New work: pint (Laravel Pint code-style fixer) using --format=json for structured per-file rule counts.

If the maintainers prefer this consolidation, close the three separate PRs in favor of this one. If they prefer the originals, close this PR and let the per-tool PRs continue independently. Either is fine with me.

Composer custom-bin-dir support

composer_bin_dirs() reads COMPOSER_BIN_DIR and composer.json's config.bin-dir, so tools/bin/phpunit classifies identically to vendor/bin/phpunit. registry.rs normalizes tool paths before matching, so a single rule covers the standard Composer layouts.

@pszymkowiak pszymkowiak added effort-large Plusieurs jours, nouveau module enhancement New feature or request labels Apr 30, 2026
@pszymkowiak

Copy link
Copy Markdown
Collaborator

[w] wshm · Automated triage by AI

📊 Automated PR Analysis

Type feature
🟡 Risk medium

Summary

Adds eight new rtk subcommands for PHP tooling (php, artisan, phpunit, phpstan, pest, paratest, ecs, pint) with structured output parsers and compression. Consolidates work from three prior PRs by different authors into a single module, including shared test output filtering, Composer custom-bin-dir support, and registry normalization.

Review Checklist

  • Tests present
  • Breaking change
  • Docs updated

Linked issues: #1503, #1246, #874, #1110


Analyzed automatically by wshm · This is an automated analysis, not a human review.

@pierresh

pierresh commented May 1, 2026

Copy link
Copy Markdown

Hi,
Thanks a lot for this consolidated PHP tooling. I tested it, and it works well. Just 2 suggestions: to add Behat and Rector as they are both popular tools in the PHP world.

@iliaal

iliaal commented May 2, 2026

Copy link
Copy Markdown
Contributor Author

Hi, Thanks a lot for this consolidated PHP tooling. I tested it, and it works well. Just 2 suggestions: to add Behat and Rector as they are both popular tools in the PHP world.

Can be added maybe as a follow-up, PR is already quite big, but 100% a good idea.

@iliaal

iliaal commented May 3, 2026

Copy link
Copy Markdown
Contributor Author

Added f9e139a on top of this branch: swaps the make filter from max_lines = 50 (head-only) to head_lines = 10 + tail_lines = 40, so make test runs (e.g. PHPT, which puts ~17000 progress lines before the pass/fail summary) keep both the prologue and the trailing summary instead of dropping the latter.

Companion fix at the recovery layer: #1696 — same head-bias bug in write_tee_file, where the tee log capped at 1MB by keeping raw[..1MB]. With both landed, tail-heavy output survives at the filter and at the fallback log.

@iliaal iliaal force-pushed the feat/php-tooling branch from f9e139a to a944865 Compare May 17, 2026 15:58
@iliaal iliaal force-pushed the feat/php-tooling branch from a944865 to 6ba52ae Compare May 30, 2026 01:38
@webagil-kevin

Copy link
Copy Markdown

Tested this branch on a production Symfony monorepo (27 packages, PHP 8.4/8.5, Pest 4.6.3 + PHPStan level 9), built from source and wired through the Claude Code PreToolUse hook.

Works well across the toolchain:

  • rtk pest / rtk phpunit — failures + summary only, drops the passing noise while preserving diffs and file:line
  • rtk phpstan — compact file:line errors instead of the full table + preamble
  • rtk pint, rtk php artisan test — resolve and filter as expected
  • vendor/bin/* rewrites and php artisan* routing all behave correctly
  • cargo test is green on the branch (2035 passed)

Coverage matches our PHP stack exactly and the filtered output stays faithful to each tool's native format. Would love to see this land — happy to help test further. Thanks for the thorough work here.

@EliW

EliW commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Really looking forward to this landing as part of rtk. It's needed to get real benefits from rtk for those of us on PHP monoliths at the moment :)

Pretty please?

@iliaal iliaal force-pushed the feat/php-tooling branch from 6ba52ae to 16c12bc Compare June 9, 2026 15:26
@KuSh

KuSh commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Hi @iliaal,

Since you're aggregating contributions from others in this PR, please ensure you include the "Co-authored-by:" trailer in the commit message to properly credit them.

Example:

Co-authored-by: Name <email@example.com>

This helps maintain proper attribution for all contributors. Thanks!

@CLAassistant

CLAassistant commented Jun 18, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@iliaal iliaal closed this Jun 18, 2026
@iliaal iliaal deleted the feat/php-tooling branch June 18, 2026 16:16
@iliaal iliaal restored the feat/php-tooling branch June 18, 2026 16:21
@iliaal iliaal reopened this Jun 18, 2026
@evaldnet

Copy link
Copy Markdown

Tried this branch locally against current PHP tool versions (Pint 1.27.1, PHPStan 2.1.40, ECS 13.0.4) — great work, the savings are substantial. I hit two parser bugs where the expected schema no longer matches what current tooling emits. Both stem from test fixtures using an older schema, so the suite stays green. Details + fixes below.

1. pint — JSON field names changed (name/appliedFixerspath/fixers)

PintFile requires name + appliedFixers, but Pint ≥ ~1.14 (confirmed on 1.27.1) emits:

{"result":"fail","files":[{"path":"app/Foo.php","fixers":["concat_space"]}]}

The fields are required with no aliases, so serde rejects it and filter_pint_json falls back to raw output — no compression. Fix is backward-compatible aliases:

#[serde(alias = "path")]
name: String,
#[serde(rename = "appliedFixers", alias = "fixers")]
applied_fixers: Vec<String>,

2. phpstan — reports phpstan: ok while errors exist ⚠️

filter_phpstan_json gates the "ok" case on totals.errors == 0, but PHPStan puts per-file errors in totals.file_errors and leaves totals.errors (global/config errors only) at 0. A real run with 78 file errors — {"totals":{"errors":0,"file_errors":78}, ...} — returns phpstan: ok, silently hiding the failures (the summary line under-reports for the same reason). Note file_errors is currently #[allow(dead_code)]. Fix:

if phpstan.totals.file_errors == 0 && phpstan.totals.errors == 0 { return "phpstan: ok".to_string(); }
// and report phpstan.totals.file_errors in the summary line

Both fixtures set errors == file_errors (phpstan) / use the old field names (pint), which is why tests don't catch these. Happy to open a PR against this branch with both fixes + regression tests using the current-version schemas if that's useful.

@iliaal

iliaal commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the precise report, @evaldnet — both confirmed and fixed on this branch (184ced3).

  • pint: added backward-compatible serde aliases (path/fixers) so current Pint output compresses again instead of falling back to raw.
  • phpstan: the ok gate and summary now read file_errors, so runs with file-level errors no longer report phpstan: ok.

Both ship with regression tests on the current-version schemas — the old fixtures set errors == file_errors and used the old pint keys, which is why the suite stayed green.

@EliW

EliW commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

@evaldnet Thanks for finding those. --- out of curiousity @iliaal --- Does this mean that the code here now may depend on which version of phpstan you are using for example? Or did I grok that incorrectly. Some folks end up stuck on older phpstan's due to framework incompatibilities at times...

@iliaal

iliaal commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

@EliW No. Both old and current tool output parse; I checked each schema against the new code.

pint: the fix adds serde aliases, not a rename. The struct accepts both the old keys (name/appliedFixers) and the current ones (path/fixers), so it parses Pint from either side of the ~1.14 rename. Before, only the old schema parsed and current Pint fell back to raw. Confirmed: old-key and new-key JSON both compress.

phpstan: no new version dependency. totals.errors and totals.file_errors have both been in the --error-format=json output since at least 0.12, unchanged through 1.x and current 2.1.40. errors counts non-file-specific errors, file_errors counts file-specific ones. The parser already required file_errors, it was just unused. The bug read errors where it should read file_errors, which is correct on old and current PHPStan alike.

Safety net under both: if the JSON ever fails to parse, rtk falls back to raw tool output. You lose the compression for that run, not the result. So anyone stuck on an older phpstan or pint won't get broken output; worst case is unfiltered passthrough.

@evaldnet

Copy link
Copy Markdown

Rebuilt on 184ced3 and re-ran against current versions. Both fixes hold:

  • phpstan (2.1.40): file-level errors now report instead of "phpstan: ok".
  • pint (1.29.1, the new path/fixers keys): an 8-fixer file compresses to "pint: 8 changes in 1 files" with the rule list.

One separate thing I noticed while testing: the leading ./ form isn't matched for most tools, so it runs raw with no compression.

  • ./vendor/bin/pint runs raw, while vendor/bin/pint rewrites to rtk pint. Same for pest, paratest, ecs, phpunit.
  • The phpstan pattern already handles this with an optional \.?/? prefix (it also requires the analyse subcommand). The other five patterns are ^(?:vendor/bin/)?<tool>, so a leading ./ defeats the match.

Since ./vendor/bin/... is how most people invoke these in Laravel, adding the same \.?/? prefix would close it.

@iliaal

iliaal commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Fixed on the branch (bd91048). One correction on the mechanism: the regex isn't the blocker. classify_command already normalizes a leading ./ (normalize_php_tool_path strips it), so ./vendor/bin/pint classifies fine without \.?/? in the pattern. The gap is the rewrite step, which strips literal rewrite_prefixes from the raw command. The five rules only carried vendor/bin/<tool> and bare <tool>; phpstan works because ./vendor/bin/phpstan is in its prefix list, not its regex. Fix adds ./vendor/bin/<tool> to the five lists, with a regression test covering ./vendor/bin/{pint,pest,paratest,ecs,phpunit}.

@aeppling aeppling self-assigned this Jun 26, 2026
…pest, paratest, ecs, pint)

Consolidates the PHP-tooling work from three upstream PRs plus a new
Pint module, leaving phpt to its own PR (rtk-ai#1503).

- rtk php / rtk artisan: syntax check (-l) and Laravel artisan wrapper.
- rtk phpunit: structured-state parser, aggregate counts, bounded
  failure list. Uses runner::run_filtered.
- rtk phpstan: typed serde::Deserialize parser for --error-format=json,
  groups errors by file, sorts by count desc. Utility commands
  (--version, list, clear-result-cache) pass through unchanged.
- rtk pest / rtk paratest: shared test_output helper.
- rtk ecs / rtk pint: code-style fixers; pint uses --format=json for
  structured per-file rule counts.

Composer custom-bin-dir detection: composer_bin_dirs() reads
COMPOSER_BIN_DIR and composer.json config.bin-dir, so tools/bin/phpunit
classifies identically to vendor/bin/phpunit. registry.rs normalizes
tool paths before matching.

Sources:
- rtk-ai#1246 (aaronflorey, self-closed): php, artisan, ecs, pest,
  paratest, test_output, utils, composer_bin_dirs, registry normalization.
- rtk-ai#874 (Beninho, open): phpunit state-machine parser.
- rtk-ai#1110 (LucianoVandi, open): phpstan typed parser.
- New: pint_cmd.rs.

Tests: discover::registry 253 pass; cmds::php 36 pass; cargo build
--release 0 errors; cargo fmt --check clean.
@iliaal iliaal force-pushed the feat/php-tooling branch from bd91048 to 9bdcd7c Compare June 26, 2026 14:11
@iliaal

iliaal commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current develop (this branch was ~50 commits behind) and fixed the two CI failures:

  • test (test_every_subcommand_is_classified): develop added this test plus the PASSTHROUGH const after this branch forked, so the merged tree had the php subcommands with no classification entry. Added php/phpunit/phpstan/pest/paratest/ecs/pint to PASSTHROUGH.
  • semgrep (dynamic-command-execution): php_tool_command built the command via Command::new on a resolved path. Routed it through resolved_command (the sanctioned PATHEXT-aware constructor) instead — no behavior change.

The six php-tooling commits replayed with zero conflicts. Locally: test_every_subcommand_is_classified passes, fmt/clippy clean, full suite green. The semgrep fix I verified by inspection (no Command::new(<var>) left in the php dir); CI will confirm.

Heads up: the workflow run on the new head is sitting in action_required — it needs a maintainer to approve before test/semgrep execute.

@KuSh KuSh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — feat(php): consolidated PHP tooling

Review performed using Claude Code (claude-sonnet-4-6) by the RTK maintainers.

This is a solid and well-scoped contribution. The PHPUnit state machine, the PHPStan errors vs file_errors distinction, and the Pint dual-schema handling via serde aliases are all done correctly and are well-tested. The php_tool_command() abstraction for Composer-aware binary resolution is the right design.

That said, there are three blocking bugs and two maintainability issues to address before merge.


🔴 Blockers

1. filter_phpstan_text: "not found" is dangerously broad (inline comment on phpstan_cmd.rs:165)

2. phpstan_cmd.rs bypasses php_tool_command() (inline comment on phpstan_cmd.rs:48)

3. is_numbered_failure_heading() has false positives (inline comment on phpunit_cmd.rs:89)


🟡 Should-fix

4. compact_php_path() is Laravel-only (inline comment on phpstan_cmd.rs:192)

5. short_path() calls current_dir() in a per-file loop (inline comment on pint_cmd.rs:126)


What's working well

  • PHPStan errors vs file_errors bug is correctly identified and tested (test_filter_phpstan_file_errors_not_hidden) — this was a real prior bug fixed here
  • PHPUnit state machine correctly handles success, multi-failure, errors, skipped, and truncation cases with 9 test cases
  • Pint dual-schema serde aliases (name/appliedFixers vs path/fixers) cleanly handle the Pint ≥1.14 rename
  • php_tool_command() centralises Composer bin-dir resolution correctly — pity PHPStan doesn't use it (see inline comment)
  • Token savings are verified against a real 380-line phpstan_raw.json fixture, not synthetic data
  • No unwrap() in production paths, fallback pattern respected throughout

Comment thread src/cmds/php/phpstan_cmd.rs
Comment thread src/cmds/php/phpstan_cmd.rs Outdated
Comment thread src/cmds/php/phpunit_cmd.rs Outdated
iliaal added 2 commits June 28, 2026 19:42
short_path() called current_dir() on every file (up to MAX_FILES_SHOWN per
invocation). Hoist the cwd prefix out of the loop and strip it inline.
…ewrite

The six Composer-tool rules had inconsistent patterns, and their
rewrite_prefixes only worked because classify normalizes the command while
rewrite stripped literal prefixes off the raw text. A form the pattern
accepted but the prefix list missed (e.g. ./bin/phpunit) would classify yet
silently fail to rewrite.

rewrite_segment_inner now normalizes the leading invocation for these tools
(php wrapper, ./, vendor/bin, composer bin-dir) the same way classify does, so
the prefix list collapses to the residual canonical forms: bin/<tool> + bare
name for phpunit/phpstan, bare name for pest/paratest/ecs/pint. Patterns
standardized to ^(?:php\s+)?(?:\./)?(?:(?:vendor/)?bin/)?<tool> for
phpunit/phpstan and ^(?:\./)?(?:vendor/bin/)?<tool> for the rest. Adds a
form-coverage test asserting every accepted spelling maps to one rewrite.
@iliaal iliaal force-pushed the feat/php-tooling branch from 9d05729 to 6117bb4 Compare June 28, 2026 23:52
@iliaal iliaal requested a review from KuSh June 28, 2026 23:53
Comment thread src/cmds/php/pint_cmd.rs
PHP-CS-Fixer omits the fixers key in dry-run/diff modes (it is optional
in the JSON reporter), so a valid report like {"files":[{"name":"x.php"}]}
failed the whole parse and fell back to raw output. Mark the field
#[serde(default)] so a missing key deserializes to an empty vec.
@iliaal iliaal requested a review from KuSh June 30, 2026 00:01

@KuSh KuSh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @iliaal, this looks mostly good to me, just a few minor nitpicks and suggestions. Sorry the PR is a bit huge; I wasn’t able to review it all in one go

Comment thread src/cmds/php/phpunit_cmd.rs
Comment thread src/cmds/php/phpstan_cmd.rs Outdated
Comment thread src/cmds/php/utils.rs Outdated
Comment thread src/cmds/system/pipe_cmd.rs Outdated
Comment thread src/core/utils.rs
Comment thread src/cmds/php/phpunit_cmd.rs Outdated
Comment thread src/cmds/php/phpstan_cmd.rs Outdated
iliaal added 5 commits June 30, 2026 08:18
filter_phpunit_output matched on raw bytes, so `--colors=always` output
(colorized "OK ("/"FAILURES!"/"Tests:" lines) defeated every anchor and
counts were lost. Strip ANSI first, mirroring the other PHP filters.

Also report PHPUnit errors (thrown exceptions) distinctly from failures
(assertion mismatches) instead of lumping both under "failures".
…format

`phpstan -c phpstan.neon analyse src/` put a global option before the
subcommand, so the first-arg-only check missed it and the run fell back
to unfiltered passthrough. Scan all args for analyse/analyze.

Replace the has_custom_format bool with a 3-state ErrorFormat enum so an
explicit `--error-format=json` is preserved without appending a second
`--error-format json`. Strip ANSI in the text fallback for `--ansi`.
Pest has no root `pest.php` file — its bootstrap lives at tests/Pest.php
and its canonical marker is the vendor/bin/pest binary. The root-level
check never matched real Pest projects and false-positived on unrelated
utility files, misrouting PHPUnit-only projects to the Pest filter.
Matching only "by Sebastian Bergmann" misrouted any LICENSE, composer
metadata, or git log mentioning the author to the phpunit filter. Require
the input to start with "PHPUnit " as well.
composer_bin_dirs() re-read composer.json and the env on every call; the
rewrite hot path queries it several times per command segment via
normalize_php_tool_command (both classify_command and rewrite_segment_inner).
Resolution is constant for a process, so cache it in a OnceLock.
@iliaal iliaal requested a review from KuSh June 30, 2026 21:34

@KuSh KuSh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add tests for paratest? That’s what’s causing the CI failure. See .claude/rules/cli-testing.md for the full testing guide. After that, LGTM!

paratest_cmd.rs lacked the mandatory #[cfg(test)] module, failing the
check-test-presence CI gate. Add tests exercising the ParaTest-specific
path in filter_test_runner_output: banner + "Random Seed:" + dot progress
stripped, result summary and failure details preserved.
@iliaal

iliaal commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

Added in fe75595. paratest_cmd.rs was missing the mandatory #[cfg(test)] module — that's what tripped the check-test-presence gate. The two tests exercise the ParaTest-specific branch of filter_test_runner_output (banner, Random Seed:, and dot-progress stripped; result summary and failure details preserved), rather than duplicating pest's coverage.

Verified locally: bash scripts/check-test-presence.sh origin/develop now passes all 9 modules, fmt/clippy clean.

iliaal added 2 commits July 1, 2026 06:17
.claude/rules/cli-testing.md marks token-savings verification as a 🔴
requirement for filters. Add a ≥60% savings assertion over a realistic
paratest run (banner + config + parallel-worker progress → one-line
summary), alongside the existing behavioral tests.
@iliaal iliaal requested a review from KuSh July 1, 2026 10:20
@KuSh

KuSh commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

LGTM, thanks for all the work!

@KuSh KuSh merged commit 1052c1c into rtk-ai:develop Jul 1, 2026
11 checks passed
@github-actions github-actions Bot mentioned this pull request Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort-large Plusieurs jours, nouveau module enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants