Skip to content

Load cops lazily instead of requiring all 271 files at startup - #1122

Open
tas50 wants to merge 2 commits into
chef:mainfrom
tas50:lazy_load_cops
Open

Load cops lazily instead of requiring all 271 files at startup#1122
tas50 wants to merge 2 commits into
chef:mainfrom
tas50:lazy_load_cops

Conversation

@tas50

@tas50 tas50 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

What

lib/cookstyle.rb globbed lib/rubocop/cop/**/*.rb and required every file, so every invocation parsed and defined all 270 cop classes before inspecting a single line — including cookstyle --version and runs narrowed to one cop with --only.

This applies the technique RuboCop core added in 1.89 and rubocop-performance adopted in rubocop/rubocop-performance#530. RuboCop::Cop::LazyLoader#register_cop does two things: it sets an autoload for the cop's constant, and it registers the cop's name with RuboCop's global registry as a string. The registry can then validate the config, resolve --only arguments, and list departments knowing only names. A cop's file is read the first time the class itself is needed — which, for a cop disabled in the config, is never.

Registrations live in two new files, lib/rubocop/cop/chef.rb and lib/rubocop/cop/inspec.rb, in the same order the files were previously required, so cop execution order is unchanged.

module RuboCop
  module Cop
    module Chef
      module Correctness
        extend LazyLoader

        register_cop :BlockGuardWithOnlyString, "#{__dir__}/chef/correctness/block_guard_clause_string_only"
        register_cop :ChefApplicationFatal, "#{__dir__}/chef/correctness/chef_application_fatal"
        ...

No cop files were moved or renamed. register_cop takes an explicit path, so the 92 cops whose class name doesn't match their filename and the directories that don't match their department (chef/deprecation/Chef::Deprecations, chef/redundant/Chef::RedundantCode, chefstyle/ruby/Chef::Ruby) all work as they are.

Benchmark

Ruby 4.0.6, macOS arm64, RuboCop 1.90.0. Median of 15 runs, before and after measured back to back on the same machine. Fixture is a 4-file cookbook.

Scenario Before After Change
require "cookstyle" (isolated, excludes Ruby/Bundler boot) 488.4 ms 394.9 ms −93.5 ms / −19.1%
cookstyle --version (end to end) 755.8 ms 632.9 ms −122.9 ms / −16.3%
cookstyle --only Chef/Modernize/FoodcriticComments (1 file) 760.2 ms 687.7 ms −72.5 ms / −9.5%
cookstyle full lint (4-file cookbook) 1014.3 ms 992.7 ms −21.6 ms / −2.1%

Cop files read during require "cookstyle":

Before After
Cop classes loaded at require time 271 0

And loaded over the life of a real run:

Run Cops loaded
cookstyle --version 0 of 270
cookstyle --only <one cop> 1 of 270
cookstyle full cookbook lint 249 of 270

The full-lint case still improves because the 21 cops disabled in cookstyle.yml are now never loaded at all. The --version and --only cases are where the win is largest, and those matter for editor integrations and for the tools that embed Cookstyle.

Reproduce the isolated require measurement with:

for i in $(seq 1 15); do
  bundle exec ruby -e 't0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    require "cookstyle"
    puts(((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round(1))'
done | sort -n

Correctness

  • --format json output is byte-for-byte identical before and after, both on a cookbook fixture and across this repository's 557 Ruby files (11 offenses, same cops, same lines, same messages).
  • Full suite green: 1245 examples, 0 failures.
  • rake validate_config, bundle exec cookstyle, --chefstyle, --show-cops, --only <cop>, --only <department>, and -a all verified by hand.
  • rake generate_cops_yml_documentation and rake update_readme_cop_count still work — the docs task goes through Registry#cops/#sort!, which force every lazy cop to load.

Keeping registrations honest

Lazy loading means adding a cop file is no longer enough to ship a cop, so spec/lazy_cop_loading_spec.rb turns that into a test failure. It asserts that every cop file is registered exactly once, that no registration points at a missing file, that each registered constant matches the badge its class actually declares, and that requiring cookstyle still loads zero cop classes. Each of those was mutation-tested against a deliberately broken registration.

A missing registration is loud in two other places anyway: rake validate_config fails, and so does require "cookstyle" itself, since the config entry then names a cop RuboCop has never heard of.

unrecognized cop or department Chef/Correctness/InvalidChecksum found in config/cookstyle.yml

rake validate_config needed one fix. It used const_source_location to tell a Cookstyle cop from a Chefstyle cop, but on a pending autoload const_source_location returns the register_cop call site rather than the cop's own file. It now asks autoload? first, falling back to const_source_location for cops that are already loaded.

WRITING_RULES.md and DEVELOPER_GUIDE.md are updated: "the five files" is now six, with the register_cop line as step 2.

Requirements

None new. Cookstyle already pins RuboCop 1.90.0, well past the 1.89.0 that introduced LazyLoader.

`lib/cookstyle.rb` globbed `lib/rubocop/cop/**/*.rb` and required every file,
so every invocation paid to parse and define all 270 cop classes before it
inspected a single line — including `cookstyle --version` and runs filtered
down to one cop with `--only`.

RuboCop 1.89 added `RuboCop::Cop::LazyLoader`, which rubocop-performance
adopted in rubocop/rubocop-performance#530. `register_cop` sets an `autoload`
for the cop's constant and registers its *name* with the global registry, so
config validation, `--only` resolution, and department listings all work
without reading the cop's file. The class is loaded the first time it is
actually needed, which for a disabled cop is never.

Registrations live in the new `lib/rubocop/cop/chef.rb` and
`lib/rubocop/cop/inspec.rb`, in the order the files were previously required,
so cop execution order is unchanged. Offense output on both a cookbook and
this repository is byte-for-byte identical before and after.

Measured on Ruby 4.0.6 / macOS arm64, median of 15 runs:

  require "cookstyle" (isolated)   488.4ms -> 394.9ms   -19.1%
  cookstyle --version              755.8ms -> 632.9ms   -16.3%
  cookstyle --only <one cop>       760.2ms -> 687.7ms    -9.5%
  full lint, 4-file cookbook      1014.3ms -> 992.7ms    -2.1%

Cop files read at require time drop from 271 to 0. A full cookbook lint loads
249 of 270 (the 21 cops disabled in the default config never load), a single
`--only` cop loads 1, and `--version` loads none.

Adding a cop now means adding a `register_cop` line as well as the file.
`spec/lazy_cop_loading_spec.rb` makes a missing, duplicated, or misnamed
registration a test failure, and asserts that requiring cookstyle still loads
no cop classes. A missing registration also fails `rake validate_config` and
`require "cookstyle"` itself, because the config entry then names a cop
RuboCop has never heard of.

`rake validate_config` needed one fix: `const_source_location` on a pending
autoload returns the `register_cop` call site rather than the cop's own file,
so it now asks `autoload?` first to tell Cookstyle and Chefstyle cops apart.

Signed-off-by: Tim Smith <tsmith84@proton.me>
@tas50
tas50 requested review from a team and jaymzh as code owners August 26, 2026 01:39
The new files follow the same header convention as the rest of the tree, but
with the copyright attributed to their author rather than to Progress, which
is what the existing individually-authored files here do (see the Kantrowitz
and Henry headers) and what Chef/Style/CopyrightCommentFormat documents as
the correct form for an individual holder.

Signed-off-by: Tim Smith <tsmith84@proton.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant