Skip to content
/ mutant Public

Mutation testing for Ruby. AI writes your code. AI writes your tests. But who tests the tests?

License

Notifications You must be signed in to change notification settings

mbj/mutant

Repository files navigation

mutant

Build Status Gem Version Discord

What is Mutant?

AI writes your code. AI writes your tests. But who tests the tests?

Copilot, Claude, and ChatGPT generate code faster than humans can review it. They'll even write tests that pass. But passing tests aren't the same as meaningful tests.

Mutant is mutation testing for Ruby. It systematically modifies your code and verifies your tests actually catch each change. When a mutation survives, you've found either:

  • Dead code - the AI over-engineered something you can delete
  • A blind spot - the AI forgot to test a behavior that matters

The more code AI writes for you, the more you need verification you can trust.

Quick Start

# lib/person.rb
class Person
  def initialize(age:)
    @age = age
  end

  def adult?
    @age >= 18
  end
end
# spec/person_spec.rb
RSpec.describe Person do
  describe '#adult?' do
    it 'returns true for age 19' do
      expect(Person.new(age: 19).adult?).to be(true)
    end

    it 'returns false for age 17' do
      expect(Person.new(age: 17).adult?).to be(false)
    end
  end
end

Tests pass. But run mutant:

gem install mutant-rspec
mutant run --use rspec --usage opensource --require ./lib/person 'Person#adult?'

Mutant finds a surviving mutation indicating a shallow test:

 def adult?
-  @age >= 18
+  @age > 18
 end

Your tests don't cover age == 18. The mutation from >= to > doesn't break them.

This is just one of many mutation operators. Mutant also mutates arithmetic, logical, bitwise operators, removes statements, modifies return values, and more.

A full working example is available in the quick_start directory.

Next Steps

  1. Learn the nomenclature (subjects, mutations, operators)
  2. Set up your integration: RSpec or Minitest
  3. Run mutant on CI in incremental mode

Ruby Versions

Mutant is supported on Linux and macOS.

Version Runtime Syntax Mutations
3.2 ✔️ ✔️ ✔️
3.3 ✔️ ✔️ ✔️
3.4 ✔️ ✔️ ✔️
4.0 ✔️ ✔️ 🚧*

*Mutations specific to Ruby 4.0 syntax pending parser gem support.

Licensing

Free for open source. Use --usage opensource for public repositories.

Commercial use requires a subscription ($90/month per developer). See commercial licensing for pricing and details.

Documentation

Communication

Contributing

See CONTRIBUTING.md for development setup and guidelines.

Sponsoring

Mutant, as published in the opensource version, would not exist without the help of contributors spending lots of their private time.

Additionally, the following features where sponsored by organizations:

  • The mutant-minitest integration was sponsored by Arkency
  • Mutant's initial concurrency support was sponsored by an undisclosed company that does currently not wish to be listed here.

Rust Implementation

Parts of Mutant are being incrementally rewritten in Rust for improved performance. This is currently opt-in and requires no changes for existing users. See RUST.md for details.

Sponsor this project

 

Packages

No packages published

Contributors 54