muto: mutation testing for Swift that reports every test faithfully

muto is a Swift mutation testing framework and command-line tool that plants deliberate bugs and reports, per test, whether your suite notices them.

The problem

Code coverage says which lines ran during the tests, not whether the tests would notice if those lines were wrong. Mutation testing answers that by changing the code on purpose and checking that a test fails.

What muto does

muto generates mutants (small deliberate changes to Swift source), builds a test plan, runs the tests and reports the result for every test. It reports faithfully and never decides: judging the results is left to whatever consumes the report.

The muto command line covers generate, test-plan, test, prepare, version and prune-simulators. Results are a versioned JSON contract, so other tools can rely on the shape.

Your tool (pesi or another)mutofilesmutantscommandsrun testresultSwiftSwift projectSwiftPM or XcodeApply each mutantIn a scratch copyDecideCaught or missedmuto generateSorts production from test files,mutates with swift-syntaxa < b → a <= bmuto test-planWrites build and test commandsmuto testChecks toolchain and build identity,runs on a fresh simulator,records every test result
filesmutants and commandsrun testresultSwiftSwift projectSwiftPM or Xcodeyour toolmuto generateSorts production from test files,mutates with swift-syntaxa < b → a <= bmuto test-planWrites build and test commandsApply each mutantIn a scratch copyyour toolmuto testChecks toolchain and build identity,runs on a fresh simulator,records every test resultDecideCaught or missedyour tool
muto reports every test faithfully and never decides; the calling tool judges whether a mutant was caught.

How it’s built

  • Mutants are generated from the syntax tree with swift-syntax rather than by text substitution.
  • Tests run through xcodebuild. Each run can use a fresh simulator with a memory ceiling, and muto checks the expected toolchain and build identity before it starts.
  • It started from research into forking an existing Swift mutation tool, and is the engine pesi’s Swift mutation plugin pins to.
  • Releases are verified end to end on a clean machine and carry build provenance attestations.

Get in touch

Text me:

Open in new tab

Loading…