testing

Testing Skill

Write tests that are valuable — tests that protect against regressions, survive refactoring, run fast, and stay maintainable. A test suite that slows down development or breaks on every refactor is worse than no tests at all, because it creates the illusion of safety while taxing every change.

Language-specific frameworks and examples are in the references/ directory. Read the relevant file when generating tests:

• references/cpp.md — GoogleTest, GoogleMock, dependency injection in C++
• references/python.md — pytest, unittest.mock, fixtures, parametrize
• references/rust.md — built-in test framework, mockall, test organization
• references/java.md — JUnit 5, Mockito, AssertJ, test organization

---

The Four Pillars of a Valuable Test

Every test should be evaluated against four qualities. A test that scores poorly on any one of them is a candidate for rewriting or deletion.

1. Protection against regressions. The test should catch bugs when someone changes the code it covers. A test that passes regardless of what you do to the production code protects nothing. The more production code a test exercises — and the more complex that code is — the higher its regression protection.

2. Resistance to refactoring. The test should not break when you restructure code without changing its behavior. A test that fails on every internal refactor produces false positives — alarms that train the team to ignore test failures. This is the most undervalued pillar and the one most tests fail on. Tests that verify implementation details (method call order, internal state, private method invocations) have low resistance to refactoring. Tests that verify observable behavior (outputs, state changes visible to the caller, side effects on external systems) have high resistance.

3. Fast feedback. The test should run in milliseconds to seconds. Slow tests don't get run during development, which means they only catch regressions in CI — too late to be useful for the feedback loop. A test that takes 30 seconds is a test that developers skip locally.

4. Maintainability. The test should be easy to read, easy to understand, and cheap to modify. A 200-line test with intricate setup is expensive to maintain. Tests are code — they require the same care for readability as production code.

The tradeoff: you cannot maximize all four simultaneously. The key insight is that resistance to refactoring is non-negotiable — you either have it or you don't, and tests without it actively harm development velocity. The real tradeoff is between regression protection and fast feedback: unit tests optimize for speed, integration tests optimize for regression coverage. Choose based on the code's characteristics.

---

Test Behavior, Not Implementation

The single most important testing principle for large codebases. Test what the code does, not how it does it.

Observable behavior is anything visible to the caller or to an external system: return values, state changes accessible through the public API, calls to external dependencies (database writes, HTTP requests, messages sent). Test these.

Implementation details are everything else: private method calls, internal data structures, the order of internal operations, which helper classes are used. Never test these directly. If a test breaks because you extracted a method, renamed an internal class, or changed an algorithm without changing the result, the test is coupled to implementation.

The practical test: after a refactoring that doesn't change any public contract, how many tests break? If the answer is more than zero, those tests are testing implementation. Fix them or delete them.

Mocks and implementation coupling. Mocking internal collaborators couples tests to the interaction between classes — a form of implementation detail. Reserve mocks for unmanaged dependencies (external systems you don't control: APIs, databases, message queues, file systems). For internal collaborators within your codebase, use real instances. This is the core difference between the classical and London schools, and the classical approach produces tests that survive refactoring.

---

The Three Styles of Testing

Output-based testing. Supply inputs, verify the output. No state changes, no side effects. This is the ideal style — pure function in, value out, assert on the value. Highest resistance to refactoring, easiest to maintain. Prefer this whenever possible.

result = calculate_discount(price=100, tier=Gold)
assert result == 85

State-based testing. Perform an operation, then verify the resulting state through the public API. More setup than output-based, but necessary when the operation mutates observable state.

cart.add(item)
assert cart.item_count() == 1
assert cart.total() == item.price

Communication-based testing. Verify that the system under test interacts correctly with an external dependency. Uses mocks or spies. The most fragile style — use only for interactions with unmanaged dependencies (external systems), never for internal collaborations.

order_service.place_order(order)
verify(payment_gateway).charge(order.total)  // external system
verify(email_service).send(confirmation)      // external system

Priority: prefer output-based > state-based > communication-based. Push your design toward pure functions and value objects to maximize the proportion of output-based tests.

---

The TDD Cycle

The fundamental rhythm of test-driven development:

Red. Write a test that fails. The test defines the next increment of behavior. It should be small — one assertion, one concept. Run it and watch it fail. If it passes immediately, either the behavior already exists or the test is wrong.

Green. Write the minimum production code to make the test pass. Don't generalize, don't optimize, don't clean up. Just make it green. The goal is to get from red to green as fast as possible.

Refactor. Now clean up both the production code and the test. Remove duplication, improve names, extract functions, simplify. All tests should remain green throughout. This is where design emerges — driven by real needs, not speculation.

The discipline: never write production code without a failing test. Never refactor with a failing test. The cycle keeps you in a state where the code always works and every behavior is tested.

When to use TDD vs. test-after: TDD works best for new logic with clear inputs and outputs. For exploratory work, prototyping, or integration code, it's often more practical to write the code first and add tests immediately after, before moving on. The non-negotiable part is: code does not ship without tests, regardless of which came first.

---

Test Doubles

Five types of test doubles, each with a specific purpose. Using the wrong type leads to brittle or meaningless tests.

Dummy. An object passed to satisfy a parameter but never actually used. The test doesn't care about it. Example: a null logger passed to a constructor that requires one.

Stub. Returns predefined answers to calls made during the test. It feeds data into the system under test. Stubs replace incoming dependencies — things the system reads from.

Spy. Records calls made to it so you can verify them later. A spy is a manually written mock. Use for verifying outgoing interactions with external systems.

Mock. Like a spy but with built-in verification — pre-programmed with expectations about which calls should happen. Mocks verify outgoing interactions with unmanaged dependencies.

Fake. A working implementation with a shortcut that makes it unsuitable for production. An in-memory database, a local file system instead of cloud storage, a fake HTTP server. Fakes replace managed dependencies — things you control but want to simplify for testing.

The critical distinction:

• Incoming interactions (system reads data): use stubs or fakes. Never assert on them — asserting that a stub was called is testing implementation.
• Outgoing interactions with external systems (system writes to database, sends email, calls API): use mocks or spies. Assert on these — they're part of the observable behavior.
• Outgoing interactions with internal collaborators (system calls another class in your codebase): use real instances. Don't mock what you own.

---

Property-Based Testing

Instead of specifying individual examples, define properties that should hold for all valid inputs and let the framework generate test cases. Property-based testing excels at finding edge cases that example-based tests miss — off-by-ones, empty inputs, Unicode, integer overflow, and combinations you didn't think of.

When to use: algorithms with clear invariants (sort output is ordered and has same elements), serialization round-trips (encode then decode equals original), idempotent operations, commutative operations, any function with a known relationship between input and output.

When not to use: UI interactions, integration tests with external systems, code where the expected output can only be determined by running the code itself.

The process: identify a property ("for all valid inputs, sorting then checking order should hold"), write a generator for the input space, let the framework find counterexamples, then shrink to the minimal failing case. When a property test finds a bug, add the minimal failing case as a regression example-based test.

See reference files for language-specific frameworks: Hypothesis (Python), proptest (Rust), jqwik (Java), RapidCheck (C++).

---

Testing Legacy Code

Legacy code is code without tests. The challenge: you can't refactor safely without tests, but you can't write tests without some refactoring. Feathers provides a disciplined way to break this cycle.

The Legacy Code Change Algorithm

1. **Identify chang

---

Content truncated.