hypothesis

Hypothesis - Property-Based Testing

Property-based testing framework that generates test cases automatically, finds minimal failing examples through shrinking, and verifies invariants.

Official Docs: https://hypothesis.readthedocs.io/en/latest/

Key Features:

• Automatic test data generation from strategies
• Minimal failing example shrinking
• Stateful testing with rule-based state machines
• pytest integration
• Deterministic reproducibility

Quick Start

from hypothesis import given
from hypothesis import strategies as st


@given(st.integers())
def test_property(x):
    """Test properties that should always hold"""
    assert abs(x) >= 0


@given(st.lists(st.integers()))
def test_list_property(lst):
    sorted_lst = sorted(lst)
    assert len(sorted_lst) == len(lst)
    # Check monotonic property
    for i in range(len(sorted_lst) - 1):
        assert sorted_lst[i] <= sorted_lst[i + 1]

Strategies

Full reference: https://hypothesis.readthedocs.io/en/latest/data.html

Common strategies:

• Primitives: st.integers(), st.floats(), st.text(), st.booleans()
• Collections: st.lists(), st.dictionaries(), st.tuples(), st.sets()
• Dates/Times: st.dates(), st.datetimes(), st.timedeltas()
• Combinators: st.one_of(), st.sampled_from(), st.recursive()
• Type-based: st.from_type(MyClass)

Composite Strategies

from hypothesis import strategies as st
from hypothesis.strategies import composite


@composite
def user_strategy(draw):
    age = draw(st.integers(min_value=18, max_value=100))
    name = draw(st.text(min_size=1))
    return {"name": name, "age": age, "is_adult": age >= 18}


@given(user_strategy())
def test_user(user):
    assert user["is_adult"] == (user["age"] >= 18)

Strategy Combinators

st.integers().filter(lambda x: x % 2 == 0)  # Filter
st.integers().map(str)  # Transform
st.one_of(st.integers(), st.text())  # Choose between strategies
st.sampled_from([1, 2, 3, 4, 5])  # Pick from collection
st.from_type(MyClass)  # Infer from type hints
st.builds(MyClass, arg1=st.integers())  # Build instances

Settings

from hypothesis import given, settings
from hypothesis import strategies as st


@given(st.integers())
@settings(
    max_examples=1000,  # Default: 100
    deadline=None,  # Remove time limit
    derandomize=True,  # Deterministic ordering
)
def test_example(x):
    pass


# Profiles for different environments
settings.register_profile("dev", max_examples=10)
settings.register_profile("ci", max_examples=1000, deadline=None)
# Activate: HYPOTHESIS_PROFILE=ci pytest

Full settings reference: https://hypothesis.readthedocs.io/en/latest/settings.html

Helpers

from hypothesis import given, assume, note, example, seed


@given(st.integers(), st.integers())
def test_division(x, y):
    assume(y != 0)  # Skip invalid cases (prefer .filter() instead)
    note(f"Testing {x} / {y}")  # Add debug info
    assert (x / y) * y == x


@given(st.integers())
@example(0)  # Always test specific cases
@seed(12345)  # Reproducible run
def test_something(x):
    pass

Stateful Testing

For testing complex stateful systems with rule-based state machines.

from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
from hypothesis import strategies as st


class MyStateMachine(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.data = []

    @rule(value=st.integers())
    def add(self, value):
        self.data.append(value)

    @invariant()
    def check_invariant(self):
        assert isinstance(self.data, list)


TestMachine = MyStateMachine.TestCase

Full stateful testing guide: https://hypothesis.readthedocs.io/en/latest/stateful.html

Polars Integration

Polars provides built-in parametric testing strategies for generating DataFrames.

Official docs: https://docs.pola.rs/api/python/stable/reference/api/polars.testing.parametric.dataframes.html

from hypothesis import given
import polars as pl
from polars.testing.parametric import dataframes, column


# Generate DataFrames with specific column schemas
@given(
    dataframes(
        cols=[
            column("id", dtype=pl.Int64),
            column("name", dtype=pl.String),
            column("value", dtype=pl.Float64),
        ],
        min_size=1,
        max_size=100,
    )
)
def test_dataframe_property(df: pl.DataFrame):
    """Test properties of DataFrame operations"""
    assert df.shape[0] >= 1
    assert set(df.columns) == {"id", "name", "value"}
    assert df["id"].dtype == pl.Int64


# With Narwhals wrapper
import narwhals as nw


@given(dataframes(cols=[column("a", dtype=pl.Int64)]))
def test_narwhals_operation(df: pl.DataFrame):
    nw_df = nw.from_native(df)
    result = nw_df.select(nw.col("a") * 2)
    assert result.shape[0] == nw_df.shape[0]

Key functions:

• dataframes(): Generate DataFrames with specified columns
• column(name, dtype, ...): Define column schemas with constraints
• series(): Generate standalone Series

Column constraints:

• null_probability: Control null value frequency
• min_size/max_size: Control row count
• allow_null: Enable/disable nulls
• unique: Generate unique values
• strategy: Custom strategy for column values

Best Practices

• Use constraints over filters: st.integers(min_value=0) not st.integers().filter(lambda x: x >= 0)
• Test properties, not examples: Focus on invariants that always hold
• Combine with @example(): Test specific edge cases explicitly
• Avoid assume() overuse: Makes tests slow; use filtered strategies
• Document properties: Clear docstrings explain what invariant is tested
• Set size limits: Always bound collection sizes to prevent memory issues
• Use .hypothesis/ in .gitignore: Stores example database locally

Troubleshooting

Common issues and solutions:

• HealthCheck failures: Too many rejected examples → use constrained strategies or suppress_health_check
• Flaky tests: Non-deterministic code → use @seed() or @settings(derandomize=True)
• Slow tests: Too many examples → reduce max_examples or use profiles
• Deadline exceeded: Complex operations → increase deadline or set to None

Resources

• Main docs: https://hypothesis.readthedocs.io/
• Strategies: https://hypothesis.readthedocs.io/en/latest/data.html
• Stateful testing: https://hypothesis.readthedocs.io/en/latest/stateful.html
• Ghost writer (auto-generate tests): hypothesis write mymodule.myfunction