python-hypothesis-patterns (v1.0.0)

Rationale

Example-based tests prove a function works on the cases the author thought of. Property-based tests assert an invariant and let the framework search for COUNTEREXAMPLES — including edge cases the author would never have thought of. For SDK code, properties shine on serializers / parsers / validators / state machines: "for any input, parse(serialize(x)) == x", "for any sequence of operations, the final state is deterministic", "for any URL the validator accepts, the request building doesn't crash."

hypothesis is the canonical Python library. It generates random inputs, shrinks failures down to a minimal reproducer, and persists failing seeds across runs.

This skill is cited by code-reviewer-python (test-quality), python-pytest-patterns (Rule 2 — parametrize for finite cases, hypothesis for infinite spaces), python-mypy-strict-typing (st.from_type uses annotations), and sdk-convention-devil-python (C-14 testing).

Activation signals

• SDK has a parser, serializer, validator, or state machine.
• TPRD §3 declares "for any input, X must hold".
• Reviewing whether a fixed table of test cases is sufficient (probably not).
• A bug ships that example tests missed — would a property have caught it?
• Code under test has a round-trip property: decode(encode(x)) == x.

Core decision — property or example?

Use property-based testing when:

• The input space is large or unbounded (strings, integers, lists, JSON).
• An INVARIANT is articulable: round-trip, idempotent, monotonic, commutative, ordering-preserving.
• The function is pure (or has a fakeable seam — see python-mock-strategy).

Use example-based testing when:

• Specific inputs and expected outputs are the contract (e.g., REST API validates topic matches a known regex).
• The function has side effects you specifically test for.
• The bug surface is narrow.

Both layers complement each other; ship both for any non-trivial parser / validator.

Rule 1 — Strategies are the input space

from hypothesis import given, strategies as st


@given(st.text())
def test_normalize_topic_idempotent(t: str) -> None:
    assert normalize_topic(normalize_topic(t)) == normalize_topic(t)


@given(st.integers(min_value=0, max_value=10_000))
def test_backoff_bounded(attempt: int) -> None:
    delay = estimate_backoff_seconds(attempt=attempt)
    assert 0.0 <= delay <= 30.0


@given(st.lists(st.integers(), min_size=1, max_size=100))
def test_sort_then_min_equals_min(xs: list[int]) -> None:
    assert sorted(xs)[0] == min(xs)

hypothesis.strategies (st) builds typed input spaces:

Strategy	Generates
st.text()	Strings (unicode by default)
st.text(alphabet=st.characters(min_codepoint=32, max_codepoint=126))	ASCII printable
st.binary()	bytes
st.integers(min_value, max_value)	int
st.floats(min_value, max_value, allow_nan=False, allow_infinity=False)	float
st.booleans()	True / False
st.lists(elements, min_size, max_size, unique=True)	list
st.dictionaries(keys, values)	dict
st.sets(elements)	set
st.tuples(s1, s2, ...)	tuple
st.one_of(s1, s2, ...)	union
st.sampled_from([...])	element from finite set
st.builds(Class, **strategy_kwargs)	Class instance
st.from_type(Cls)	infer from type annotations
st.datetimes(), st.dates(), st.times(), st.timedeltas()	datetime types
st.uuids()	UUID
st.emails()	email-shape strings
st.ip_addresses()	IPv4/IPv6
st.fixed_dictionaries({"k": st.integers()})	dict with fixed shape

For SDK domain objects, build a strategy:

@st.composite
def topics(draw) -> Topic:
    """Strategy for valid Topic instances.

    Constraints (per domain rules): namespace and name non-empty, no '.' inside.
    """
    alpha = st.characters(min_codepoint=ord("a"), max_codepoint=ord("z"))
    namespace = draw(st.text(alphabet=alpha, min_size=1, max_size=20))
    name = draw(st.text(alphabet=alpha, min_size=1, max_size=20))
    return Topic(namespace=namespace, name=name)


@given(topics())
def test_topic_qualified_round_trip(t: Topic) -> None:
    assert Topic.parse(t.qualified) == t

@st.composite is the canonical pattern for building related-field combinations. The draw callable pulls from inner strategies; the function body returns the assembled object.

Rule 2 — st.from_type for typed dataclasses

from dataclasses import dataclass
from hypothesis import given, strategies as st


@dataclass(frozen=True, slots=True, kw_only=True)
class Config:
    base_url: str
    timeout_s: float = 5.0
    max_retries: int = 3


@given(st.from_type(Config))
def test_config_post_init_invariant(cfg: Config) -> None:
    """Whatever Config we generate, timeout_s > 0 must hold per __post_init__."""
    assert cfg.timeout_s > 0


# Register a custom strategy if the default is too broad
st.register_type_strategy(Config, st.builds(
    Config,
    base_url=st.sampled_from(["http://x", "https://y"]),
    timeout_s=st.floats(min_value=0.1, max_value=60.0),
    max_retries=st.integers(min_value=0, max_value=10),
))

st.from_type(T) reads T's type annotations and synthesizes a strategy. For dataclasses + plain int/str/float/bool fields, it works out of the box. For custom types with constraints, st.register_type_strategy() overrides the default.

Rule 3 — @settings for time / iteration budgets

from hypothesis import given, settings, HealthCheck


@given(...)
@settings(
    max_examples=200,                  # default 100; raise for thorough coverage
    deadline=500,                      # ms; default 200; raise for slow properties
    suppress_health_check=[HealthCheck.too_slow],
)
def test_thing(...) -> None:
    ...

Defaults are conservative for unit tests. For integration-grade properties (touch a fake / fixture), raise deadline to the realistic upper bound. NEVER set deadline=None to silence flakes — the test then hangs CI.

For long-running properties run as a separate suite, profile via:

@settings(profile="ci")           # uses the "ci" profile registered globally

# In conftest.py:
from hypothesis import settings, Phase

settings.register_profile("ci", max_examples=500, deadline=2000)
settings.register_profile("dev", max_examples=20, deadline=200)
settings.load_profile("dev")      # default

HYPOTHESIS_PROFILE=ci pytest switches to the CI profile.

Rule 4 — @example seeds known cases

from hypothesis import example, given

@given(st.text())
@example("")                                     # empty string edge
@example("a" * 256)                              # max length edge
@example("")                               # null byte edge
@example("orders.created")                       # known good
def test_normalize_topic(t: str) -> None:
    result = normalize_topic(t)
    assert isinstance(result, str)

@example adds DETERMINISTIC seeds before random generation. Use for:

• Known edge cases the random search may miss.
• Regression seeds — when a bug was found in production, add the bug input as @example so it never re-shrinks away.

Rule 5 — @reproduce_failure for replay

When hypothesis finds a failure, it prints a @reproduce_failure(...) decorator with a database key. Add it to the test to replay deterministically:

@given(st.text())
@reproduce_failure("6.103.5", b"AXicY2BgYAAAAAQ...")
def test_normalize_topic(t: str) -> None:
    ...

The first two arguments — hypothesis version and seed bytes — make the failure reproducible across machines + runs. Once the bug is fixed, REMOVE @reproduce_failure (it pins to the fix; otherwise wastes CI time replaying a now-passing seed).

For team workflows, hypothesis stores failing examples in .hypothesis/examples/ (gitignore'd by default) — commit a copy to share regression seeds, OR rely on @example for permanent regression coverage.

Rule 6 — @assume filters; don't over-filter

from hypothesis import assume, given

@given(st.integers(), st.integers())
def test_division_remainder(a: int, b: int) -> None:
    assume(b != 0)                              # filter zero-divisors
    quotient, remainder = divmod(a, b)
    assert quotient * b + remainder == a

assume(condition) silently rejects examples that don't satisfy the precondition. Better: tighten the strategy to never generate them:

@given(st.integers(), st.integers().filter(lambda b: b != 0))
def test_division_remainder(a: int, b: int) -> None:
    quotient, remainder = divmod(a, b)
    assert quotient * b + remainder == a

Strategy .filter() and .map() are more efficient than assume because hypothesis can avoid re-generating. Use assume only for cross-input constraints (e.g., assume(a < b)).

If >33% of examples are filtered, hypothesis emits a Flaky warning. Tighten the strategy.

Rule 7 — Common property templates

Round-trip (encode/decode, parse/serialize)

@given(st.from_type(Message))
def test_message_roundtrip(msg: Message) -> None:
    encoded = serialize(msg)
    decoded = deserialize(encoded)
    assert decoded == msg

The most useful pattern — a parser bug almost always shows up here.

Idempotence

@given(st.text())
def test_normalize_idempotent(t: str) -> None:
    assert normalize(normalize(t)) == normalize(t)

Commutativity / associativity

@given(st.dictionaries(st.text(), st.integers()), st.dictionaries(st.text(), st.integers()))
def test_merge_commutative_for_disjoint_keys(a, b) -> None:
    assume(set(a) & set(b) == set())            # no key collision
    assert merge(a, b) == merge(b, a)

Monotonicity

@given(st.integers(min_value=0, max_value=100), st.integers(min_value=0, max_value=100))
def test_backoff_monotonic_in_attempt(a: int, b: int) -> None:
    if a < b:
        assert estimate_backoff_seconds(attempt=a) <= estimate_backoff_seconds(attempt=b)

Bound (output stays within range)

@given(st.integers(min_value=0, max_value=10_000))
def test_backoff_capped(attempt: int) -> None:
    delay = estimate_backoff_seconds(attempt=attempt)
    assert 0.0 <= delay <= 30.0

Preserves a property

@given(st.lists(st.integers()))
def test_sort_preserves_length_and_multiset(xs: list[int]) -> None:
    sorted_xs = sorted(xs)
    assert len(sorted_xs) == len(xs)
    assert sorted(sorted_xs) == sorted(xs)

Rule 8 — Stateful testing for state machines

For SDK components with internal state (rate limiter, circuit breaker, retry counter):

from hypothesis.stateful import RuleBasedStateMachine, rule, invariant


class CircuitBreakerStateMachine(RuleBasedStateMachine):
    """Drives random sequences of success/failure through the breaker;
    asserts state-transition invariants hold throughout."""

    def __init__(self) -> None:
        super().__init__()
        self.breaker = CircuitBreaker(threshold=3, ttl=10.0)
        self.state_history: list[str] = []

    @rule()
    def record_success(self) -> None:
        self.breaker.on_success()
        self.state_history.append(self.breaker.state.name)

    @rule()
    def record_failure(self) -> None:
        self.breaker.on_failure()
        self.state_history.append(self.breaker.state.name)

    @invariant()
    def state_is_valid(self) -> None:
        assert self.breaker.state.name in {"CLOSED", "OPEN", "HALF_OPEN"}

    @invariant()
    def opens_after_threshold_failures(self) -> None:
        # If the last 3 events were failures (no successes in between), state must be OPEN
        last_3 = self.state_history[-4:-1]                    # exclude current
        if all(s == "OPEN" for s in last_3):
            assert self.breaker.state.name == "OPEN"


TestCircuitBreakerStateMachine = CircuitBreakerStateMachine.TestCase

Hypothesis generates random @rule() orderings; checks every @invariant() after every step. Catches state-machine bugs that linear example tests miss.

For async state machines, hypothesis 6.x has @async_rule — coverage is platform-dependent, so check the docs.

Rule 9 — Database — persist failures across runs

hypothesis stores failing examples at .hypothesis/examples/<test_id>/<hash> by default. On the next run, those examples re-run FIRST. Rationale: once you fix a bug, the same seed shouldn't re-fail; if it does, the fix is incomplete.

In CI:

• Cache .hypothesis/ between runs (via GitHub Actions cache, GitLab cache:, etc.). Without caching, CI loses regression history.
• Don't commit .hypothesis/ to git — it's per-machine and grows unboundedly. Use @example for regressions you want PERMANENTLY captured.

Rule 10 — target for guided search

For numerical properties where you want hypothesis to MAXIMIZE / MINIMIZE a metric:

from hypothesis import given, target

@given(st.lists(st.integers(min_value=0, max_value=1_000_000), min_size=1, max_size=1000))
def test_max_finds_max(xs: list[int]) -> None:
    result = my_max(xs)
    target(result)                             # tell hypothesis "bigger is more interesting"
    assert result == max(xs)

target biases generation toward inputs that increase the metric — useful for finding numerical edges (overflow, precision loss). Don't use for boolean assertions; use target to find the WORST case of a continuous metric.

GOOD: full hypothesis-driven test file

# tests/property/test_message_roundtrip.py
from __future__ import annotations

import pytest
from hypothesis import HealthCheck, example, given, settings, strategies as st

from motadatapysdk.codec import deserialize, serialize
from motadatapysdk.domain.models import Message, Topic


@st.composite
def topics(draw) -> Topic:
    alpha = st.characters(min_codepoint=ord("a"), max_codepoint=ord("z"))
    namespace = draw(st.text(alphabet=alpha, min_size=1, max_size=20))
    name = draw(st.text(alphabet=alpha, min_size=1, max_size=20))
    return Topic(namespace=namespace, name=name)


@st.composite
def messages(draw) -> Message:
    return Message(
        topic=draw(topics()),
        payload=draw(st.binary(min_size=0, max_size=4096)),
    )


@given(messages())
@settings(max_examples=200, deadline=500)
@example(Message(
    topic=Topic(namespace="orders", name="created"),
    payload=b"",                                # empty payload edge
))
@example(Message(
    topic=Topic(namespace="a", name="b"),
    payload=b"\x00" * 4096,                     # max-size null payload
))
def test_message_serialize_roundtrip(msg: Message) -> None:
    """For any valid Message, deserialize(serialize(msg)) == msg."""
    encoded = serialize(msg)
    assert isinstance(encoded, bytes)
    decoded = deserialize(encoded)
    assert decoded == msg


@given(st.binary())
def test_deserialize_never_crashes(data: bytes) -> None:
    """Deserialize must raise a typed error or return a Message; never crash."""
    try:
        result = deserialize(data)
    except (ValueError, TypeError):
        pass                                    # malformed input → typed exception
    else:
        assert isinstance(result, Message)

The first test exercises the round-trip property with seeded edges; the second exercises the parser's robustness against arbitrary bytes.

BAD anti-patterns

# 1. deadline=None to silence flakes
@given(...)
@settings(deadline=None)                        # hides actual perf bugs

# 2. assume() filters >33% of inputs
@given(st.integers())
def test_thing(n):
    assume(n > 1000)                            # rejects most; hypothesis warns Flaky
# Use st.integers(min_value=1001) instead.

# 3. Hidden side effects in property
@given(st.integers())
def test_thing(n):
    real_db.insert(n)                           # property tests should be pure

# 4. Asserting on RANDOM output
@given(st.lists(st.integers()))
def test_random_choice(xs):
    assume(len(xs) > 0)
    assert random.choice(xs) == xs[0]            # flaky; non-deterministic

# 5. Modifying inputs in-place
@given(st.lists(st.integers()))
def test_thing(xs):
    xs.sort()                                   # mutates the example; shrinking confused
# Use sorted(xs) instead.

# 6. @reproduce_failure left after fix
@given(...)
@reproduce_failure("6.103.5", b"AXicY2Bg...")
# Re-runs the now-passing seed forever; CI cost; remove after fix.

# 7. No edge-case examples
@given(st.text())
def test_normalize(t):
    ...                                          # never seeds "" or whitespace-only

# 8. Property test with no invariant
@given(st.text())
def test_thing(t):
    result = normalize(t)
    assert result is not None                    # trivial; doesn't catch bugs

# 9. RuleBasedStateMachine without @invariant
class M(RuleBasedStateMachine):
    @rule()
    def step(self): ...
    # no @invariant — drives random; checks nothing

# 10. Mixing example tests and property tests in one function
@given(st.text())
def test_normalize(t):
    assert normalize("X") == "x"                 # NOT generative; should be a separate test
    assert normalize(t) == normalize(normalize(t))

Tooling configuration

[project.optional-dependencies]
test-property = [
    "hypothesis >= 6.100",
]

[tool.pytest.ini_options]
markers = [
    "property: hypothesis property-based tests",
]

# tests/conftest.py
from hypothesis import HealthCheck, Phase, settings

settings.register_profile(
    "dev",
    max_examples=20,
    deadline=200,
    suppress_health_check=[HealthCheck.too_slow],
)
settings.register_profile(
    "ci",
    max_examples=500,
    deadline=2000,
)
settings.register_profile(
    "release",
    max_examples=10_000,
    deadline=10_000,
    phases=[Phase.explicit, Phase.reuse, Phase.generate, Phase.target, Phase.shrink],
)
settings.load_profile("dev")

CI: HYPOTHESIS_PROFILE=ci pytest -m property. Pre-release: HYPOTHESIS_PROFILE=release pytest -m property (overnight run; finds rare failures).

Cross-references

• python-pytest-patterns Rule 2 — parametrize for finite enumerations; hypothesis for infinite.
• python-mypy-strict-typing — st.from_type uses dataclass annotations.
• python-mock-strategy — fakes for impure code under property test.
• python-sdk-config-pattern — strategy for Config via st.builds or st.from_type.
• python-exception-patterns — properties of exception types (every public Raises path raises a typed exception).
• python-doctest-patterns — example-based; complementary, not substitute.
• sdk-convention-devil-python C-14 — testing convention.