As a good software engineer I have written lot's of unit testes (my lawyer told me to state this).
Recently I have discovered the hype train of PBT, DST and all that (thanks Quint and Antithesis :D).
It's a great hype. I am sure of it since I discovered some fundamentally flawed assumption of mine with regards to software verification.

Why test coverage won't save you

When ever I archived it, I took great pride in telling everyone: "Look I fully covered this implementation. It has 100% test coverage!".
I just now realized that this is not equivalent to having tested my code/software/system in every state.
Regular code coverage does only care about what lines where traversed in any of your tests.
It does not care if you have traversed all branches in your logic through all possible paths.

This means that your codebase with 100% code coverage still can have lot's of states[^1] that are not covered by your tests.

Profound. Stupid me. Luckily I realized now.
Unit test coverage measures only what branches if the decision tree (that your software encodes) is covered by your tests.
It's not a metric for how many of all the possible variant's you have tested for, that can result in each branch being traversed.

Let's have a look on a visual attempt to explain this:

That's quite something.

But don't let this disturb your Zen.
Seemingly we all had the Yin part of testing. But for full on harmony we need a something complementary.

PBT: The missing Yang

This beautiful acronym stands for: Property-based Testing.
It's the thing that allows you to do stuff that you maybe thought you need a fuzzer for: generate test-cases with random inputs that adhere to provided constraints.
Here an example test with Antithesis' Hegel PBT framework:

use hegel::generators as gs;
use hegel::TestCase;

#[hegel::test]
fn test_respects_lru_capacity(tc: TestCase) {
	let capacity = tc.draw(gs::integers::<usize>().min_value(0));
	let mut cache = MyLRUCache::<String, i64>::new(capacity);

	let entries = tc.draw(
	    gs::vecs(gs::tuples!(gs::text(), gs::integers::<i64>()))
	);
	for (key, value) in entries {
		cache.put(key, value);
	}

	assert!(cache.size() <= capacity);
}

Amazing, right?

Yin meets Yang

For me the role of static tests now clearly is to specify test-cases for obvious extremes and saddle points of the inputs.
Property-based tests then fill in the "boring range" between those.

Parting words

"The more I seem to know, the less I think I know."
That's it. Do you testing. Think about it deeply. Check-marks are cool, but not the goal of testing. Actually prove something with your tests, not just that your code compiles.
All of this is a note to myself only - of cause.

Footnotes

[^1]: you can read states here like: "Data permutations that lead to the traversal of certain branches"

Title picture: "Lemon Meringue Pie Slices" by Wayne Thiebaud (1990)