202109251100 Verify behaviors and assert properties

Software systems should have verifiable behavior and assertable properties. These are vital to the maintainability but moreover the evolvability of a system. Moving behaviors and properties up through the chain of verifiability and assertability is required for 202109251102 Reclaiming software systems.¹

Behaviors

Behaviors are aspects of software that can be validated empirically against a specific environment.

1. Observed — behaviors you notice in your logs
2. Tracked — behaviors you get informed about, but only after a change to the software is deployed to the environment. Tend to degrade over time without a dedicated org-led program ensuring their upkeep. Architectural change is preceded by a period of fixing regressions in tracked behaviors before the intended change can be made.
3. Verified — behaviors you can rely on being consistently true because they can be checked pre-deployment. Three common ways to verify behaviors: E2E tests, fault injection tests, and load/performance tests. Can be annoying to maintain verification status (including needing an attempted deploy), but help us design around them because we can depend on them being true.

Properties

Properties are aspects of software that can be validated statically or through local exercise

1. Intended — properties that you ask people to maintain. These are often "enforced" in peer reviews or best practices docs.
2. Tracked — properties that are reported on without an enforcement mechanism. Many companies put code coverage in this category and don't block deployment for coverage reasons.
3. Asserted — properties that are enforced before deployment begins, typically through static analysis, linting, or tests.

Properties can be actively harmful at worst but the easiest to design around and provide the best DX when they're working at their best. A great example of how asserted properties are even more powerful than verified behaviors is static typing in a large codebase. Static types provide an immediate feedback loop on several categories of errors that would otherwise require a full deployment cycle to get feedback on.

It's shocking how many companies have zero verified behaviors and asserted properties, not because they're hard to implement, but because they've never considered just how powerful it is to have software they can reason about.