202402251455 Seeking the wrong goal
Seeking the wrong goal is a particularly deadly trap for systems. One of the most powerful ways to influence a system is to define its purpose or goal. If that definition is incomplete or inaccurate, it could be satisfied aggressively by the system without achieving the actual spirit of the goal.1
If we want to increase the quality of a software system and define the goal to be measured by the number of pull requests reviewed, we may get a system that rewards and optimizes for more pull requests without any increase in quality. We might profess to be increasing national security by increasing military spending but all we're measuring with spending goals is whether the money is spent, not whether we're more secure.
In order to fix this trap, we specify goals completely and accurately. If we are unable to do that, we need proxy metrics or indicators that truly measure the desired state of the system and can only be satisfied if that goal is progressing. Otherwise we'll end up with a system that produces a lot of effort without a lot of results.
-
Meadows, D. H., & Wright, D. (2011). Thinking in systems: A primer (Nachdr.) (pp. 138–140). Chelsea Green Pub. ↩