The Cycle of Complexity
Complexity can have many different sources. Things do not materialise the way we imagined, experiments fail, new technologies or tools do not prove as useful as advertised. Our understanding of the problem improves overtime, sometimes the problem itself or the direction we want to go changes. Exposing projects to time results in layering of different, even conflicting, ideas. Having more than a single person working on a system only amplifies those effects.
The perception of this process is typically negative. There’s a variety of terms we use to describe the outcome: “accrued complexity”, “legacy”, “technical debt” or more emotional like “a big ball of mud”. But the process of accumulating complexity over time is rather organic, almost unavoidable, unless the project is trivial, brief or fully specified. Frustrating over inevitable is quite unproductive.
One can view the whole process as a natural cycle of complexity. At the start everything is simple, clean and clearly understood. Years or sometimes only months later the same project is unwieldy, burdensome and irritating. While one could slow down this process significantly by exercising discipline and vigour, it can’t be fully stopped. Another approach is to accept the cyclic nature of complexity as a part of life and have a coping strategy.
Everything that’s build will eventually have to be rebuilt. Sometimes a gradual evolutionary process is possible and sometimes starting from a clean slate and approaching the problem with an open mind is the right choice. Going back to the drawing board and questioning the existing assumptions could be extremely fruitful. The key is to carefully balance creating something new, adapting what you’ve got or leveraging what’s out there.
Ultimately constantly adding something new without making space for it is not sustainable. There’s a need for regularly shedding off the complexity accumulated and getting rid of the things at the source of it. Such an approach has a very powerful outcome - it gives vantage points for reversing decisions. Confidence that all the decisions can be amended allows to move fast and learn quickly.