Why TDD Enhances Developer Productivity

An exploration through the lens of the DevEx framework and cognitive psychology.

Test-Driven Development (TDD) is a design technique involving a specific sequence of steps: writing a failing test, implementing the simplest possible code, and refactoring the code. While TDD is often perceived as a design technique, I propose a new perspective: viewing TDD as a productivity tool. Practising TDD, I argue, will improve your productivity.

Before diving into how TDD enhances productivity, it’s crucial to understand what productivity entails for developers. The idea of how to measure developer productivity has often been misunderstood. Fortunately, Noda et al. recently published a paper that provides a new perspective on developer productivity. The team who publishes the paper includes experts on developer productivity, including Nicole Forsgren (co-author of Accelerate) and Margaret-Anne Storey (co-author of SPACE).

The paper includes the proposal of the DevEx framework, which outlines the three dimensions that affect developer productivity: feedback loops, cognitive load, and flow state. Instead of summarising their findings, I will take the excerpt from

Abi Noda

‘s newsletter post, which succinctly explains these three dimensions:

1. Feedback loops refer to the speed and quality of responses to actions performed. Fast feedback loops allow developers to complete their work quickly with minimal friction. Slow feedback loops, by contrast, interrupt the development process, leading to frustration and delays as developers wait or decide to switch tasks. Organizations should shorten feedback loops by identifying areas in which development tools can be accelerated (e.g. build and test processes, development environment setup), or human hand-off processes improved. 

2. Cognitive load encompasses the amount of mental processing required for a developer to perform a task. When cognitive load remains high as a result of problems such as poorly documented code or systems, developers need to devote extra time and effort to complete tasks and avoid mistakes. To improve developer experience, teams and organizations should aim to reduce cognitive load by finding ways to eliminate unnecessary hurdles in the development process. 

3. Flow state is a mental state in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment. Frequent experiences of flow state at work lead to higher productivity, innovation, and employee development. Similarly, studies have shown that developers who enjoy their work perform better and produce higher quality products. Teams and organizations should focus on creating the optimal conditions for flow state.

The three dimensions of the DevEx framework highlight the areas organisation can focus on to improve developer productivity. However, the limitation of this study is its focus on organisational or team-level change, which is externally focused from the developer’s point of view. There’s room to have a more internally focused point of view, drawing what we can learn from these dimensions into our own individual productivity.

To manage these dimensions effectively from an individual level, we need to understand what limits us from an individual perspective. What limits individuals are the process that goes in our head, our cognitive limits. Understanding our cognitive limits is beneficial as it allows us to see why adopting TDD can be helpful.

I don’t use a hammer because it’s a great tool to drive nails. It’s because I can’t drive nails without a hammer. Similar to our minds, we require mental tools to drive our cognitive tasks. Without tools, we may not visibly bleed, but we may experience an unproductive day filled with frustrations.

Based on my experiences, these are the developer-centric way of summarising our cognitive limits, the three Ws:

1. Wit

   Our mind operates in two systems: System 1 for fast, automatic, emotional thoughts, and System 2 for slower, more deliberate thinking¹. Developers tend to use System 1 by mistake to make important decisions. As a result, their quick wittiness will create a clever solution that doesn’t solve a problem. These clever solutions will increase the cognitive load on our future selves and others, and slow down our feedback loop when others review our work.

2. Working memory

   Our working memory typically has a limit of four², measured in chunks³. Developers handle diverse tasks that demand complex thinking, such as user needs, programming language, architecture, refactoring, testing, etc. With our working memory limits, simultaneously thinking about all these thoughts is not feasible. For example, thinking about user needs and refactoring simultaneously will introduce an unnecessary cognitive load.

3. Willpower

   We have limited willpower that we can use in a day. When you’re about to start coding, the feeling of dread may happen when you have run out of your willpower. The constant use of willpower eventually depletes it, leading us to the loss of joy or flow state.

These cognitive limits are hindering the dimensions outlined by the DevEx framework. Given this understanding, it’s clear that we need a mental tool to help with our productivity. Think about it: Usain Bolt, the greatest sprinter of all time, is slower than a domestic cat but faster if riding a bicycle.

---

The mental tool that we need to circumvent our cognitive limits is workflow. I’m not talking about an organisational workflow like how we pass on work to others. Rather, it’s an individual’s workflow, the sequence of steps of work that you do.

You might be surprised to hear how an individual’s workflow can circumvent our cognitive limits, so let’s consider a real-world example highlighting the importance of an effective workflow: the story of Niklas Luhmann. Luhmann authored more than 70 books and 400 articles on various subjects. His secret was a unique workflow called Zettelkasten, which enabled him to organise and connect his thoughts effectively. The amount of writing he had written is not only a matter of quantity but of quality. His workflow has given him abnormally impressive productivity.

Do you know why his workflow, Zettelkasten, is not widely adopted? When he shared Zettelkasten, the world of writing thought it was too simplistic to be true to adopt. Unconvinced writers believe that great outcome requires great effort, or a complex problem requires a complex solution, which is untrue.

In software development, we often fall into a similar trap as those sceptical of Luhmann’s Zettelkasten. The process of TDD may seem overly simplistic. Yet, we often neglect that this seemingly simple approach can circumvent our cognitive limits:

1. TDD forms a positive feedback loop

   Each of the changes in the test from red to green represents a feedback loop. When the test passes, we know our implementation works. This short feedback loop allows us to observe a sign of progress. The regular and fast signals of the progress we get are the best kind of feedback because it motivates us to carry on working, forming a virtuous cycle, and reducing our reliance on our limited willpower.

2. TDD manages cognitive load

   What’s the simplest way to make this failing test green? That’s the question we hold when we move from red to green. This question forces us to use System 2, the more deliberate mode of our thinking, therefore circumventing the use of our wittiness by mistake.

   TDD’s Red > Green > Refactor structure helps us form a chunk of the diverse knowledge we need to use, allowing us to pull the right knowledge into our working memory at the right time. This helps us manage our cognitive load. See the illustration below.

   

3. TDD promotes flow state

   Two prerequisites to experience flow state are clear goals and immediate feedback⁴. Compared to having a big goal, such as having a feature in the hands of the customer, the immediacy of feedback by practising TDD, such as seeing green passing tests, help satisfy these prerequisites for us to experience flow.

In conclusion, TDD is a powerful tool that circumvents our cognitive limits, improving all the DevEx dimensions of productivity. I hope future studies around TDD will look into its productivity element through these three dimensions and developers’ subjective experience.

Although this essay focuses on improving individual productivity with TDD, it’s important to stress that improving developer productivity through an organisational lens should also be a key consideration. Improving one or the other is not mutually exclusive.

If you like this post, you may like my series Test First vs Last, where I explore how the sequence of writing tests affects our minds.

---

Quantum Steps exists to transform our approach to work for the better. If you discovered something new or love what you’ve read, please do me a favour, and share this post with your friends. Thank you.

Special thanks to my friend, who prefers to keep his name hidden, for thoroughly reviewing and criticising this essay.

1. Dual systems theory: Two systems

2. Magical number four

3. Chunking

4. Components to experience flow