Frustration is a common reaction among engineers new to AI. Why is AI not telling me the truth? Why did AI lead me down the wrong path? Engineers need to build a good intuition about AI for successful adoption, and we could achieve a better intuition by unpacking the magical black box of AI.

Building this intuition is important as the non-deterministic nature of AI often makes engineers operate, in Cynefin terms, in a Complex domain, where the cause and effect relationship is only partly visible. With better intuition, we understand better what we can and can not do against AI. A better intuition makes part of the work sit in the Complicated domain, where teams could analyse patterns and make improvements to their approach to AI.

A better intuition of how to use a technology effectively can be done by peeling the technology’s abstraction one level deeper. When my daughter got frustrated because the TV remote did not put the volume up, we went one abstraction level deeper. Understanding that the remote control blasts infrared light that needs to be received by the TV, built her intuition. She needs to point the remote at the TV's infrared receiver.

A similar idea can be applied to understanding AI chatbots and agents, too. Just like the TV remote control, we don’t have to go too deep into what infrared is. Even though the topics of attention and transformers are interesting, you don’t have to go that deep to build the initial intuition.

What follows is a blend of the learning hour I’ve designed, my thought process, and the result of the learning hour (Hat tip to Pete Hodgson, who has greatly influenced the content of the workshop.)

LLM inputs and outputs

We kick-start the learning hour by focusing on LLM. Simple boxes and arrows are engineers’ favourite tools to understand abstractions. Let’s use that. I deliberately exclude coding agents or AI chatbots from this initial picture.

This idea is an oversimplification at best, but one thing I emphasise over and over again throughout the learning hour is the idea that there are only two inputs to LLM to produce output (next tokens). Arguably, the training data could be framed as a database, but it doesn’t really matter at this point. I thought this oversimplification was necessary to build the foundational understanding of LLM.

Like any technology, we learn best when we can play around with it and observe the input and output in real time. This has proven to be difficult, as many AI tools out there have built a thick harness around LLM. Luckily, I found this fully hosted GPT-2 app that I could use for the demo. I got an ooh and an aah at this point. Something clicked in people’s heads.

“But AI chatbots don’t seem to respond to me with next tokens”

We understand now that a language model's job is to produce the next tokens. However, it does not explain how AI chatbots work. Clearly, these chat assistants are responding to us conversationally. They’re not completing our sentences like what’s been demonstrated so far.

Think about it for a second. Imagine that you’re sending a prompt to an AI chatbot, “What’s the capital of”, the chatbot wouldn’t complete your sentence. It might even ask a question back to you instead and make an attempt to have a conversation.

Here, I asked the team to spend 5 minutes hypothesising how an AI chatbot might work, hoping for them to build a stronger learning connection. Unfortunately, I didn’t get any good answers as my questions were poorly written.

The magic here lies in some invincible texts that got injected into the input sent to the LLM by the chatbot. Our user prompts are modified by the chatbot to influence the response returned by LLM.

I used the same hosted LLM to demonstrate this.

Notice how, in this LLM, getting the LLM to role-play produces better prediction. (Do note that in reality, role-playing is only one trick the harness does. There’s much more to it.)

So, back to the original question, why is an AI chatbot not telling me the truth? Why is it hallucinating? In this LLM example, even with the role-playing trick, it would have told me that the capital of France is France!

By this point, hopefully, the team would have built enough intuition to answer this question. And the next time they send their prompt, they could visualise what’s going on behind the scenes.

What about agents?

Given that the team has been hyped-driven, uses a plethora of coding agents already, I thought it was worth unpeeling the abstraction further for agents. We started by reading the definition of agents by Anthropic’s Hannah Moran:

Agents are models using tools in a loop

All engineers would understand what a loop is, but what is a tool? How does the agent decide on what tool to call? Here again, we’re a bit stuck. Just like AI chatbots, the harnesses were quite thick, so activities are happening behind the scenes that we couldn’t really observe.

Thanks to Pete Hodgson, I discovered that I could observe the tool call in detail in Claude Workbench.

And before we close, I reiterated the idea that the only LLM input we could control is the context, and the reason LLM understands this specific tool is that the detail has been injected into the context window.

This is the final boxes-and-arrows diagram.

Closing

We first closed this learning hour with a quiz, as I felt that the activities were fairly theoretical. I’m pretty pleased with the results.

And with the quiz answered, we closed it with reflective questions. The answers made me pretty confident that we’re making good progress in this AI adoption journey.

Speaking of good progress, next up, I’ll write about what I’ve decided to measure. Stay tuned.

Our AI half-hype-driven team is now at the beginning of our adoption, or rather, formalising our adoption. When I was asked to get this initiative going, I knew clarifying our AI stance should be our first step. This step was inspired by the DORA AI Capability Model, which is called “Clear and communicated AI stance,” which is pretty well described by the name.

Having a clear stance helps reduce the ambiguity that our team members might have. Are we going to lose our jobs? Are we going to get replaced by machines? Would our current skills be irrelevant? It’s hard to tell if anyone in our remote-first team is anxious around these topics. However, it’s not hard to imagine people having such a prediction and developing their own fears.

We begin our journey through a 30-minute workshop.

Where are we heading?

We kicked off the workshop by having our Head of Product Engineering act as a sponsor to endorse our AI adoption initiative. Our sponsor mentioned that AI is here to stay, and let’s not fall behind. The goal that we set by this point is a directional one, and we did not declare where the finish line is, by intention.

As much as I dislike any top-to-bottom approach, a power dynamic exists in any organisation, and we have to attack this adoption from all possible angles. This three-minute endorsement served as an enabling constraint, as it gave an explicit permission for the team to adopt AI. We made it firm that we, as a company, will go through this together and learn new skills around AI if we have to, tightening our stance on AI.

Where are we?

While setting a goal could be aspirational, as a student of complexity, I thought it’s more important for us to understand where we are as an organisation. We are unlikely to get to where we want to be if we don’t recognise our current dispositional state. To surface signals about our current dispositions, I created three Likert-scale statements using a screenshot from the recent DORA report:

As you can see from the result, thankfully, we have quite a good starting position as an organisation. We did not have strong disagreements around the statements presented.

I recall having a conversation with our sponsor where he suggested us having a strong transformation and training programme. Having encountered charlatan coaches in the past gave me a hard knee-jerk reaction. I promised him that we’re not as bad as he thinks we are, and this survey should reassure him.

How do we move forward?

It’s clear now that we’ll adopt AI, but some question marks remain. How do we adopt it safely?

There were worries about our use of multiple AI vendors and the impact of this practice on the business. A couple of us have been experimenting with various AI agent tools with the belief that it’s best to let engineers use the tools they like. We’re scrappy, and we disliked the idea of company-approved tools. These vendors are releasing new models every minute. We should always use the best one, should we not? There was an argument for consolidation for cost reduction (which I thought was a weak argument).

With this context in mind, I revealed the next set of Likert-scale statements in the workshop. To make the workshop more productive, I prepared the questions to be more concrete, articulating the worst and best outcomes of consolidation, expecting to spur some debate. I thought this was more productive than asking a generic, should we consolidate question.

One of the most important outcomes of the debate was that our business does not have any special secret, patented source. We, as a business, are not too concerned about the use of LLM against our codebase.

We also discovered a new tension, which was revealed by the third Likert statement. Some of us still like model hopping and are attached to our tools. We did not resolve this tension in the workshop as we ran out of time. The good news was that I did not see strong disagreements.

Concluding the tension

Seven days after the workshop, after thinking and researching, I’ve decided to make a Request for Comments to resolve the tension.

Below is the original RFC I’d written for the team:

We’re going to invest more in Claude Code as our coding agent (full Anthropic stack, agent, and LLM), so we can adopt it well as a team.

What matters most: Brand trust and adoptability, Agent and model compatibility, One tool to share

1. Brand trust and adoptability

This one is subjective, but I personally trust the Anthropic brand more than OpenAI. They’ve positioned themselves around safety since day 1. I’m also cautious about any company whose business model is strongly tied to advertising. More importantly, regardless of vendor, I want us to be confident about clear enterprise/API data guarantees—especially around retention and whether our data is used for training. As we onboard more engineers, we’ll naturally have more opinions. In the industry, there are also strong feelings about leadership (Sam Altman) and company reputations, and that can affect adoption for tools we standardize on.

2. Agent and model compatibility

We learned today that context is king for influencing an LLM’s response. (Editor note: We had just finished a LLM learning hour when I posted this RFC.) Picking an agent that’s designed around the model should produce better results. Different models respond better to different conventions (e.g., structured tags vs. plain text), and they behave differently under the hood. Claude Code, for example, can inject system reminders into the context window, like:

<system-reminder>

Note: [FILENAME] was modified, either by the user or by a linter. This change was intentional, so make sure to take it into account as you

This is a similar trade-off to adopting hybrid cloud vs. going all-in on a single cloud provider: tighter integration can be a big advantage, but it comes with more coupling.

3. One tool to share

All of these tools inevitably have different configs and different ways to make usage safer. The community is usually behind on sharing consistent standards (think “skills” protocols, reusable workflows, etc.). It’s also easier to share concrete knowledge when we’re all using the same default tool, rather than translating ideas across multiple setups. I know many of us are trying different models. In my head we’re hitting Fredkin’s Paradox:

The more equally attractive two alternatives seem, the harder it can be to choose between them—no matter that, to the same degree, the choice can only matter less.

My hypothesis is we’ll get way more benefit by using one vendor really, really well as a team. I’m not saying we’re banning Cursor or other tools right now. You’re still free to explore. I’m saying: we’ll pave the default path toward Claude Code and its surrounding tooling so the team can align and compound improvements.

I had good feedback from the RFC. Our sponsor, who had a strong desire to consolidate, approved it, which is great news.

The idea of forced consolidation was one of the biggest worries I had. A paved road would form a permeable constraint, not a rigid one. In a complex system such as ours, a rigid constraint might not be the best one, as it may create shadow IT and resistance.

We also had an engineer who was worried about our Cursor subscription being stopped, to which I responded, that’s not what’s happening.

In making this decision, I mostly thought about our junior members whose progress might stifle with the idea of multi-tool-multi-model chaos. I know that the cool kids have moved on to gastown and codex, and I have a burning desire to try them out, too, but it wouldn’t have been the right call. We have to get everyone on the journey together.

If we get this right, this decision should kick off a flywheel of deeper learning in the teams, sharing tips or tricks for Claude Code, rather than sharing hypes.

Closing: This concludes our first step of clarifying our AI stance, turning the adoption from a set of hype-driven experimentation into the start of building an organisational capability.

As I’ve been writing documents productively at work through voice-to-text and LLMs (or rather, productively generating tokens), I realise my thoughts are not as clear as they used to be. When my mentees asked for advice, I found it harder to articulate and form my thoughts live. This realisation is immense. I’m now jotting each and every letter on my keyboard again for this post to force myself to think about each of the tokens I’m about to generate from my mind.

Why this series now?

Regaining mental clarity works as the inner force, but the external forces for me to write about this series are even greater. As the industry is moving fast, the force for business leaders to increase the team’s throughput through AI adoption is strong, and I’ve been asked to help with that.

As I go through the journey, I struggle to find content that I find relevant. A lot of the contents are pretty focused on greenfield projects, prototypes, and new LLM models and features. I’ll write about what I can’t easily find, i.e. work experiences and challenges which are grounded in reality.

Is there a value in capturing experiences? I’ve managed to develop injuries on my forearm through ashtanga yoga and badminton. Thanks to a random stranger sharing their experience on Reddit, suggesting me to try Tyler’s Twist exercise, I do feel better now. Before I discovered that post. Feeling irritated that I can’t hold Tolasana and swing my racket without pain, I asked ChatGPT to do deep research, dig up all the bloody anti-fragile science papers it could, and give me the one single thing I should do. No, none of them helped, and yes, I suffer from an episodic scientism now and then.

That Reddit post was about tennis elbow, and even though I do not actually have tennis elbow, the exercise is helping me. That lack of causal relationship is the nature of a complex system, which would also be present in a team setting. A failed experiment in one team could be a successful one in another.

The team’s context

As I write the future posts in this series, understanding the context that I’m operating in is important.

Our codebase is about 2.5 years old, accumulated tech debts through constant change of business priorities. We work in the telecom industry. We’re delivering sustainably, deploying every day.

We are a company of nine engineers, organised into three small outcome-focused teams. We’ve got a good spectrum of juniors and senior+ engineers in the team. We are a remote team, with engineers in the US, the UK, and India.

All of us are in a different phase of AI adoption. Some are a bit more conservative than others. Some are a bit more hype-driven, pushing for multiple agents and switching models as frequently as they can. Some would think that we’ll lose our jobs in 6 months or that junior engineers would cease to exist. I probably write 95% of my code through agents, and some are still at 0% as they’re still using AI chat assistants.

A few of us have been in an indefinite state of experimentation, trying things out for about a year, while some are lagging.

A few of us have used agents, then started to move back to coding by hand.

I’ve reviewed LLM written PRs that have not been reviewed by the authors.

A beautiful mess.

Defining the series

I hope that the term “AI-enabled teams” is self-descriptive.

I originally liked the term AI-assisted engineering, as it’s also used by DORA when they published their AI-assisted software development report last year. However, as I’ve been adopting agents more, I don’t think the word ‘assist’ is fully capturing the evolution of where we are now.

Some used the term Agentic Coding, but this sounds like a specific activity within the umbrella of software engineering. The industry is moving so fast that agents might become obsolete.

AI-native is an interesting term for me, and many are using it. It sounds bold, as it implies that there exist brand new engineering practices could not live without AI. The heuristic being, if you subtract AI from your system now, would it still operate? I found this hard to believe. I believe that all the evergreen engineering practices like TDD and Continuous Delivery will hold regardless of AI, if not becoming more important.

I still don’t like the word AI-enabled, but it’s the least worst option I have.

Teams implies the idea that we have a shared codebase where multiple engineers are working on it together. It’s my belief that this idea would not go away anytime soon. I believe that if you want to build something great, you’ll need to put multiple minds on it.

While there is a lot of excitement around what the tool could do, software development is still software development, and we have to deal with all the anthro-complexity around it.

It did come across my mind, why not let each of the engineer in our team build their own microservices in their own repo, agenting away? Let’s not get there for the time being.

What to expect in the series

The posts would be a mixture of essays, our progress, and some dojos or learning hours that I have organised for the team and some commentaries around that.

This is not a series where I’ll help you replicate our AI adoption journey, in fact, I don’t even know what our outcome would be. My intention is to share our effort and results in the hope that you could help your team, too.

Subscribe now

Is the delay in reviews always about the size of the pull requests? Maybe, maybe not. Just the other day, a colleague cried, “I submitted a small pull request, and it still takes 3 days to be reviewed!” Meanwhile, the work in progress started piling up.

These are frustrating experiences. We know what good practices should look like—Continuous Integration (CI). One key aspect of CI is that engineers should commit to the mainline at least daily. Delays in reviews, like the examples above, are counterexamples of CI.

While describing what good looks like is important, it’s equally crucial to define what we should avoid. We have a term for the CI counterexamples: CI theatre, where a team assumes they’re practising CI just because they have set up a CI server, ignoring the frequency of commits to mainline.

I think the term CI theatre is insufficient for us to shift our behaviours, as it doesn’t describe the issues caused by such behaviours. When engineers submit large pull requests or delay reviews, they often don’t realise the impact. They’re blocking production. Value to customers is blocked. I want to have a clearer way to describe this, especially on the individual level.

I suggest the term Production Blocker, an individual who plays well in a CI theatre.

When I procrastinate on reviewing large pull requests, I ask myself, “Do I want to be a Production Blocker? No, let’s review it.”

This idea shouldn’t be used to point fingers but used as a personal anti-goal instead. Anti-goals are a powerful concept that applies the idea of inversion to goal setting. I’ve seen this idea being used by Jack Dorsey with his ‘won’t do list’1. My colleague recently shared TypeScript Design’s ‘Non-Goals’2, which are similar.

To better understand the anti-goals of a Production Blocker, I’ve described them in a ladder format:

Junior Production Blocker

As a reviewer:

• Don’t bother checking your inbox for review requests. Your sole purpose is to code; leave the inboxes to the seniors.

• Approve everything without reading the code to ensure potential issues go straight to production.

As a changer:

• Merge your code on broken mainline builds. Once approved, don’t check the build status—just merge. As long as your code is in the mainline, your work is done.

• Don’t review your own pull request. Let others discover your typo, temporary logs, or unfinished work.

Senior Production Blocker

As a reviewer:

• Comment on every minor style issue and nitpick relentlessly. Don’t allow merging until all trivial issues are resolved.

• If your teammates ask if they can merge first and resolve later, firmly say no.

• Delete review request emails because your current work is far too important to be interrupted.

As a changer:

• Do not add any explanation, context, or screenshot to your pull requests, your code is self-explanatory.

• Submit only large pull requests. Big tickets need big pull requests. The larger the better.

• Never revert your changes if they break the mainline. Spend a week investigating the root cause instead.

Lead Production Blocker

As a reviewer:

• Be the only one who can approve pull requests, no one else is qualified. Make others wait, even if you’re on holiday.

• Provide overly detailed feedback. If the changers don’t understand the feedback, close the pull request, and complete the change by yourself.

As a changer:

• Focus solely on what you’re working on, and reject every collaboration attempt from your teammates.

• Reject all review feedback from your teammates. They are not mature enough to understand your superior code.

Principal Production Blocker

As a Principal, you transcend the roles of reviewer or changer. Your mission is to block the root of all production by taking it to the psychological level, you block people’s minds. Production Blocking3 is the psychological term used to describe the tendency of one person to block other people from offering ideas, that’s your ultimate goal.

In a group discussion, always be the first one to speak and make sure you win by getting the last word4. If the discussion lasts an hour, talk for the entire hour, and block production. Suck all the oxygen in the room. When people interrupt, you’ll say “I’m not finished!” This way, no pull requests are even possible to be submitted.

What else would you add to be a successful Production Blocker?

Subscribe now

In software design, a deeper understanding of a problem is important. Gaining that deep understanding will also need to be efficient, as domain experts are typically busy. Through my experience in Eric Evans’ Software Design Masterclass, I learned a small yet impactful step to gain a deeper understanding more efficiently.

During a conversation with a fellow attendee at the DDD Europe 2024 conference, I shared this significant insight about concrete examples. “Well, that’s pretty obvious isn’t it?”, they responded, highlighting a misconception I believe needs addressing.

If this technique was truly obvious, Eric wouldn’t have felt the need to emphasize it repeatedly throughout the two-day course. His constant reiteration reveals how deeply this idea is engrained in his head as a heuristic. It’s so pervasive that I noticed he repeated the phrase ‘concrete example’ in his unrelated closing conference talk, which was about LLM and DDD.

The effectiveness of a concrete example became clear during our exercise. Consider a simplified snippet of the interview we did with Gien Verschatse, who acted as a domain expert in the class:

Eric: “Alright, class, remember, ask for concrete examples.”
Gien: “The problem is the fridge shows an error when I scan my plate.”
Class: “How does the fridge operate?”
Gien: <Long irrelevant explanation>
Class: “How does the barcode look? Is it a QR code?”
Gien: <Long irrelevant explanation>

Eric stepped in, “Could you walk us through a concrete example of when this problem happened?” Suddenly we were getting a great detail of what the problem is. I could notice the class were having a lightbulb moment, we understood the problem deeper now that we could come up with a better solution.

The first part of the phrase ‘concrete example’ is the word ‘concrete’. The word concrete implies a real-world example, not a hypothetical or abstract one. While abstract examples can enhance our understanding, a concrete example keeps a discussion more grounded in reality, therefore we’ll be more confident that we’re solving the right problem.

The second part of the phrase is the word ‘example’. In my opinion, this word requires more emphasis given none of the class participants asked for any form of examples in the exercise.

Explain, example, extrapolate, all these words share the same prefix Latin root that implies “out of” or “from”, to mean a process to bring something to light, to make something clear. Giving an example is a process of making something clearer by representing a single item from a larger whole to illustrate a point.

On reflection, I realised how obvious it feels to explain concepts using examples, yet how non-obvious it often is to ask for them. I found it easier to help my eight-year-old daughter understand the concept of inertia through examples. In the London Underground, I would point out her experience when the train is about to move or about to stop, as an example of inertia, providing a concrete point of reference to a more abstract concept. Examples helped her gain that understanding.

Building on this, let’s consider the word ‘dog’. Imagine how you would explain this abstract concept to a child, without examples, it would have been challenging. Examples are fundamental to our knowledge acquisition process. My daughter’s understanding of dogs expanded through a series of examples and counterexamples, such as mistaking a cat for a dog.

While it may seem unintuitive to ask for concrete examples initially, integrating this practice has proven valuable, even when I felt that I understood a particular problem. For instance, in a recent conversation with our team:

Product Manager: “I’d like a single analytics dashboard, so I don’t have to look to multiple places to analyse our customer conversion.”
Team: “We’re going to need to adopt a data platform. We’re worried about the associated cost.”
Product Manager: “I think we need to invest in a data platform, it’ll allow us to collect and gather more data in the future.”
Team: <Long discussion and conversation>

Notice how even though we were discussing a problem, the problem was not concrete enough. This is exactly what happened in Eric’s class. The team were trapped in the discussion that didn’t deepen our understanding.

I asked for a concrete example, “Could you show us, share your screen, what is it that you’re doing, and found painful?” Upon seeing the real-life demonstration, the team immediately had a clearer understanding of the problem. We also started to think like a designer, we dug deeper into the pain points. We ask, how often does our Product Manager need to do this?

At the end of the discussion, we thought a simpler solution would work so long as we make it evolvable. Better yet, we thought the problem wasn’t painful enough for now we deferred from solving it.

Ultimately, grounding our understanding of problems through concrete examples anchors our discussions in reality. This technique proved invaluable in efficiently comprehending a problem.

If enjoyed this post, you might enjoy these other DDD posts and my deeper thoughts about the intuitiveness of TDD.

Like any other practices or techniques I learned, I’d like to validate my understanding against the source of truth, and I can’t get closer to the source of truth than attending Eric Evans’ class, a pre-conference workshop in DDD Europe 2024. It’ll be an opportunity to observe the original author of the big blue book, the man himself, operate in real life.

Validating against the source of truth is important for me to reduce misinterpretation. For example, I joined ThoughtWorks 7 years ago to validate my understanding of various Agile practices. I was first exposed to Domain-Driven Design (DDD) at around the same time, but unsatisfactorily until today, I haven’t felt that I have validated my understanding of DDD. That’s what I aimed to do in this class, uncovering Eric’s secrets.

This brings me to the structure of the class. The class was running for two days. There were 2 instructors, Eric Evans and Gien Verschatse, and there were roughly about 15 participants. Gien wore two hats, she’s a software engineer, and she’s also playing the role of a domain expert in DNA building or strain engineering, a domain she worked in for 5 years. Eric, and all of us participants, were to knowledge crunch and model this problem.

I have zero experience in DNA building. The domain is fascinating, but I had to remind myself that I paid a significant amount to attend this class. This means I’m more interested in “domain modelling” Eric’s thought processes than the problem domain itself. I kept 80% of my energy observing Eric.

Given how unstructured the class is, I told Eric that I’m struggling to domain model his head and DNA building together. I’m glad I did that as despite my best effort to read his mind, it was easier when he started to add commentaries about his thoughts.

There were many observations and learnings I’ve made, but to cut the length of this narrative essay, I’ve decided to scope it to knowledge crunching, the hardest, crucial, yet often neglected skill to master in DDD.

Expertise in listening

Despite the plethora of techniques and methods in the DDD community, such as Event Storming, Eric didn’t use any of them. At some point, a group in the class were debating on what techniques to use, and they received advice to use whatever that works for them.

There wasn’t any specially DDD-named technique he was using. Eric spent most of his time sitting and listening. He didn’t hold a notebook. He didn’t draw anything. There were 5 instances where he would stand up, show a gesture of having a light bulb moment, and write a short sentence on a whiteboard. Apart from that, he’s primarily listening.

What strikes me is how good his intuition is for what’s important to capture and expand. He shared his heuristic: “What’s important you’ll hear over and over again.” He’s able to separate signals from noise extremely well.

When a participant asked, “Could we solve the problem by defining DNA pipeline declaratively?” Eric noticed Gien’s reaction, her eyebrow raised, lost in words. Then Eric intervened and suggested using language that resonated, the word declarative was too technical. He’s actively listening, and paying attention to subtle cues.

Expertise in interviewing

Listening is one expertise, he’s also demonstrated how to ask great relevant questions at the right time. During the class, it’s becoming apparent that the participants tend to try to go deep into irrelevant points or understand everything at once.

Like any interview, time is of the essence, and going deeper into an irrelevant topic will waste everyone’s time. In fact, he mentioned that “time” is always in his head, so he wants to be careful about how he spends time with a domain expert.

The room often jumps ahead and talks about solutions, and Eric often refocuses the room on pain points, rather than solutions. I have observed a similar tendency to jump into solutions from many engineers throughout my career.

At one point Gien uttered so much jargon no one understood anything in the room. Eric caught his confusion and executed a manoeuvre that left the room in shock and awe. The class later summarised this technique as “zooming in and zooming out”, which I don’t think captures the nuances of what he did. In his own words, what he did is to “latch” onto a smaller problem, within the big problem that’s been described. This small problem is not known to him, but he felt that he could understand that better than the bigger problem. Once he understood the smaller problem better, which took some time, he climbed back up to the bigger problem with a newfound understanding.

Expertise in learning

“Is this how you would operate in real life? No tools, no notes, no nothing?”, I asked at some point. The answer was a resounding yes. He further expanded on how he doesn’t default to UML either. Instead, he would learn and absorb diagrams being drawn by domain experts.

He’s deeply committed to being immersed in the domain. Never once he would say that he’s trying to domain model, or he’s trying to knowledge crunch, he listens to a problem, and he’s off having that deep conversation to further his understanding.

Understanding requires elaboration, and I was amazed by how much effort he’s showing in elaborating with the domain language. He’s the first one that would verbalise such a sentence, “So we would use THCAS Forward Primer and Reverse Primer on a plate to amplify the gene through a PCR process to create PGAL1.” (I don’t remember the exact details). As you can see, we were in a different world. As far as I can remember, no participants attempted to elaborate like this. Eric was demonstrably struggling, stuttering, and showing pauses, when he was trying to finish that sentence. The result, he learnt the fastest in the room.

What I love about the themes I’ve observed is how Eric opened up about the fact that we don’t always need to rely on new methods, but we could get better results by being experts in our basics. These basics, the idea that designers should have curiosity and empathy, were shown in all the observation themes.

Unfortunately, though, I felt underwhelmed by what I’ve learnt. These were the things I already knew, these are no secrets to me.

It took me time to realise that there was an observation that I overlooked. It took me 5 days to realise this and I later shared that one key learning with Eric, jokingly during our dinner: “Eric would like to be the first one who kills his baby, and if it’s not his baby, he’ll cuddle it first before killing it.”

This metaphor was used when we talked about how to deal with an old broken existing domain model that has been created, either by ourselves or someone else. Eric made an important point: Most designers could not come up with good domain models because the designers are too attached to what they have produced in the past. Then he further said:

“I want to be the one who comes up with a better model. I want to be the one to find flaws in my model.”

- Eric Evans

That’s the secret I’ve been looking for. I have not thought of approaching a model in that manner.

That statement itself may sound plain, but it expresses his attitude towards domain modelling, the level of care for domain models he conveyed in that sentence is on another level. Two individuals may practice the same techniques, but they’ll show different results when they care.

If you have a deep belief that a better model will help us solve problems better, you’ll do whatever it takes to produce a better model. And in that belief, I found not just a lesson in DDD, but a lesson in the relentless pursuit of excellence.

If you liked this essay, you might like another one about DDD:

For many months, Petruk's primary goal had been to elevate the coding standards of his team. He diligently provided feedback with lists of suggestions, but his team struggled to grasp his comments. They’re all struggling and frustrated. It seems Petruk can see things that others can’t see.

Dewi, his colleague, approached with her calming presence. “What’s that big sigh about, O Petruk?” she inquired.

Petruk explained his situation. Dewi, closing her eyes and nodding thoughtfully, posed an unexpected question.

“O Petruk, do you know how to cross the road?”
”What? What do you mean?”

”Answer the question, do you know how to cross the road?”
”Well, you just… Cross the road. It’s easy.”

”How do you avoid getting hit by a car?”
”Hmm… You cross properly, with attention.”

”Tell that to toddlers. What do you think will happen?”
”Well… They might not understand and could get hurt.”

”O Petruk, tell me, how would you show a toddler how to cross properly look like?”

Petruk paused, then began explaining to Dewi. He mentioned the importance of stopping and pausing before crossing, looking left and right, and being aware of oncoming traffic. He realised that some aspects of crossing the road, like judging the speed of cars or understanding when it’s safe to cross, are difficult to explain in words.

Dewi, still with her eyes closed, looked satisfied with Petruk’s explanation. “Do your teammates know how to cross the road?” she inquired. Petruk wasn’t sure how to answer.

“O Petruk, expertise is knowing what to pay attention to. By using pull request comments, you’re only pointing out the signs along the road. To show others what you’re paying attention to, try crossing the road together.”

Subscribe now

In the following weeks, Petruk started experimenting with different techniques. “I need to find ways to show what I’m paying attention to, “ he thought. He began sitting with his teammates more. He experiments with pair programming. He experiments with live demonstrations, where he’ll sit down to refactor and improve the quality of the codebase while thinking out loud. Rather than leaving asynchronous pull request comments, he experiments with doing a live code review, where his teammates can hear his thought process.

Sometimes, he noticed, his teammates still couldn’t see certain issues, not because they weren’t trying, but because they hadn’t developed the necessary perspective. Just like how even with great technique, a toddler is still not tall enough to see past parked cars. Petruk started to build more empathy, helping them see what he saw when those parked cars were blocking their views.

A year quickly passed. Petruk was again sitting and sipping his cup of coffee, reviewing a pull request, when he let out a big sigh. Dewi noticed, “O Petruk, what’s the big sigh about this time?”

“A sigh of relief”, Petruk responded.

I previously wrote about how attention is one of our cognitive functions.

If you enjoyed this kind of story, you might enjoy:

This reminds me of a deep discussion I had with Ben yesterday. We were looking at a part of our codebase, which I’ll simplify here:

// Version 1

if (env === "prod") {
  return {
    API_URL: "prod-url",
    STRIPE_KEY: "prod-key",
  };
} else {
  return {
    API_URL: "dev-url",
    STRIPE_KEY: "dev-key",
  };
}

I suggested a different approach:

// Version 2

return {
  API_URL: env === 'prod' ? 'prod-url' : 'non-prod-url',
  STRIPE_KEY: env === 'prod' ? 'prod-key' : 'non-prod-key',
}

In Version 1, you might notice the duplication of the `API_URL` line. Reflecting on how I suggested for Version 2, I realise I didn’t use the word duplication. I think this is probably because I know that Ben is a deep thinker. He really digs into the ‘why ‘of things. Simply arguing about duplication might not have been persuasive enough for him.

Instead, I highlighted a potential problem we might face in the future: the risk of forgetting to update the `prod` part of the code when changes are made to the `non-prod` part. From my previous essay, Remove duplication to deliberately forget:

This heuristic, the act of forgetting and remembering, also implies the aspect of time. Time is the primary difference between programming and software engineering. We need to adapt to changing needs over time, and this context of what probabilities might come in the future, helps us make a better judgement.

Subscribe now

Imagine a scenario where the prod section is overlooked during an update, the production environment might break:

if (env === "prod") {
  return {
    API_URL: "prod-url",
    KEY: "prod-key",
    // <--- not updated
  };
} else {
  return {
    API_URL: "non-prod-url",
    KEY: "non-prod-key",
    NEW_BUCKET_NAME: "non-prod-bucket",
  };
}

In this case, Version 1 requires you to remember well. On the other hand, I describe Version 2 as “forgettable code” (I don’t like this name).

I’ve found that discussing this concept with others leads to more agreement, given that forgetting is a human limitation we share. Debates about duplication often end up like discussions about personal writing styles - “I prefer the code to be centralised” vs “I prefer the code to be shorter”. Such discussions rarely lead to consensus.

Of course, we can see the whole code visually, so you might think, “I’ll remember, I can see them!” But our current codebase has examples where the complete prod config doesn’t fit on one screen.

You might further say, “I’ll still remember, even if it doesn’t fit my screen.” However, consolidating new information into our Long-Term Memory (one of our nine cognitive functions) is challenging. We tend to forget over time. If there’s an option that requires you to spend less cognitive effort, why not do it?

The challenge grows when working in teams. You move from dealing with one head to multiple heads, i.e. multiple Long-Term Memories. With Version 1, you have to ensure that everyone remembers to keep the values in sync. You might note this in a Pull Request or add comments to the code as reminders. You know how the story ends, your teammate will forget.

Subscribe now

This forgetfulness piqued my curiosity, or more accurately, it instilled fear, prompting me to delve into books about memory. I remember over a decade ago, I read “Moonwalking with Einstein” by Joshua Foer, the 2006 U.S. Memory Championship winner. Ironically, just days after his victory, he drove his car to a celebratory dinner and took the subway back home, completely forgetting his car.

After devouring many books, I toyed with the idea of returning to university to delve deeper into the subject, but I couldn’t afford it. That’s when I realised that perhaps a textbook is a cheaper path to finding the truth1. I picked up “Cognitive Psychology” by E. Bruce Goldstein, a university textbook. Back in my university days, I’d despise reading a textbook cover to cover, but this time, I finished the textbook (still processing) and felt energised. All the things I wanted to know, they’re all here.

Nine cognitive functions

I don’t think there’s an official list of cognitive functions. The following is derived from the textbook as hinted by its chapters. As you read through this list, imagine that I’m seated in front of my computer, about to make a change to a codebase:

1. I engage my perception and notice multiple problems in the codebase.

2. I pay attention to one problem that worries me the most.

3. This particular problem turns into information and goes into my working memory, ready for manipulation.

4. As I manipulate the information, I rake through my long-term memory to determine if I have solved similar issues.

5. I’ve seen this before. I have the knowledge. This is a code smell. It’s called the long method. And I know multiple refactoring techniques to address this code smell.

6. I use visual imagery, to remind me of the architectural style we’ve adopted, ensuring the solution is aligned to it.

7. I reason and make the judgement to choose which solution is best.

8. Using all the above cognitive functions, I problem-solve, and the problem is resolved.

9. I commit the change, writing the commit message in a language that my future self and others will understand.

Each of the emphasised words represents a cognitive function, I won’t go deep in defining each one here. I have illustrated these cognitive functions linearly for simplicity, but they’re more intricate than that. Consider a recent task, maybe making an architectural decision, and reflect on it, every cognitive function is at play.

Cognitive psychology is the study of the mind. An interesting observation is that these cognitive functions can often be metaphorically replaced with the concept of ‘mind’:

• “I’ve changed my mind” (judgement)

• “If you put your mind to it, I’m sure you can find the solution” (problem-solving)

Subscribe now

How might this be useful?

After reading the illustration above, you might ponder, what’s the point? You might think, I can do my job without understanding all that.

This is where my exploration has led me, and so far, some of my coachees have begun to see its usefulness:

1. Decompose a skill into multiple parts

Observe how a child learns to hold a pencil to write. You can break this down into fine motor skills (the strength to hold the pencil) and recognising the alphabet. Both are essential.

Similarly, in a refactoring exercise, you can dissect the skills into multiple cognitive functions, as illustrated before. This is what Quantum Steps is all about, I want to break things down into tiny parts.

2. Identifying gaps

You may notice that a child can recognise the alphabet but still can’t hold the pencil. The gap here is the fine motor skills.

In the context of refactoring, an engineer might know the steps to refactor but fail to perceive code smells. Some can’t judge well when or when not to refactor. Some architects may know architectural patterns well but fall short in problem-solving or communicating in effective language.

3. Filling the gap effectively

Once a specific gap, such as fine motor skills, is identified, you could strengthen the child’s finger with other toys, which could be more effective as they learn through play, and the skills are transferable to writing.

With the plethora of courses, many people consume vast amounts of content to learn about a topic. Ask yourself, what’s the most effective way to learn? Let me reframe this from a cognitive psychology lens: what’s the most effective way to consolidate information into long-term memory? The answer might surprise you.

Following my goal to dive more into small yet impactful steps, below are the themes that float around in my head (some might overlap).

Comment with your thoughts or multiple themes; if short on time, take the poll. Your input is invaluable. Thank you!

1. The Power of Naming

   We name things all the time without knowing what good or bad naming would look like. I’m not talking about not using acronyms as variables; this theme is beyond that. Naming has powers, it influences boundaries and could determine trajectories. This may come primarily from stories to start with. I hope to create a framework for naming at some point that may benefit us.

2. Task Decomposition Patterns

   Many software engineers I work with love to work on small steps but struggle to break big things down into < 1/2 day work. Sometimes, I show them how to break it down, and they have a moment of revelation. May connect to productivity improvements, pomodoro, flow state, etc.

3. Initiative, Influence, Introverts

   This theme is about turning that big cloud into raindrops (bigger task than previous theme). Many great engineers I met have high expectations for their team but get frustrated to make the change. Some attribute this to their introversion. I’ll share some of my stories, and case studies, about what works and doesn’t work, broken down into small steps.

4. Minimal Momentum

   I attribute many of my learnings to what I do outside of work, such as reading or working on a pet project, whether reading or shipping. Many engineers I spoke to feel they can’t finish their pet projects. In other words, they have big motivation but they lose their momentum. With the right technique, we can work on something decent with just 10 - 30 minutes a day.

5. Deliberate Expertise (Research Insights)

   Many found their expertise by accident, we can do better by making it deliberate. I’m self-studying some topics and will try to tie this back to our work as software engineers. To give some colour to this theme, this is what I’m currently studying:

   1. Cognitive Psychology. I’m studying a university book and am currently deep in Zettelkastening it. Lots of insights on how to learn and work better.

   2. Science of Expertise: How did I learn TDD? What’s deliberate practice? (lots of misunderstanding out there).

      I might tie this back to ‘s Mastering Programming.

   3. Science of Habits: What habits? Which bad habits? How do we build/undo it?

      : “I'm not a great programmer; I'm just a good programmer with great habits.”

   4. Cynefin: Which practice, when, and where?

Leave a comment

Expert’s Aside

As I wrote the last theme, I realised I believe in the idea of ‘competency over career’. This means I’d prefer to use the language of novice to expert, rather than junior/grad to lead/staff, etc. This is ironic because the idea of competency is often discussed in the context of a career. More ironically, I also associate the word ‘expert’ with this hilarious video:

Leave a comment

Welcome to the latest issue of my newsletter series, ‘Test First vs Last’, where we’ll explore how the sequence of writing tests affects our minds. The full context of this series can be found in the introduction post.

Test-driven development (TDD) is often rejected due to its perceived unintuitiveness. Is there a way to make TDD more intuitive? I believe there is, and it requires us to think more about specification first, rather than testing first.

In this essay, I’ll explain why spec-first might be more intuitive than test-first. I’ll further explore why we still don’t write a spec even though it’s an intuitively small yet impactful step, with the hope that more of us will think about this in our work.

Spec first, test last

I previously wrote about how unintuitive TDD might be for some. The root of the unintuitiveness of TDD comes from the word test:

The idea of test to be after the fact is simple enough for even young children to grasp. They are tested, once they have studied. They got tested for an illness, once there’s a symptom for it. They can test a dish, once it’s started cooking. Testing, always comes last, not first!

Let’s consider the term “spec” instead. The terms specification, specify, and specific all stem from the Latin word “specificus”, which pertains to defining or identifying with clarity. You make something specifically clear by specifying in a specification.

The idea of specifying is simple enough for young children to grasp, even when they don’t use the word spec. The recipe they follow to cook is a spec. Builders don’t start laying bricks randomly; they follow a spec. Before you are about to drive that nail to the wall, you mark the spot first (spec). Spec mostly comes first, not last.

In the TDD process, one might view the first step more as defining a specification than merely setting up a test. Some of you might be creating spec files without realising why it’s called this way, i.e. foo.spec.js or foo_spec.rb. The next time you write that spec file, think about it for a moment: why is it called a spec?

When you think in spec-first, a specification is the defined behaviour of the elements you’re about to create.

As I have described so far in this series, the benefits of TDD hinge on the concept of laying out specifications. I believe you’ll gain the benefit even when you don’t write the spec in code. Write spec in human language if you feel that it’s easier. TDD helps you save one step by encouraging you to write the specification in code.

Unfortunately, although the word spec should intuitively come first before coding, we still don’t intuitively spec first. Knowing is not enough, we tend to miss this step.

Subscribe now

Coding mindlessly

Seth Godin frequently emphasises the significance of specifications in his writings. In his blog post, Creating a useful spec, he wrote:

If you want someone to help, you’ll do better with a spec. It lists just four things:

1. What is the problem to be solved?

2. How are we going to solve it?

3. How can we test that the thing we built matches what we set out to build?

4. How will we know if it’s working?

[…]

If you’re not spending at least 5% of your project budget on the spec, you might be doing it wrong.

This blog is not about software, but pay attention to what Seth has written for points 3 and 4 and notice how similar it is to what we need as software engineers.

Next, pay attention to the last sentence I’ve emphasised in the excerpt. Seth implicitly suggests that many of us don’t spend enough time defining a spec, and I agree. From the misunderstood waterfall model to the misunderstood Agile, we have gone from 100% to 0% in defining specifications. The waterfall model describes distinct activities before code: requirement (spec), analysis, and design. Many of us don’t do these activities anymore after the transition to Agile.

Vaughn Vernon, the author of DDD Distilled, observed a similar experience called “the taskboard shuffle”. The taskboard shuffle happens when the entirety of the team’s design process is to move a sticky note from the “To Do” column to the “In Progress” column, which is equal to nonexistent.

While product manager may contribute to specifications, often their focus is at a broader system level. When you break the system down into smaller elements, who is going to spec the component level if not us software engineers?

Why don’t we spec?

I delved deeper into the idea of spec-first thinking as a means to make TDD more intuitive. Therefore, it’s essential to understand and address the reservations many engineers might have around thinking or writing about spec. Why might some shy away from the spec-first approach, even when its benefits are clear? Five reasons:

1. Spec is perceived as optional.

   Some of us have survived so far in our career without thinking in spec. In my experience, we survive because someone else pays for the cost down the line. There is a misalignment of incentives.

2. Spec is perceived as a new skill.

   Like any new skill or methodology, spec-first thinking may come with a learning curve. Those already adept at their current method might hesitate to feel like novices again.

3. Spec is perceived harder than code.

   Even after you have acquired the necessary skills for it, you might still have the bias to do what’s easier i.e. code. This is related to the idea of task aversion, which I wrote previously here: Task Aversion and Confirmation Bias.

4. Spec is perceived as bureaucratic.

   Some may view thinking in spec as an additional layer of bureaucracy to the development process, slowing down the joy of coding. This could be related to the misunderstanding of being Agile.

5. Spec is perceived as unknown or uncertain.

   There are works where we can’t really define the spec upfront unless we muddle through the code. The resulting code becomes the spec (spec comes last).

In recognising these reasons, hopefully, we can address the underlying factors that might deter us from incorporating spec-first thinking and, finally, TDD in our workflow.

I believe the fifth reason causes a significant misunderstanding of TDD as some treat TDD as a one-size-fits-all approach. In such a scenario, does thinking in spec-first make sense? What do you think?

Leave a comment

Conclusion

I’ve described why thinking spec-first could be more intuitive than test-first. Even though it’s more intuitive to think about it this way, we shouldn’t forget that we still have to do it. Knowing is not enough.

In my following essay on this series, I may tackle why TDD is not a one-size-fits-all approach so that we understand when it is applicable. It’s a difficult one to write. Stay tuned.

P.S. Apologies for the sound of my washing machine in the voiceover recording.

Welcome to the latest issue of my newsletter series, ‘Test First vs Last’, where we’ll explore how the sequence of writing tests affects our minds. The full context of this series can be found in the introduction post.

Test-Driven Development (TDD) is often rejected due to its perceived unintuitiveness. Given the seemingly unintuitive nature of TDD, some people have seen me as unethical, manipulative, or forceful in writing about TDD’s benefits. While I question being labelled as unethical, manipulative, or forceful, I do agree that TDD is unintuitive.

In this essay, I’ll explain why TDD may feel unintuitive, and concurrently, challenge the reliability of our intuition.

Tests are intuitively after the fact

We don’t have to look far to understand why TDD is unintuitive. Tests are traditionally conducted after the fact, and we would intuitively understand this. If there’s no code has been written, what is there to test?

The idea of test to be after the fact is simple enough for even young children to grasp. They are tested, once they have studied. They got tested for an illness, once there’s a symptom for it. They can test a dish, once it’s started cooking. Testing, always comes last, not first!

Intuition serves as our subconscious navigator, built from our past experience and knowledge, allowing us to interact with our environment without conscious reasoning. We don’t need conscious reasoning to realize that testing has traditionally come last.

However, TDD challenges this traditional sequence. If it’s seemingly unnatural or unintuitive, does that inherently make TDD an ineffective technique?

Unintuitive? Does not make it wrong

The foundational argument against TDD here seems to be its lack of intuitiveness, concluding that what is unintuitive is not effective. If there’s a more intuitive way of doing things, why change?

Observing the daily routines of my 7-year-old, it’s evident that not all effective practices are intuitive. Would she grow up to brush her teeth regularly, naturally, by her intuition? This habit is not instinctively formed. It’s been cultivated and reinforced.

Similarly, the notion of the Earth being spherical is not immediately intuitive. How does our Earth look like the Blue Marble (a picture only taken 50 years ago)? To our ancestors, and to anyone climbing high, the world appears flat, contradicting the actual spherical model.

Another example is riding a bicycle. It’s widely appreciated, but it’s not something one can intuitively do. It requires learning and practice. It appears that not everything unintuitive is inherently disadvantageous.

Subscribe now

Fosbury’s unintuitive gold medal

TDD is unintuitive as it needs us to reorder our work sequence. This simple alteration can yield superior results, analogous to Dick Fosbury’s revolutionary high jump technique.

The aim of the high jump is straightforward: to leap as high as possible without knocking the bar over. Before Fosbury, athletes embraced techniques that had them facing down and leading with their legs. Fosbury, however, flipped this traditional technique by facing up and leading with his head.

The initial scepticism around his technique, due to its unintuitiveness, diminished when he won the Olympic gold medal in 1968. This unintuitive technique, the Fosbury Flop, is now becoming the norm.

The Fosbury Flop circumvents our biological limit by shifting our centre of gravity, allowing us to jump higher with the same amount of force. I believe that we can apply the same principle to knowledge workers, and one technique that we have uncovered is TDD.

What biological limits are we circumventing in TDD? Read the essays in this series and Why TDD Enhances Developer Productivity.

Subscribe now

The illusion of intuition

The daily routines, understanding of the Earth, and sports exemplify how unreliable our intuition is. Comparing these examples, one major difference in developing software is how much more involved our minds are. Could intuition be more reliable, perhaps in our line of work?

The Invisible Gorilla, a book by Chabris and Simons, argues against the excessive reliance on intuition, highlighting the numerous biases and limitations inherent in human cognition. One illusion discussed in the book is the illusion of attention, which I observe to be affecting us software engineers to overlook our work.

To understand this illusion, watch the following video and follow its instructions.

Did you watch the video? In the original study, 50% of participants did not report seeing the gorilla. If you did watch the video, I bet you noticed the gorilla, that’s because you expected some sort of an illusion when you watched it!

As software engineers, our minds constantly fall prey to this illusion. We can’t notice the gorillas in our work. We tend not to notice things that we don’t expect, here are a few examples from this series:

• We don’t expect false positive test results

• We don’t expect tickets to be that complicated

Software engineering is not the only profession that’s affected by the illusion of attention. In the healthcare industry, even experienced doctors can misdiagnose an illness. Similar to TDD, a structured approach, such as having a checklist, has been proven effective in improving patient care.

One might wonder why Dr. Atul Gawande chose to write the book Checklist Manifesto. My guess? Checklists are unintuitive.

Conclusion

TDD, while seemingly unintuitive, doesn’t mean it’s a technique to be dismissed. Numerous examples illustrate how unintuitive methods can prove effective. TDD effectiveness is independent of its unintuitiveness.

In my subsequent essay, I’ll propose how to make TDD more intuitive for us software engineers. Stay tuned.

This is the latest issue of my newsletter series, ‘Test First vs Last’, where we’ll explore how the sequence of writing tests affects our minds. The full context of this series can be found in the introduction post.

In my previous essay, Task Aversion and Confirmation Bias, I wrote about how I may experience aversion when writing tests:

After all, I’m done with the change, why bother writing tests? Suddenly, when I saw the written code, I felt it was hard to write tests. Maybe it’s not worth the effort. Oh, and that test framework I haven’t learned properly, it will be tough writing the test. I’m experiencing task aversion.

The aversion could come from many reasons, but aversion is not the only reason I might not write my test when I use the test-last approach. I’ll take an excerpt from Cecilie Vennevik’s blog post, A corner-cutter’s confession, as she brilliantly describes her experience (emphasis mine):

And yes, yes, I didn’t write any tests first. I know it would have made my work easier, and refactoring less error-prone. And it would have helped me out in the design process. But I knew what I needed to do (mostly)! And I could just boot up the app and test it manually - it was familiar and convenient!

What’s that? Did I write any tests after, then? Oh, no, I didn’t do that either. I’d, uh, already tested all the use cases manually by the time I finished. It felt like a waste of time not to merge the work and move on. Yes, I have no cheap way of catching regressions now, I know, I know. Can we move on?

Because there’s so much to move on to. I can’t afford to feel like I’m slow. There’s so many more things to do! Important things! Features! Stories! Bugs! Pull requests! The board is so long and the days are so short and I need to do my part.

Test-last may induce present bias

I have a higher probability of not writing a test when I practise the test-last approach too, just like what’s been illustrated in the excerpt. You might think what Cecilie or I would do to be lazy, but I prefer to make an argument that we’re delivering more value if we skip tests. The present bias influences us.

Present bias is our tendency to overvalue immediate rewards and undervalue long-term consequences. We tend to choose what we can do now that rewards us immediately rather than what can give us a larger reward in the future.

When it comes to writing a test, there are two types of rewards that you can get:

1. Instant reward - You know your change satisfies the expected behaviours

2. Delayed reward - You know your change breaks the existing behaviours

When I practice the test-last approach, I tend to test my change manually. This manual test will make the instant reward disappear because I understand that my change is working. This is not only happening for frontend development but also valid for backend development, where I would pull up an API client, like Postman, to test my changes.

Could the delayed reward be a good reason to write the test last? I’m not sure. My job as a software engineer is to deliver value to my customers, and that value can now be delivered because my change is done. Given the options to work on new features or write tests? I choose to work on new features!

The delayed reward of writing a test, to know that my future change may break an existing behaviour, regardless of how large the reward is, is seemingly small now compared to delivering a new feature.

To battle the present bias, you must draw rigour and discipline in your work. The leadership team may mandate that you write a test, therefore, it’s part of your performance goal, giving you an extrinsic reward. I’ll dread writing the test even more, as extrinsic rewards do not drive me.

Subscribe now

Test-first rewards by design

When I practice the test-first approach (or TDD), I’m incentivised with the instant reward, writing my test so that I know my change satisfies the expected behaviour. Call me cheap, but I feel joy seeing my test turning green. It’s incredibly rewarding. It’s possibly the same kind of instant reward I get when I do manual tests in the test-last approach.

The delayed reward comes automatically in the test-first approach. Given that you have written the test, that will also benefit you in the future. Win-win.

Conclusion

When practising the test-last approach, the incentive from the intrinsic rewards perspective seems to disappear. Practice the test-first approach, and you may feel more rewarded when writing the test, both instantly and in the long run.

So far, in this series, I seem to have a strong bias towards the test-first approach regardless of how objective I tried to be, and I’m not happy with that. In the next essay, I’ll focus on one downside of the test-first approach.

This is the latest issue of my newsletter series, ‘Test First vs Last’, where we’ll explore how the sequence of writing tests affects our minds. The full context of this series can be found in the introduction post.

Apologies for the inactivity, as I’ve been away for the summer holiday to spend more time with my family. I suspect that I’ll be pretty busy in the upcoming weeks. Therefore, your support will help me commit to my writing!

In my previous essay of this series, the Mere-Exposure Effect, I wrote about how practising the test-first approach encourages me to pause, then makes me more deliberate on the subsequent steps. When I practise test-last, though, I don’t pause, and the test I wrote didn’t discover a bug. Why is that happening? In this essay, we’ll explore why the test-last approach often leads to less effective testing due to task aversion and confirmation bias.

Four aversive scenarios

My job as a software engineer is to deliver value to my customers, and this is primarily done by shipping changes. When I write tests last, the test I’m writing feels like a chore. If what I’m changing has been tested manually, I’ll probably dread writing the tests even more.

After all, I’m done with the change, why bother writing tests? Suddenly, when I saw the written code, I felt it was hard to write tests. Maybe it’s not worth the effort. Oh, and that test framework I haven’t learned properly, it will be tough writing the test. I’m experiencing task aversion.

Typically, four scenarios may happen in my mind:

1. Experience aversion → Write superficial test → Ship code

2. Experience aversion → Write test → Threatened by red tests → Revert to scenario 3 or persevere

3. Experience aversion → Skip writing test → Ship code

4. Experience aversion → Write test → Ship code

When I write the test last, I mostly fall under the scenario of 1, 2, or 3. In the subsequent sections, I’ll explain some of the terms I’ve used here.

If you feel that 100% of your time you get to scenario 4 without any bugs after you ship, this essay is not for you. If you’re a mere mortal like me, though, let’s carry on.

Share

Confirmation bias

Scenario 1: Experience aversion → Write superficial test → Ship code

What I mean by a superficial test is the kind of test that covers the current behaviour instead of the expected behaviour. Given my aversion, I must rush and ship, and my biases take over. Under the influence of confirmation bias, my tests affirm that the code executes as I initially programmed it, ignoring whether or not the application satisfies user needs or handles edge cases effectively.

Confirmation bias is a type of cognitive bias where there is a tendency for people to find information to confirm their preconceptions. My preconception when I write the test last is that my code works. Therefore, my bias will lead me to write a test to cover that behaviour.

It’s an odd experience. The moment I have written my code, the code becomes my new expected behaviour, overriding what I had in my mind before I wrote my code.

Being aware that we have confirmation bias in all of us is a good first step in tackling this issue. We can deliberately spend time writing the tests and make sure we think about the expected behaviour—no more superficial tests.

Unfortunately, though, this is not the end of the problem caused by confirmation bias.

Share

Red tests become a threat

Scenario 2: Experience aversion → Write test → Threatened by red tests → Revert to scenario 3 or persevere

I’m done with the code, pushed through my aversion, and now am writing the test rigorously, with the awareness of confirmation bias. This is the last mile of what I’m trying to deliver. I can see the end of the tunnel.

I wrote how I expected the code to behave… Oops, my tests are red. This is not what I expected, and I expected my code to work (confirmation bias at play). This can’t be my code.

These red tests are threatening my glory! A frustration and a conflict of interest began to crop up in my mind. Is this test worth writing? I’m feeling impatient. I expect to ship the change today, and I’ve told my team about this!

Sometimes, when this happens, I will revert to scenario 3, where I skip writing the test altogether. Sometimes, I persevere with rationality and a sense of identity and complete my work properly. When this happens, I write the test with a sense of rush and displeasure.

Subscribe now

We all have confirmation bias

The feeling of threat comes from the conflict of information I perceive; my code works ready to be shipped, but the tests show otherwise. The sense of threat doesn’t only happen to software engineers because confirmation bias can occur to everyone.

Scientists, like anyone else, can feel threatened by information or results that contradict their expectations or existing work. A lot of emotional and intellectual investment has gone into scientific research, and negative evidence that contradicts their existing hypothesis can feel like a personal failure, which is further intensified by time pressure and limited funding.

Even the best thinkers experience confirmation bias. What sets them apart is their awareness of this bias and having strategies to combat it. Charles Darwin, for example, had a “golden rule”1 to actively note any information contradicting his current beliefs. This strategy requires a significant amount of discipline.

While this kind of discipline works, it requires much mental effort to persevere. What I’m interested in more is to have a system, a workflow, that can help me battle this bias without much mental effort.

If writing tests, an activity supposed to help us deliver software better, is becoming a threat to our work, it’s a sign that we have an ineffective approach to work.

Share

Reversing the sequence

What I’ve found to be effective in tackling confirmation bias is simply reversing the sequence: writing test first.

It’s worth noting that I still experience task aversion when I take the test-first approach, even after decades of practising TDD. But once I get past the initial reluctance, the rest of the development becomes smoother.

I don’t experience the sense of conflict between getting things shipped and ensuring that tests are written. That’s because I know I haven’t delivered the expected value, the application code has not been written yet. It’s clear that there are still more things that I have to do. In contrast to the test-last approach, skipping the test or writing a superficial test is not a strong option on the table.

When I write a list of pending tests first, I also have a better expectation of what’s left in my work. As I began to write a test, more scenarios occurred in my head, and I started writing more of the pending tests.

And lastly, when I write tests first, the feeling of threat associated with failing tests is significantly decreased. I don’t see failing tests as conflicting information, given that I expect to see failing tests in the test-first approach. Red tests are happening by design.

Conclusion

As a software engineer, there are two points where confirmation bias will impact our work:

1. Tests do not reflect the expected behaviour, but how the code currently behaves.

2. Failing test results are perceived as a threat to progress.

Even the best thinkers experience confirmation bias. Therefore, we need to be aware and have an approach to tackle this bias. When tackling this bias with the test-last approach, you will require discipline and be ready to set your expectations when the test turns red.

Adopting the test-first approach, on the other hand, reduces the cognitive load and minimises the negative impact of confirmation bias, making it a more effective approach.

Up next: I’ll cover how the reward mechanism differs when we write test first vs last and how it affects us. Subscribe so it lands in your inbox.

This is the latest issue of my newsletter series, ‘Test First vs Last’, where we’ll explore how the sequence of writing tests affects our minds. The full context of this series can be found in the introduction post.

My product manager reached out to me and told me that what I shipped was not what he had in mind. “What are you doing, Wisen? It seems like you didn’t read the ticket. I have written a lot there!”

The day before

Let’s transport ourselves to the day when I pick up the ticket. I picked up the ticket and looked at it:

Title: Play next song if current song fails
Description: [… Lengthy description …]

Simple, I thought. This is just a try-catch issue, and I don’t have to read the description. I quickly checked the code I’ve worked with before, yes, there’s a missing try-catch. I wrote the following code, added the test, and shipped it.

try {
  playSong()
} catch (e) {
  playNext()
}

Once the feature is shipped, users on a slow network keep moving to the next songs in their playlists indefinitely. I introduced a bug. I later discovered a separate error handling where the change should have been made. (Yes, pull requests, desk checks, and all the other processes could have caught this, but that’s not the point of this essay).

I didn’t read the ticket thoroughly, and I didn’t investigate the code. Rather than deliberately trying to understand what this ticket was all about, I jumped to a conclusion on what needed to be shipped based on what I was familiar with.

This situation is a classic example of how the mere-exposure effect affects a developer, leading us to prefer things just because they’re familiar. Let me explain quickly what this cognitive bias is about and how writing tests first can counteract this bias.

Subscribe now

Mistaking familiarity for understanding

I’ll give you one simple way to know if you have experienced the mere-exposure effect in knowledge work. Think about the time when you have finished reading a non-fiction book. You nod along when you read it, thinking that you understand what you’re reading.

You later meet a friend and have a conversation.

“I just finished reading that book, and it’s so interesting.”
”Oh great. What have you learnt from it?"
”Hmmm… Err… Ahh…”

You were taken aback. You don’t seem to remember anything. You uttered two sentences. What else? Nothing. You thought you understood this book well.

You go back home, re-read the book, and you nod along again. You’re familiar with the topic, you feel like you know it, but you don’t. You mistake familiarity for understanding.

Just like reading a book, in software development, it's also possible to mistake familiarity with understanding.

Elaboration tests your understanding

The best way to distinguish familiarity and understanding is elaboration. Elaboration takes effort. This is why you were taken aback when your friend asked about that book. Your mental resources worked hard to answer what you learned from that book.

This is the process of elaboration. It’s slow and takes time, but it helps you test if you truly understand. Verbal conversation is one way to elaborate. Writing is another form of elaboration.

In software, writing tests is a good way to test your understanding.

Subscribe now

Writing tests is a form of elaboration

Let’s replay the day I picked up the ticket, but this time I’m writing the test first.

I picked up the ticket. I thought, okay, just a try-catch issue. Easy. I start writing the test.

it("plays the next song when ...

When what? What kind of error should I throw in the test? Hang on, why am I writing a new test file? Maybe there’s a test that has already been written elsewhere? Is this the right place to test this thing?

I paused briefly because I had to elaborate on what I was about to test. This led me to re-read the ticket again and double-check if this was the right place to make a change. I broke my familiarity barrier and began seeking more understanding.

Conclusion

When writing tests last, we are more prone to the mere-exposure effect. This means we tend to build something and make changes in places we’re familiar with. Elaboration helps us reduce this effect of familiarity, leading us to ship better software.

Writing test first helps us elaborate, but elaboration itself is not foolproof. For example, you may go through the process of summarising a book you’ve read as a form of elaboration, but it doesn’t necessarily mean you will reach the right conclusion.

While the test-first approach is not a foolproof solution, resequencing our steps is a cheap way to mitigate this bias, making us less prone to the mere-exposure effect.

Yes, writing test helps us elaborate, but when I wrote the test last at the top of this essay, I didn’t have that brief pause of questioning what I was doing. Why does this happen? There’s another bias at play. I’ll cover that in the next post of this series.

First, I’ll explain what I mean by flipping around, then why it isn’t good, propose an alternative, and end with some practical tips on what app to use.

The flippant flipping

To fully understand the issue, let’s first look at what I mean by flipping around. This gif illustrates creating an object in TypeScript based on Stripe’s documentation.

This habit of flipping windows around is not such a problem if you do it for a single window once or twice. The problem is when you’re doing it fairly frequently in your workflow. This is why I use the word flip, not switch or cycle. You’re literally flipping, very frequently, sometimes struggling to find the windows that you’re looking for. I also like this word because it reminds me of the word flippant; this habit seems clever and fast (it’s not).

Well, I see that some of you are giving me a tsk-tsk, creating an object structure based on documentation is so simple, you could have copy pasted that. This problem is not limited to object structure, some other examples:

1. Flipping test code and production code when coding.

2. Flipping Figma and browser and CSS files when changing UIs.

3. Flipping IDE and terminal when running tests.

4. Flipping between existing code and currently written code for inspiration or examples.

5. Flipping between the issue tracker and test file when writing tests.

Yes, what’s being flipped around is not limited to windows. You may flip tabs around in your IDE or the browser. You could also flip desktops, for example, in MacOS.

Addressing this simple habit seems difficult, yet I notice how it affects developer productivity. Whenever I watch someone doing this, it’s like watching someone trying to pull out the tablecloth fast without sending glasses flying, and every time I see those glasses shatter into pieces, just like their thoughts.

You may have a good reason to flip around if you have a small screen estate like a 13” monitor. But the problem is I frequently see engineers with big or dual monitors doing the same thing. We probably carry this habit because we’re used to flipping around smartphone apps? Nevertheless, we should utilise the available screen estate that we have.

Share

Self-inflicted cognitive load

Now, let’s delve into the cognitive psychology behind this flippant flipping and why it burdens our heads. When you zoom in on this flipping phenomenon, you’ll realise that you’re rehearsing, trying to remember what you’re seeing, as if what you'll see will soon disappear. Think about the situation when someone handed you their phone number, you don’t have a notepad, and you attempt to remember that phone number, that’s rehearsing.

When you are flipping around, you’re rehearsing unnecessarily, because these windows won’t disappear like the phone number scenario, as long as you put them side-by-side! We, developers, dislike being interrupted by others, yet, we’re interrupting ourselves by doing this.

We also know that we have a limited working memory capacity. I notice that when I accidentally fall into the flipping trap, it also consumes my working memory capacity. Given that I needed to rehearse and remember, suddenly, all the high-level context I had in my head was gone, overridden by the tiniest detail I needed to rehearse.

Creating a view of what you need doesn’t take that long either, and once you do it more, you can do it fast, especially with keyboard shortcuts (I’ll cover that in the last section).

Subscribe now

Side-by-side, input and output

This constant flipping is a self-inflicted cognitive load that reduces our productivity. Pay attention to what’s happening in your mind when you flip things around. What was the CSS property I was trying to modify? Flip… flip… What was that JSON structure look like again? Flip… flip… Is it similar to Figma now? Flip… flip… When you catch yourself doing this, stop. Put them side-by-side.

Many of the tasks we have as software engineer is crunching information from one medium to another. What I’d like you to try instead is to pause and make all the information you need to execute your task visible on your screen.

If you have one large monitor, organise your windows or tabs, and put them side-by-side. If you have two monitors, with the same principle, put them side-by-side, even if they’re on a different monitor. If you work with only one small monitor, you need more screen estate to work effectively. As Jeff Atwood said, choose your own path to happiness:

Choose own your path to happiness, whether it's upgrading to a single 30" display, dual 24" widescreen displays, or three standard 20" displays. As long as it results in more usable desktop space, it's a clear win.

With the idea that we engineers are crunching information from one medium to another, you can frame that our head is processing an input and spitting an output. Put the input windows on the left and the output windows on the right. It looks like this on my large monitor.

When working with a smaller monitor like 16”, I can bear writing on the left-hand side, but on a bigger monitor, I’ll have to turn my neck, and it’s uncomfortable.

Share

Use keyboard shortcuts

Organising your windows can be painful because it is quite a mouse-heavy operation.

The following assumes the setup I’m currently having, which is on MacOS, have a 27” monitor, and uses VS Code, but you can apply the same idea to your setup too.

MacOS has a built-in feature to put apps side-by-side. However, I found this feature too slow and distracting for my workflow. To organise my windows, I use Rectangle. There are only a few shortcuts that I use:

• Left Half

• Right Half

• Center Half

That’s good enough when I only have two windows to manage, and that’s only three shortcuts to remember. I recently started to have three windows to work, then I use First Third, Center Third, and Last Third shortcuts in Rectangle.

When I use VS Code, there are only four essential shortcuts to remember:

• Toggle Primary Side Bar Visibility (Cmd + B)

• Toggle Terminal Visibility (Cmd + J)

• Split Editor Right (Cmd + \)

• Sometimes, I press Cmd+Shift+P, then type “Split down” to have even more interfaces on my screen.

Subscribe now

Try it out

Once you've mastered your shortcuts, it's time to test how well you’re organising your screen! I find the most effective way to try this is by pair programming. When my pair flipped things around, I instantly got lost. I don’t know what my pair is looking at. At the same time, we work better when we spend 1 minute reorganising our screen, creating a view where all the needed information is on the screen. This also surprisingly helps us pair better when our remote pairing setup turns laggy.

The next time you notice yourself flipping things around rapidly, pause and put those windows or tabs side-by-side. It’s a small thing to do, it doesn’t take more than 1 minute, and you’ll save that 1 minute fast. Try it, and let me know your thoughts!

This is the latest issue of my newsletter series, ‘Test First vs Last’, where we’ll explore how the sequence of writing tests affects our minds. The full context of this series can be found in the introduction post.

I picked up a small feature to work on. It seemed simple at first, so I started to write the code. After a few hours, the production code was ready.

Being diligent, I ensure to write a test for the feature:

it("must return truthy", () => {
  expect(newFeat())
    .resolves
    .toBeTruthy()
})

The test result is green, my code works! I pushed the change to the deployment pipeline, which also indicates green for the newly written test. I call it a day, satisfied with my work.

The next day, our support team reached out and told me that many users had started using this new feature, but it didn’t work at all! Wait, I tested this, didn’t I? The test was green.

Two green reasons

As I dug into the issue, I discovered the new feature was indeed broken, but why did the test not fail? Apparently, I had missed including the ‘await‘ operator. Once I added the operator, my test failed. I experienced a false positive test.

it("must return truthy", () => {
  await expect(newFeat())
    .resolves
    .toBeTruthy()
})

Reflecting on my false positive test experience, I realised that there are two potential reasons why a test might show “green”:

1. The code is working

2. The test is incorrect (false positive)

When I write my test last, I often expect the code to work and overlook the possibility of the test being incorrect, which gives me a false sense of security.

Subscribe now

Narrowing down to one reason

To eliminate the risk of false positives, it’s important to see the test fail. You need to see the test red. To see a red test, you have to deliberately introduce a bug to your code, therefore breaking it. The steps might look like this:

1. Write code

2. Write tests (green)

3. Break the code to make tests red

4. Change the code back to green

If you compare these steps against writing tests first, the production code doesn’t exist yet when you write your test. The absence of the production code removes the possibility of the “code is working” reason when you see a green passing test. This means if your test is green, the only possible explanation is “the test is incorrect”. The steps for the test-first approach therefore are shorter:

1. Write failing tests (red)

2. Write minimal code (red)

3. Make the test pass (green)

2023-07-13 Edit: has raised an important point in the comment, do read that and join the discussion if you have thoughts. I added “Write minimal code” as a step for the test-first approach, as otherwise, you’ll get false positives even in the test-first approach with this example! The minimal code may look like this const newFeat = async () => {}; I have also removed “Refactoring” as irrelevant in this post.

Although the ‘test last’ approach involves more steps and can be inefficient, you can be pretty fast in executing these steps once you get used to it. However, the challenge lies in the potential for human error, especially forgetting to deliberately introduce a bug. When you forget these steps, you increase the risk of encountering false positives.

In contrast, the ‘test first’ approach means you have fewer things to remember, as it automatically prompts you with a failing test first. Having fewer things to remember is beneficial for us engineers (see my other essay: Remove duplication to deliberately forget).

While you might assume that the likelihood of encountering a false positive is low, it’s important to remember that these false positives can significantly impact developer productivity.

Share

The impact on productivity

The three dimensions of the DevEx framework highlight what we can focus on to improve developer productivity1: cognitive load, flow state, and feedback loop. When I experience a false positive test, all these dimensions could be affected:

1. Flow state: When our reality doesn’t match what we expect, we experience negative emotions and frustrations, which is the antithesis of the state of flow and feelings of joy. In reality, I have tested the feature and expected the feature to work.

2. Feedback loop: A feedback loop containing a false positive means I don’t actually have feedback from the test. In this example, my feedback loop was from the support team, not the test.

3. Cognitive load: When I hear from the support team, I might have moved on to a different task. The need for a sudden context switch can lead to an increased cognitive load.

Conclusion

So what does all this mean for the ‘Test First vs Last”? As we’ve seen, choosing the test-first approach makes you less likely to forget to eliminate false positives, given that the production code has not been written yet. The test-first approach provides shorter steps to eliminate false positives. Reducing the likelihood of false positives is important as they can affect developer productivity.

While fixing the broken feature, my product manager reached out to me and told me that what’s been built was not what he had in mind. That’s frustrating! In my next post, I’ll cover how testing first vs last may affect this scenario.

1. Why should I switch to writing tests first if I’m already writing tests last?

2. Do you always do TDD, even in your startup?

This post introduces my experiment on publishing a series in this newsletter containing smaller, more digestible insights. If you have similar questions to the ones above, follow along. Hopefully, they’ll be answered at some point.

What is being compared?

What’s going to be compared in this series will be the reduced version of Test-Driven Development (TDD) and Test-Last Development (TLD). I’ll be focusing solely on the difference in the sequence of writing tests, despite TDD containing more principles. I’ve hinted previously that the sequence of steps matters more than you might think, as explained in Making a Nice Cup of TDD, and I plan to delve even deeper. Because of this focus, I’ve intentionally chosen not to name the series “TDD vs TLD”. Besides, these two acronyms are hard to distinguish with one letter difference.

Ultimately, we’ll be comparing writing tests first and tests last. We’ll assume that in both methodologies, engineers will have the discipline to refactor after.

What you should expect from this comparison will be less about the code, as that’s been covered very often. Rather, I’ll compare what’s happening in my mind - the psychological experience in practising these two techniques. This follows my previous theme of exploring our cognitive limits and the DevEx framework in my other essay, Why TDD Enhances Developer Productivity.

Having said that, you may wonder if my views are entirely objective or if there’s a preference for one technique over the other.

Will this series contain a bias towards TDD?

Short answer: Possibly.

However, in more depth, I attempt to be as objective as possible. I’m not really interested in siding in a certain technique, I’m more interested in understanding what to practice, when, and why, but inevitably I may conclude that one is better than the other.

I can be objective in this comparison because I have practised both writing test first and test last. Most TDD practitioners I know have practised writing tests last too before they learned about TDD, and that’s how they started. Writing test last was the default for me too.

I also coach teams to practice TDD. Therefore, I can see what engineers are noticing when they write tests last. When the emotion hits them hard, I can see it. They share a similar experience as when I practice writing tests last.

These aspects should hopefully maintain my objectivity throughout the series, so stay tuned!

Subscribe now

Test-Driven Development (TDD) is a design technique involving a specific sequence of steps: writing a failing test, implementing the simplest possible code, and refactoring the code. While TDD is often perceived as a design technique, I propose a new perspective: viewing TDD as a productivity tool. Practising TDD, I argue, will improve your productivity.

Before diving into how TDD enhances productivity, it’s crucial to understand what productivity entails for developers. The idea of how to measure developer productivity has often been misunderstood. Fortunately, Noda et al. recently published a paper that provides a new perspective on developer productivity. The team who publishes the paper includes experts on developer productivity, including Nicole Forsgren (co-author of Accelerate) and Margaret-Anne Storey (co-author of SPACE).

The paper includes the proposal of the DevEx framework, which outlines the three dimensions that affect developer productivity: feedback loops, cognitive load, and flow state. Instead of summarising their findings, I will take the excerpt from ‘s newsletter post, which succinctly explains these three dimensions:

1. Feedback loops refer to the speed and quality of responses to actions performed. Fast feedback loops allow developers to complete their work quickly with minimal friction. Slow feedback loops, by contrast, interrupt the development process, leading to frustration and delays as developers wait or decide to switch tasks. Organizations should shorten feedback loops by identifying areas in which development tools can be accelerated (e.g. build and test processes, development environment setup), or human hand-off processes improved. 

2. Cognitive load encompasses the amount of mental processing required for a developer to perform a task. When cognitive load remains high as a result of problems such as poorly documented code or systems, developers need to devote extra time and effort to complete tasks and avoid mistakes. To improve developer experience, teams and organizations should aim to reduce cognitive load by finding ways to eliminate unnecessary hurdles in the development process. 

3. Flow state is a mental state in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment. Frequent experiences of flow state at work lead to higher productivity, innovation, and employee development. Similarly, studies have shown that developers who enjoy their work perform better and produce higher quality products. Teams and organizations should focus on creating the optimal conditions for flow state.

The three dimensions of the DevEx framework highlight the areas organisation can focus on to improve developer productivity. However, the limitation of this study is its focus on organisational or team-level change, which is externally focused from the developer’s point of view. There’s room to have a more internally focused point of view, drawing what we can learn from these dimensions into our own individual productivity.

To manage these dimensions effectively from an individual level, we need to understand what limits us from an individual perspective. What limits individuals are the process that goes in our head, our cognitive limits. Understanding our cognitive limits is beneficial as it allows us to see why adopting TDD can be helpful.

I don’t use a hammer because it’s a great tool to drive nails. It’s because I can’t drive nails without a hammer. Similar to our minds, we require mental tools to drive our cognitive tasks. Without tools, we may not visibly bleed, but we may experience an unproductive day filled with frustrations.

Based on my experiences, these are the developer-centric way of summarising our cognitive limits, the three Ws:

1. Wit

   Our mind operates in two systems: System 1 for fast, automatic, emotional thoughts, and System 2 for slower, more deliberate thinking1. Developers tend to use System 1 by mistake to make important decisions. As a result, their quick wittiness will create a clever solution that doesn’t solve a problem. These clever solutions will increase the cognitive load on our future selves and others, and slow down our feedback loop when others review our work.

2. Working memory

   Our working memory typically has a limit of four2, measured in chunks3. Developers handle diverse tasks that demand complex thinking, such as user needs, programming language, architecture, refactoring, testing, etc. With our working memory limits, simultaneously thinking about all these thoughts is not feasible. For example, thinking about user needs and refactoring simultaneously will introduce an unnecessary cognitive load.

3. Willpower

   We have limited willpower that we can use in a day. When you’re about to start coding, the feeling of dread may happen when you have run out of your willpower. The constant use of willpower eventually depletes it, leading us to the loss of joy or flow state.

These cognitive limits are hindering the dimensions outlined by the DevEx framework. Given this understanding, it’s clear that we need a mental tool to help with our productivity. Think about it: Usain Bolt, the greatest sprinter of all time, is slower than a domestic cat but faster if riding a bicycle.

Share

The mental tool that we need to circumvent our cognitive limits is workflow. I’m not talking about an organisational workflow like how we pass on work to others. Rather, it’s an individual’s workflow, the sequence of steps of work that you do.

You might be surprised to hear how an individual’s workflow can circumvent our cognitive limits, so let’s consider a real-world example highlighting the importance of an effective workflow: the story of Niklas Luhmann. Luhmann authored more than 70 books and 400 articles on various subjects. His secret was a unique workflow called Zettelkasten, which enabled him to organise and connect his thoughts effectively. The amount of writing he had written is not only a matter of quantity but of quality. His workflow has given him abnormally impressive productivity.

Do you know why his workflow, Zettelkasten, is not widely adopted? When he shared Zettelkasten, the world of writing thought it was too simplistic to be true to adopt. Unconvinced writers believe that great outcome requires great effort, or a complex problem requires a complex solution, which is untrue.

In software development, we often fall into a similar trap as those sceptical of Luhmann’s Zettelkasten. The process of TDD may seem overly simplistic. Yet, we often neglect that this seemingly simple approach can circumvent our cognitive limits:

1. TDD forms a positive feedback loop

   Each of the changes in the test from red to green represents a feedback loop. When the test passes, we know our implementation works. This short feedback loop allows us to observe a sign of progress. The regular and fast signals of the progress we get are the best kind of feedback because it motivates us to carry on working, forming a virtuous cycle, and reducing our reliance on our limited willpower.

2. TDD manages cognitive load

   What’s the simplest way to make this failing test green? That’s the question we hold when we move from red to green. This question forces us to use System 2, the more deliberate mode of our thinking, therefore circumventing the use of our wittiness by mistake.

   TDD’s Red > Green > Refactor structure helps us form a chunk of the diverse knowledge we need to use, allowing us to pull the right knowledge into our working memory at the right time. This helps us manage our cognitive load. See the illustration below.

   

3. TDD promotes flow state

   Two prerequisites to experience flow state are clear goals and immediate feedback4. Compared to having a big goal, such as having a feature in the hands of the customer, the immediacy of feedback by practising TDD, such as seeing green passing tests, help satisfy these prerequisites for us to experience flow.

In conclusion, TDD is a powerful tool that circumvents our cognitive limits, improving all the DevEx dimensions of productivity. I hope future studies around TDD will look into its productivity element through these three dimensions and developers’ subjective experience.

Although this essay focuses on improving individual productivity with TDD, it’s important to stress that improving developer productivity through an organisational lens should also be a key consideration. Improving one or the other is not mutually exclusive.

If you like this post, you may like my series Test First vs Last, where I explore how the sequence of writing tests affects our minds.

Subscribe now

Quantum Steps exists to transform our approach to work for the better. If you discovered something new or love what you’ve read, please do me a favour, and share this post with your friends. Thank you.

Special thanks to my friend, who prefers to keep his name hidden, for thoroughly reviewing and criticising this essay.

A couple of weeks ago, tweeted this:

When did TDD become an identity? "I'm not a TDD person"--I heard this the other day. 😕 It's not a genetic disposition, like cilantro tasting like soap. How did this happen? What can we do now?

Intrigued by Kent’s last question, I am writing this essay hoping to change at least one mind. Therefore, If you’re “not a TDD person”, read on.

The view of “I’m not a TDD person” usually concludes that Test-Driven Development (TDD) is a matter of individuals’ preferences, which I've noticed in various discussions, responses to tweets, and even some TDD threads I’ve written. Further, this view is typically shared against one single aspect of TDD. That is, it doesn’t matter whether a developer is doing the test-first or the test-after approach. The sequence of steps in a task doesn’t matter, because it’ll produce the same result.

After all, the sequence of steps in our daily life doesn’t usually matter. When you’re making a cup of tea, whether you put in the milk or tea first, the result is the same cup of tea. Is it the same cup of tea, though?

I grew up in Indonesia. In Indonesia, the most common way I used to drink tea is to take it plain, or with sugar. We don’t drink tea with milk in Indonesia, at least not commonly.

I moved to the UK nine years ago, and apparently, tea with milk is the default for my British friends. A friend of mine, George, came to my house one day. Immersing myself in the British culture best I could, I offered him a cup of tea. He said, “Yes, please, with milk.” Believe it or not, I’ve been practising making tea with milk. Simple enough, mix milk, a tea bag, and hot water, and I’m done.

George took a sip, his face cringed, disgusted, disapproving. Being polite, he didn’t say anything, until I asked if I had made the tea wrong. “It doesn’t taste right. Could you explain to me how you made it, please?” George asked politely. I explained to him that I took a cup, poured some milk into it, and then he interrupted me. ”Stop, what? That’s not how you make a cup of tea! You put in the tea first, the milk last!”

This argument about how steps should be sequenced reminded me of TDD. So I got curious and asked him to come into the kitchen with me and show me all the steps he’d do, and with a big sigh, he agreed.

We entered the kitchen. I grabbed a cup, and to my surprise, George already interjected on this first step. He said, “Wrong, turn on your kettle first, then only ready the cup, tea bags, and milk. It’ll save you some time.” So I turned on my kettle, another interjection, “Wrong, you’re using too much water, you’re wasting your electricity, why did you make it too full?” All the minuscule, seemingly small things that I don’t quite care about, he commented on, until we reached the point where the water boiled. We now have to make a hard decision. Tea-first or milk-first?

George, being an expert in tea-making, explained to me, “Tea first because if you put the milk first, the boiling water’s temperature will drop quickly, leaving less time for the tea leaves to brew.”

It makes a lot of sense, I thought. As I poured the boiling water into the cup, my cup began to crack. I was shocked. Calmly, George said, “Well, if you want to enjoy a perfect cup of tea, you need a better cup!” Annoying, but I finally managed to make a cup of tea following his method.

It did taste nicer.

Subscribe now

In this seemingly simple task of making a cup of tea, the tiniest detail I hadn’t considered made an impact. When you zoom in on a specific task, you’ll discover that the sequence of steps to complete that task does make a difference. Similarly, when you zoom in closely at software development practices, the sequence of steps you execute can make a difference.

When I adopted the kettle-first approach and the tea-first approach, I saved time and made a better-tasting tea! In software development, test-first or test-after will make a difference. When I practice TDD, I’m more productive and produce better-quality software. There is evidence that others have gained similar results. (I’ll expand on this thought in my next post).

Perhaps the question is, how much of this optimisation is worth doing? Maybe TDD only improves our productivity and quality by 1%. What’s the point of changing a habit to get a 1% improvement? It may seem too marginal to change your “preferences”.

But most of us struggle to grasp the power of tiny continuous improvements. A 1% improvement may seem marginal, but do it every day for one year you will improve 37x! Better yet, once you adopt it as a default, you don’t need to think about making improvements because that new habit of the sequence we tweaked is already making those improvements.

Like most practices, TDD is not context-free. My cup broke when I tried the tea-first approach for the first time. If you’re a TDD practitioner, help others feel safe by remembering that context matters. Otherwise, you may be breaking lots of cups!

So does changing the sequence of steps matter at all? Consider what my wife told me as I write this essay, “When I cook an omelette, you can choose to put egg first or oil first in the pan. The sequence is obvious if you want a better-tasting omelette!” It becomes apparent that the sequence of steps in which we do things, whether in tea-making, software development, or cooking, can indeed yield significant differences in the result.

Quantum Steps exists to transform our approach to work for the better. If you discovered something new or love what you’ve read, please do me a favour, and share this post with your friends. Thank you.