I used to love system designs. I thought great engineers designed systems for the long-term, and I was eager to do that.

However, I neglected the importance of patience and timing. Besides being able to design beautiful systems, great engineers also know when it is too early to commit to a design.

In this post, I will discuss:

1. My definition of premature designs

Premature designs are untested designs. They are designs based on unverified assumptions.

Coming up with a premature design involves three steps:

1. predict the future
2. believe in the prediction
3. design a system based on the prediction

The second step often happens unconsciously. One can imagine the future and, at the same time, treat the imagination as facts. Because voices coming from within sound familiar and true, our brains struggle with separating our own conclusions from facts.

To avoid making premature designs, we change step 2 to “verify the prediction”. Given the change, the new three-step will be:

1. predict the future
2. verify the prediction
3. design a system based on the prediction

With enough information, we can verify our prediction right away. However, if we don’t have enough information, we should hold our horses.

If we commit to a design despite the lack of information, we risk wasting resources on the wrong design.

2. Symptoms of premature designs

One symptom is you try hard to imagine the future. The fact that you have to try hard indicates a lack of information.

Another symptom manifests when we purely base a design on unjustifiable feelings: “this design looks cleaner.” “It feels better to me.” “It should be like this”.

An engineer’s instinct is valuable. We should never ignore our gut feelings. However, we shouldn’t take them as the gospel either. We should examine them objectively and see where they stemmed from.

Making design decisions solely based on guts is risky. Be careful every time we catch ourselves saying words like “looks”, “feels”, and “should”. Dive deeper into the reasoning behind them. Ask “why”.

3. Dangers of committing to premature designs

1. It might be a form of perfectionism.

We need to go from A to B now. But we think we might eventually want to get to E. Therefore, we are tempted to put in extra effort to go straight from A to E in one step. The underlying motive might be perfectionism.

The thinking is "if I know I want to eventually get to E, why would I take the detour to B, C, and D?"

Again, E is an unverified vision. We should ask ourselves: ¹

• How certain are we about going to E?
• Is it the right time to expend the effort to get to E?

2. It’s a form of waterfall development.

Going straight from A to E is the waterfall development style. The agile approach is to break one big step into smaller steps. This gives us the flexibility to adjust our direction in each small step. After each step, we gather more information to test our hypotheses. New information also arrives as time passes.

Breaking a big investment into smaller ones reduces the risk we have to take now. Risk also reduces when the information at hand increases.

My proposed way of thinking.

1. Envision the future, but delay commitments.

Envisioning and anticipating the future of our systems is great. It helps us understand our domains better. It also encourages us to identify new ways to better provide values to our companies.

We should always be thinking about the future in the back of our minds. How will it look like? How can we get there? These questions help us to be more sensitive to the pain points of current systems. When we notice a new pain point, we can use it to verify the future system in mind. As a result, we adjust designs in our minds over time.

But we must be careful not to get ahead of ourselves. Do not make premature commitments. Since we will constantly collect new information and adjust the design in our head, effort not spent is better than efforts needed to be undone. Code not written is often better than written code optimized for the wrong thing.

Delay any commitments when possible. Favor work that brings clear benefits to the business now over work that feels™ right for the long-term.

Anticipate the future, but do not try to make it up out of air. Let the future emerges on its own.

Two FAQs:

We shouldn’t make big design changes in a hurry.² We should have been playing and fine-tuning those designs in the back of our minds for a decent amount of time.

When the future is unclear, minimize design investments. Deliver feature requests with the least cost. If you must, redesign the minimum amount and focus on the flexibility of the system.

In short:

• anticipate the future: keep thinking about it in the back of your mind
• be prepared for it: keep any design you must make lightweight and flexible
• avoid premature commitments

2. Verify your vision and justify its opportunity cost

No matter how much you try to delay, at some point, you need to commit to a design. This brings us to two essential questions:

• What should be the new design?
• When is the right time to change?

PS: I listed many questions in this section. They might not have clear answers. These questions are here to help us keep the big picture in mind. I will explain more in a second. But let's get to these questions first!

The “what” question is about verifying your vision.

I found the following questions helpful.

Is it based on some theoretical engineering idealism? (aka: "It should be like this because it looks cleaner.") Or is it designed to solve existing pain points in the current system?

What assumptions do you have about the business and the system? Anything that will make this design not worth pursuing?³

Are we talking about 6 months, 2 years, or 5 years? Designing the system for the next year is different from designing it for the next three year. The longer into the future, the less certain we can be.

What’s the price of being wrong? If we find out three months later that our vision was off, how hard is it to change the design? Can the design accommodate a different vision with small adjustments?

The “when” question is about justifying the opportunity cost of doing it now.

After we confirm this is the right thing to do, we should now ask ourselves "is now the right time to implement?"

For example, a new system might require three weeks of two engineers’ time. Putting two engineers to a different project for three weeks also adds value to the business. Why should we favor project A over project B?

The size of the investment also matters. If it costs less than three days of engineering work and the cost of being wrong is low, you don’t need to analyze too much.

The “when” question is hard to answer. In its essence is a global prioritization problem. It requires an overall perspective about other opportunities in the business. This is where product managers are helpful. They tend to have a broader understanding of business needs.

Not everything is measurable. It's hard to quantify the benefits of a well-designed system.

When we ask ourselves these questions, we are not after definite answers. We ask these questions to be more conscious about our design trade-offs.

Don’t let the enjoyment of (re)designing and refactoring distract you from the big picture.⁴

3. Leverage product managers’ help

Many of the questions I listed above are questions PMs will ask. There are two types of projects: PM-driven and engineer-driven projects.

PM-driven projects mostly relate to external features. PMs will challenge extra engineering investments. This helps us to be more conscious. As a result, we are less likely to over-engineer on PM-driven projects.⁵

Engineer-driven projects tend to be heavy on internal systems. We tend to over-engineer on this kind of projects. For these projects, it is invaluable to include PMs, who can challenge engineers and not take our words at face value.

Regardless of PMs’ presence, we as engineers should be conscious and honest with ourselves. We shouldn’t let the joy of engineering take over our ultimate goal: delivering values to our businesses.

 

Building for the long-term is the right thing to do. But committing to premature designs is a wrong way to go about it.

Next time before making any big design investment, let’s ask ourselves: “have we verified our design?”

---

My career plan for the year is to grow into a tech lead. I’m excited about all the learnings ahead and would love to share this journey with you in a brutally honest fashion. I will be sharing my weekly learning on the blog.

In the next few months, I will focus on growing in the following areas. You can expect to see posts related to them:

• focusing on the big picture of the project instead of near-term implementation details;
• balancing my efforts between leading projects and coding;
• work-life balance for long-term productivity;
• the human side of software development: making sure everyone riding with me enjoys the ride and feels fulfilled and inspired.

---

[1] These two questions are about risk verification and opportunity cost analysis. Risk verification: examine the chances of a vision being wrong. Opportunity cost analysis: compare its cost and returns with other projects that also benefit the business.

[2] My definition of a big design change is a change requiring more than two weeks of engineering work that has no, or limited, functional changes.

[3] The most basic assumption might be: we want to keep this service around for more than a year. Otherwise, it’s not worth making any big design changes.

[4] Engineering is a means to an end. The end goal is to provide values to the business, customers, and society at large. Engineering shouldn't be an end in itself.

[5] This assumes a healthy tension between PMs and engineers where PMs are able to challenge engineers.

Related Posts

• What Matters to Me At Work?

• Why I Chose Gusto over Facebook and Airbnb

• An Attempt to Stop Thinking About Work Outside of Work

• Seeking Approvals at Work Exhausts Me

• Three Lessons from A Frustrating Meeting