How can you plan and break down a project that you are not familiar with?

With the new quarter approaching and my team getting ready to tackle the next project, it was time for me to understand the next problem we want to solve.

Yet, I was intimated. I felt that the problem at hand had so many unknowns. I was afraid of not being able to fully uncover all of the issues. I didn’t know where to start.

I asked my manager for help. He shared with me the framework he uses for understanding and breaking down a problem. After our conversation, I applied the framework for spiking our next project. And it worked! This post shares this framework.

What is a spike

Before diving into the framework, let me clarify what I mean by a spike.

The main purpose of the spike in my case was to

It’s most useful for medium-to-large-size projects.

The goal

The goal of a spike is to understand things deeply. As a project lead, without a fundamental understanding of the problem at hand, you won’t have confidence in planning and breaking down the project.

The goal is to gain that fundamental understanding so you can lead the project with confidence. After the spike, you should become an expert on the problem. You should know the ins and outs of the problem: what’s working, what’s not working, why it’s not working, and what are the different ways to fix it. You should be able to answer most questions related to the problem.

Start from first principles instead of based on past assumptions

For complicated problems involving legacy systems, there are normally impressions and assumptions from previous experiences. These past impressions and assumptions make the problem seem either overly intimidating — “this system is always a pain in the ass to work with” or overly trivial — “I touched this part before, and it wasn’t too complicated”.

You should start from first principles instead of based on past assumptions. Past assumptions can be outdated, inaccurate, or simply wrong. They will never give you a fundamental understanding of the issue and the confidence you want.

A spike for planning a project is the time for thorough research. For that, you need to start from first principles.

By starting from first principles, I mean going to the source of truth or as close to the source of truth as possible.

For a legacy system, that normally includes:

1. current code related to the system;
2. data produced by the system;
3. people who deal with the problem closely / experts of the problem;
4. and stakeholders who care about the problem.

#1 current code related to the system. Obviously, you should read the code and understand what it’s doing. You might run into code where the previous developers went out of their ways to do things that might not make sense at first glance. Normally questions will arise: Why did they do it this way? What were the constraints they were facing or the assumptions they had in mind? What was the system designed this way? What did they optimize for?

Marking down and answering these questions is important. Answering them helps you gain a better picture of the system: why it’s the way it is, what are the past assumptions or constraints that are no longer valid but contributes to the problem, what are the things you can and cannot change about the system, what are the areas you need to be careful about.

It’s best if you can directly ask the developer who originally wrote the system these questions. A simple 30 min conversation might bring you much clarity.

If that’s not possible, you can still dig into past artifacts around it: pull request, design specs, etc.

#2 data produced by the system is another great resource. After reading the code, you might have a vague picture about why things are the way they are, some ideas about past constraints and assumptions, and initial thoughts about where the problem lies.

Data a great resource to validate all those ideas. You should play around with the data. Do some partitions and filtering on the data to test your understanding. After you test your understanding against real data, you will gain way more confidence.

#3 talk to people who deal with the problem closely and experts of the problem. There are things that are hard to discover from code and data. You should leverage experts who have been dealing with the problem or the system for a long time. These experts might not be engineers. They might be the operation team uses the system or whose day-to-day is impacted by the problem you are trying to solve.

Don’t be shy. Reach out to them. Most likely they will be thrilled to talk to you on this topic since they would love someone to fix these issues. You will be surprised by how much you would learn from a quick conversation with them.

#4 Last, don’t forget to talk to stakeholders who care about the problem or the system. You want to make sure you are solving the right problem, have a proper understanding in terms of the importance of different issues, and can prioritize them accordingly.

For #3 and #4, when you talk to people, they will tell you lots of things. Don’t take their words on the surface level. Ask why they say that. Then ask why again. You should test people’s assumptions and get to the root of what they actually care about.

Find the problem statement

After the previous step, you should have a good idea in your head about what the issue is and how to solve it. You shouldn’t leave your understanding in your head. You need to articulate it out. Putting it into words helps you exam it more clearly.

By problem statement, I mean the meat of the problem and how to solve it. I said /find/ the problem statement instead of /define/ the problem statement because the meat of the problem is already there, your job is to discover it, not to invent it.

Answer these two questions: What are the underlying issues? What you think can solve them?

When it’s time to share your learning and your plan with the team, having the problem statement clearly written out also helps tremendously.

Test your POC (proof of concepts) of the solution

Once you write down your problem statement, what’s wrong and how to fix it, the next step is to have proof of concepts of the solution you have in mind. That proof of concepts should be something that you can implement quickly in a local or a testing environment. The code can be ugly. Don’t care too much about the code quality. You should focus on getting something that you can play with quickly.

Then you want to test that POC against the data you have. You also want to show experts of the issue your POC. Optimize for short feedback loops for both testings with data and with the experts.

Expect to have many back and forth for both testings with data and with the experts. Here’s when you discover things you miss in previous steps and gain a deeper understanding of the problem.

That’s the framework my manager shared with me. Basically, it boils down to doing the research yourself instead of taking people’s words for granted.

As a tech lead, when doing project planning, it’s crucial that you understand the problem deeply and test your understanding and the potential solutions.

It’s fine to spend the time on this phase. The time you spend here is saving everyone else on the team’s time later.

My career plan for the year is to grow into a tech lead. I’m excited about all the learnings ahead and would love to share this journey with you in a brutally honest fashion. I will be sharing my weekly learning on the blog.

In the next few months, I will focus on growing in the following areas. You can expect to see posts related to them:

• focusing on the big picture of the project instead of near-term implementation details;
• balancing my efforts between leading projects and coding;
• work-life balance for long-term productivity;
• the human side of software development: making sure everyone riding with me enjoys the ride and feels fulfilled and inspired.

Related Posts

• Fear No Awkwardness — Chatting is Good

• The Power of One-on-One Meetings and Personal Connections

• How to communicate effectively for bigger impacts

• Communication: Getting on the Same Page (Part 2)

• Communication: Getting on the Same Page