202610121056 Where do you start a project?

#blog #work

Let's imagine a hypothetical scenario.

You and Jill, a coworker at your consulting firm have started working for a new client. The two of you are in charge of helping lead an initial pilot transition to the cloud using modern software design principles. The client is a Fortune 500 retailer and consider all spending in this area to be a cost center. The client has agreed to a six month project for you and Jill to work with an existing team within their e-commerce org. It’s up to Jill to decide which team to tackle and to come up with some project goals. She’s asked for your thoughts on where you two should start.

Although I phrased this as a consulting gig, the process for finding the right work and the right teams to work with comes up regularly in staff level engineeringas well.

So what do you say? Where do you start this kind of thing?

It's about the context

I would suggest that before we make any decisions, we make sure we have a crystal clear understanding of the client and their goals. There's some of that information here in the scenario (they view this as a cost center, they're a large firm, we only have six months, and they want us to pilot this with an existing team), but there's missing information that we need to make a good decision.

To start, we need to know what's driving them to do this project. For example, if they say it's purely a cost decision, that would greatly change our approach compared to them saying it was purely a reliability decision. In reality, it will almost surely be a nuanced balance where the client wants to see benefits across a wide spectrum. Making sure we have a deep understanding of what their primary motives are will help us identify which pain points are most important to tackle in the time that we have.

It's also helpful to understand the internal and social context. How did we come to contract with this client? Is there contention at the client about using contractors vs working on this in-house? What is the client's business situation? Are they growing and opening new product lines, or maybe fighting a competitor and feel like they're playing catch-up? Knowing the situation before you step into it is very important.

It's about the trade-offs

Then, provided we end up with a good understanding of the client, their goals, their current systems, and their current organization, we could go about making a decision by weighing trade-offs. As any engineer worth their salt will tell you, the only correct answer to every question is "it depends".

There's a concept called a "U-shaped optimization problem" that feels applicable here. The concept says that if you're trying to optimize a system for some two properties (e.g., speed vs reliability), there will be a point where optimizing one of those decreases the other. Perhaps as you make the system safer and safer or more and more predictable or reliable, you have to give up some speed (e.g., distributed transactions vs eventual consistency). The name of the concept comes from the diminishing returns of one vs the increasing returns of the other, which you can imagine creates a V or U shape if you were to overlap them on a graph. Many systems are actually seeking the middle of this U-shape where they're not maximally secure or reliable or fast or cost efficient. Finding the right balance where each of these properties are sufficient can be more important than trying to maximize all of them. In other cases, there could be one overriding requirement (e.g., security in a PCI storage system) that needs to be maximized regardless of the cost to other properties.

Getting back to the scenario at hand, we can apply this concept by taking our understanding of which properties the client must have, which ones they want, and which ones they don't need, to find a balance between approaches.

Once we have a good idea of the kinds of approaches we think will fit for the client and their goals, we need to turn that into a concrete plan. For this, we need to take into account the reality of the scenario instead of just the abstract goals. For instance, what languages, frameworks, infrastructure, or architectural experience does the client and their existing teams have? What experience do I, Jill, and the rest of our consulting team have that overlaps or augments that? Does the client already have accounts, provisioning, or deployments for the new environment? Are we teaching devs to operate in a new environment or building a new environment? There's a nearly inexhaustible list of scenarios that could be the case given the wording of this scenario. The important point is that we meet the client where they are to solve the most pressing pain points and provide a path for them to continue to be successful after our engagement is concluded.

It's actually about the people

I'll add one final note that reality is often messy. The above was written as a solution that may read "simply follow these steps and everything is great!", but in truth, it's rarely that simple. There are probably personal and political interests in the change we were brought in to make, fighting for maintaining the status quo, or wanting a different change. Navigating these social forces and "people problems" is a huge part of making a project like this successful.

In the best case scenario, everything is aligned, we can search for the best project, and complete it. Otherwise, we have to openly engage in an honest and positive way, do our absolute best to work on the most impactful project, and lean on our shared humanity to navigate the complex situation.