Major Tech Projects

Under construction

Last time we saw how a target architecture can guide you in your day to day technical decision making. But what do you do when your target architecture is considerably different from your current architecture?

Ways of working differ considerably between organizations, but most tech companies nowadays use some form of agile process (or at least claim to), where a Product Owner prioritizes a backlog of smallish items for the team to work on. That’s the context we’ll assume for this post.

The question then becomes how you convince a business-oriented Product Owner to prioritize technical improvement work over value-added work, such as new features. This question becomes even more pressing when the amount of tech work is not small, but rather an entire project in itself. This type of work doesn’t fit all that well into the agile model of continuous delivery of value through small increments. So what to do?

First of all, it’s always good if you can split the work into smaller pieces. Those are easier to squeeze in and, more importantly, carry less risk. But even if you manage to do that for a big tech overhaul, you’ll end up with a long list of tech items that somehow need to find their way to the top of the backlog.

You could try to persuade the Product Owner in a separate discussion about each and every one of these tech items. However, since Product Owners are judged by how much value they deliver sooner rather than later, you’re going to be fighting an uphill battle. That doesn’t mean it can’t work in your context, of course, but in general the chances aren’t great when the number of tech items grows.

Alternatively, you could treat this as technical debt, assuming you have a well-functioning process for handling that. That’s quite an assumption, by the way. Some teams use their slack time to dig themselves out of holes, but most teams are not fortunate enough to have slack time. (As an industry, we’re slow learners. Mythical man-month, anyone?) Many teams I’ve seen that keep track of tech debt just keep growing it year over year and I hate to think about the teams that don’t even keep track of it.

Under construction

Part of the problem with tech debt is that the metaphor is broken. Financial people will tell you that some amount of debt is good and that you shouldn’t get rid of all of it. That’s just not how developers see tech debt. I’ve also yet to see a team that regularly and consistently pays off tech debt, like one would make monthly payments to pay off, say, a mortgage.

Idea Flow argues that we should abandon the technical debt concept in favor of risk and that we should let the team be guided by both a Product Owner, for business value, and Technical Risk Manager, for keeping risk within an acceptable range. It’s an interesting concept that sounds like it may work, but the problem will be in estimating the risk. I haven’t seen this done in practice yet, so if you have experience with this, please leave a comment below.

A somewhat similar, but less refined, approach is to allocate two budgets for the team: one for doing value-added work and one for doing technical work. You could assign a 3:1 ratio to those budgets, for instance, which means 75% of developer time would be spent on adding business value and 25% on technical work items. Then if you have a year where you need to do some major technical work, you could ask for a temporary change in this ratio.

Yet another approach is the dreaded rewrite: you stop all value-added work for some time and do only technical work. This allows the team to make quick progress in the technical area, but is generally not liked all that much by the business. You can get away with this only every so often, as it tends to take a huge bite out of your political capital. But at some point it will become your only option if you don’t find a better way first.

What do you think? What ways to schedule time for major technical projects have worked for you? Please leave a comment below.

Target Architecture

In the last two posts, we looked at generic architecture diagrams and security-specific diagrams. These diagrams reflect the current architecture of a system. This time we will look at using diagrams to depict a desired future architecture, or target architecture.

The point of a target architecture is to paint a picture of the desired state that will act as the North Star (or Southern Cross, depending on your hemisphere), in the sense that it guides all the little decisions we make every day when working on our system. This doesn’t mean we’ll always be moving straight towards the target, but we do want to make sure we don’t stray too far of the path.

However, a target architecture is not a static thing.

Assuming you don’t have a crystal ball, it will be hard to predict how circumstances will develop over the years. So think of a target architecture more as a moving target than as something set in stone. We need to keep an open mind at all times, and always react appropriately to changes, and one such reaction could be to update the target architecture as we learn more about our system, the customers it supports, the team that builds it, the evolution of the technologies it uses, etc.

Moving towards a target architecture usually doesn’t happen in a straight line.

So how do we come up with a target architecture in the first place?

One starting point is to look at the diagram that reflects your current state and identify things that give you trouble. Few systems are optimal, so you’re likely to find some things that you would do differently if you had a chance to start all over.

Another source of ideas comes from future requirements, i.e. requests on your backlog. The fact that the current architecture supports the current requirements doesn’t necessarily mean that it will be able to support all future requirements. Sometimes we need to create some architectural runway.

Once you’ve identified an area for improvement, you need to make a decision about whether it’s actually worth fixing. Some things are very costly or risky to fix compared to their benefit.

Next, write up your decision in the form of an Architecture Decision Record (ADR). The structured format of an ADR, which lists the positive and negative consequences of the decision, will help with getting everyone on the same page. It will also give you a nice historical record, which is especially important for any newcomers as the team evolves over time.

Every ADR records a decision. Each such decision constrains the solution space of the system in its own way. As the number of decisions grows, the resulting viable part of the solution space shrinks. The target architecture is a picture of a single point in this viable part of the solution space. Note that there could be other points that are viable as well, i.e. alternative target architectures.

Obviously, the solution space is multi-dimensional. But if we project it in two dimensions, we can visualize the relationship between target architecture and ADRs as follows:

Architecturally significant decisions constrain the viable solution space in which a target architecture lives.

If you find your team arguing about two different options in the viable solution space and you want to resolve this ambiguity, then hopefully the picture above will suggest the solution: you’re missing an ADR that shrinks the viable solution space to exclude one of the options.

What do you think? Do you find a target architecture a useful tool to guide your team’s day to day technical decisions? Do you use ADRs to document why the target architecture looks the way it does? Please leave a comment below.