Celebrate Learning in Software Development

Every event is either a cause for celebration or an opportunity to learn.

celebrateI don’t remember where I came across this quote, but it has stuck with me. I like how it turns every experience into something positive.

Sometimes I need to remind myself of it, however, especially when there are a lot of, well, learning opportunities in a row.

One recent case was when I had started a long running performance test overnight. The next morning when I came back to it, there was no useful information at all. None whatsoever.

What had happened?

scdfOur system is a fast data solution built on Spring Cloud Data Flow (SCDF). SCDF allows you to compose stream processing solutions out of data microservices built with Spring Boot.

The performance test spun up a local cluster, ingested a lot of data, and spun the cluster down, all the while capturing performance metrics.

(This is early stages performance testing, so it doesn’t necessarily need to run on a production-like remote cluster.)

Part of the shutdown procedure was to destroy the SCDF stream. The stream destroy command to the SCDF shell is supposed to terminate the apps that make up the stream. It did in our functional tests.

But somehow it hadn’t this time. After the performance test ran, the supporting services were terminated, but the stream apps kept running. And that was the problem. These apps continued to try to connect to the supporting services, failed to do that, and wrote those failures to the log files. The log files had overflown and the old ones had been removed, in an effort to save disk space.

All that was left, were log files filled with nothing but connection failures. All the useful information was gone. While I was grateful that I still had space on my disk left, it was definitely not a cause for celebration.

So then what could we learn from this event?

Obviously we need to fix the stream shutdown procedure.

kubernetesCome to think of it, we had already learned that lesson. The code to shut down our Kubernetes cluster doesn’t use stream destroy, but simply deletes all the replication controllers and pods that SCDF creates.

We did it that way, because the alternative proved unreliable. And yet we had failed to update the equivalent code for a local cluster. In other words, we had previously missed an opportunity to learn!

Determined not that make that mistake again, we tried to look beyond fixing the local cluster shutdown code.

One option is to not delete old logs, so we wouldn’t have lost the useful information. However, that almost certainly would have led to a full disk and a world of hurt. So maybe, just maybe, we shouldn’t go there.

Another idea is to not log the connection failures that filled up the log files. Silently ignoring problems isn’t exactly a brilliant strategy either, however. If we don’t log problems, we have nothing to monitor and alert on.

release-itA better idea is to reduce the number of connection attempts in the face of repeated failures. Actually, resiliency features like circuit breakers were already in the backlog, since the need for it was firmly drilled into us by the likes of Nygard.

We just hadn’t worked on that story yet, because we didn’t have much experience in this area and needed to do some homework.

So why not spend a little bit of time to do that research now? It’s not like we could work on analyzing the performance test results.

It turns out that this kind of stuff is very easy to accomplish with the FailSafe library:

private final CircuitBreaker circuitBreaker = new CircuitBreaker()
    .withFailureThreshold(3, 10)
    .withDelay(1, TimeUnit.SECONDS);
private final SyncFailsafe<Object> safeService = Failsafe
    .withFallback(() -> DEFAULT_VALUE);

public void init() {
  circuitBreaker.onOpen(() -> LOG.warn("Circuit breaker opened"));
  circuitBreaker.onClose(() -> LOG.warn("Circuit breaker closed"));

private Object getValue() {
  return safeService.get(() -> remoteService.getValue());

learnI always feel better after learning something new. Taking every opportunity to learn keeps my job interesting and makes it easier to deal with the inevitable problems that come my way.

Instead of being overwhelmed with negativity, the positive experience of improving my skills keeps me motivated to keep going.

What else could we have learned from this incident? What have you learned recently? Please leave a comment below.


Three Ways To Become a Better Software Professional

The other day InfoQ posted an article on software craftsmanship.

In my view, software craftsmanship is no more or less than being a good professional. Here are three main ways to become one.

1. See the Big Picture

Let’s start with why. Software rules the world and thus we rule the world. And we all know that with great power comes great responsibility.

Now, what is responsible behavior in this context?

It’s many things. It’s delivering software that solves real needs, that works reliably, is secure, is a pleasure to use, etc. etc.

There is one constant in all these aspects: they change. Business needs evolve. New security threats emerge. New usability patterns come into fashion. New technology is introduced at breakneck speed.

The number one thing a software professional must do is to form an attitude of embracing change. We cope with change by writing programs that are easy to change.

Adaptability is not something we’ll explicitly see in the requirements; it’s silently assumed. We must nevertheless take our responsibility to bake it in.

Unfortunately, adaptability doesn’t find its way into our programs by accident. Writing programs that are easy to change is not easy but requires a considerable amount of effort and skill. The skill of a craftsman.

2. Hone Your Skills

How do we acquire the required skills to keep our programs adaptable?

We need to learn. And the more we learn, the more we’ll find that there’s always more to learn. That should make us humble.

How do we learn?

By reading/watching/listening, by practicing, and by doing. We need to read a lot and go to conferences to infuse our minds with fresh ideas. We need to practice to put such new ideas to the test in a safe environment. Finally, we need to incorporate those ideas into our daily practices to actually profit from them.

BTW, I don’t agree with the statement in the article that

Programmers cannot improve their skills by doing the same exercise repeatedly.

One part of mastering a skill is building muscle memory, and that’s what katas like Roman Numerals are for. Athletes and musicians understand that all too well.

But we must go even further. There is so much to learn that we’ll have to continuously improve our ability to do so to keep up. Learning to learn is a big part of software craftsmanship.

3. Work Well With Others

Nowadays software development is mostly a team sport, because we’ve pushed our programs to the point where they’re too big to fail build alone. We are part of a larger community and the craftsmanship model emphasizes that.

There are both pros and cons to being part of a community. On the bright side, there are many people around us who share our interests and are willing to help us out, for instance in code retreats. The flip side is that we need to learn soft skills, like how to influence others or how to work in a team.

Being effective in a community also means our individually honed skills must work well with those of others. Test-Driven Development (TDD), for example, can’t successfully be practiced in isolation. An important aspect of a community is its culture, as the DevOps movement clearly shows.

To make matters even more interesting, we’re actually simultaneously part of multiple communities: our immediate team, our industry (e.g. healthcare), and our community of interest (e.g. software security or REST), to name a few. We should participate in each, understanding that each of those communities will have their own culture.

It’s All About the Journey

Software craftsmanship is not about becoming a master and then resting on your laurels.

While we should aspire to master all aspects of software development, we can’t hope to actually achieve it. It’s more about the journey than the destination. And about the fun we can have along the way.

Software Development and Lifelong Learning

The main constraint in software development is learning. This means that learning is a core skill for developers and we should not think we’re done learning after graduation. This post explores some different ways in which to learn.

Go To Conferences

Conferences are a great place to learn new things, but also to meet new people. New people can provide new ways of looking at things, which helps with learning as well.

You can either go to big and broad conferences, like Java One or the RSA conference, or you can attend a smaller, more focused event. Some of these smaller events may not be as well-known, but there are some real gems nonetheless.

Take XML Amsterdam, for example, a small conference here in the Netherlands with excellent international speakers and attendees (even some famous ones).

Attend Workshops

Learning is as much about doing as it is about hearing and watching. Some conferences may have hands-on sessions or labs, but they’re in the minority. So just going to conferences isn’t good enough.

A more practical variant are workshops. They are mostly organized by specific communities, like Java User Groups.

One particularly useful form for developers is the code retreat. Workshops are much more focused than conferences and still provide some of the same networking opportunities.

Get Formal Training

Lots of courses are being offered, many of them conveniently online. One great (and free) example is Cryptography from Coursera.

Some of these course lead to certifications. The world is sharply divided into those who think certifications are a must and those that feel they are evil. I’ll keep my opinion on this subject to myself for once 😉 but whatever you do, focus on the learning, not on the piece of paper.

Learn On The Job

There is a lot to be learned during regular work activities as well.

You can organize that a bit better by doing something like job rotation. Good forms of job rotation for developers are collective code ownership and swarming.

Pair programming is an excellent way to learn all kinds of things, from IDE shortcuts to design patterns.

Practice in Private

Work has many distractions, though, like Getting a Story Done.

Open source is an alternative, in the sense that it takes things like deadlines away, which can help with learning.

However, that still doesn’t provide the systematic exploration that is required for the best learning. So practicing on “toy problems” is much more effective.

There are many katas that do just that, like the Roman Numerals Kata. They usually target a specific skill, like Test-Driven Development (TDD).