Suddenly, I Understand Software

Even though I don't respect podcasts as an information delivery system, I recently listened to a podcast that felt like having an epiphany.

The podcast in question was episode 61 of Future of Coding. It is essentially a read-through/review of two papers with which I was unfamiliar, but which I believe are actually quite famous and influential.

Why did listening to this feel like an epiphany? Well, I suddenly felt like I understood what the deal is with software. Why is it that when you join a company, the engineer who's been there for years seems like an incredible genius? Why do some teams that I've been on struggle while others manage to get everything right? Why is everyone always so keen to rewrite things?

The two ideas that the podcast expresses are:

• The concept of what a “Theory” is, according to Gilbert Ryle.
• That being a programmer is doing “Building theories” in the Ryle sense of the word.

Having these two ideas explained together was really helpful. If I had read Ryle by himself, I would have thought, “interesting and useless”. If I had read Programming as Theory Building without knowing the theory concept, I would just not have understood.

I recommend listening to the podcast and reading the paper. But if you don't want to do that, I'm going to try and explain the two points.

---

What is a Theory, According to Gilbert Ryle?

When Ryle says theory, he doesn't mean anything like what other people mean when they say theory. Annoying. He should have just come up with a new word. What he means is the thought object that exists in our minds which allows us to do things.

I, for example, know how to cook pasta. When I cook pasta, that's a certain expression of this knowledge. When I try and explain to you how to cook pasta, that's a different expression of it. Neither of those expressions contains everything I know about cooking pasta. And in fact, there are parts of what I know that I can't really express in any way. This knowledge is what Ryle would call a theory. I have a “Theory of how to cook pasta”. This theory is not something that exists in language or action - it’s a something that we can never fully express. The closest we can get to transferring a theory is to demonstrate the expression of the Theory to someone over and over again until they build their own theory. That theory won't be the same as ours.

What Does it Mean that Programming is Theory Building?

It means that the code base we create is not the true product of our work. The real product is the mental theory of that code base which:

• Allowed us to create it in the first place.
• Allows us to diagnose problems with it and fix them.
• Allows us to modify it easily.

If I think about times when I've worked on a team that works well and gets stuff done, it's been a team where:

• Someone has been there for a long time, since the start of whatever code base/feature we work on.
• Other team members have joined slowly, and had a chance to work with the people who know more.
• The area of focus does not change. We haven't been reassigned to a random existing project, or asked to fix some other team’s work.

The paper also talks about what happens when all the people who have a theory of a given program stop working on it. It dies. Yikes. It’s claimed that we can’t rebuild a theory from code and documentation.

This model explains a few curious phenomena:

• What “legacy code” actually is - it’s a code base which is no longer maintained by people who have a theory of it.
• The solo engineer who can make a better product than a team of equally competent professionals. The solo engineer has spent the time to build a complete theory of their program, the professionals move between projects regularly - and only have theories of what they've worked on.
• Why getting up to speed with unfamiliar projects is so much harder than just rebuilding the thing. To truly build a theory, you need to mentally rebuild the existing code base anyway.
• Why outsourcing, or hiring contractors, always seems to go so badly.

---

What You Can Learn from This

Retention of software engineers is really important!!!