Developing Perspective

#207: Irresistible Complexity


00:00:00   Hello and welcome to Developing Perspective. Developing Perspective is a

00:00:03   podcast discussing news of note in iOS development, Apple and the like. I'm your

00:00:08   host David Smith. I'm an independent iOS developer based in here in Virginia.

00:00:11   This is show number 207 and today is Saturday, January 10th. Developing

00:00:16   Perspective is never longer than 15 minutes, so let's get started. Okay, so for

00:00:20   the purposes of today's episode, I wanted to wade very carefully into the

00:00:26   kind of hullabaloo that had blew up this last week regarding the quality of Apple's software.

00:00:31   And as I always try and do on the show, rather than addressing it directly, rather than just

00:00:37   kind of weighing in, as it were, which, you know, is certainly a valid thing to do, but

00:00:42   for the purposes of what I try and do on the show isn't particularly relevant, because

00:00:46   it is only slightly helpful. What I want to do is instead take a step back and think about,

00:00:52   Well, what are the fundamental pressures going on

00:00:54   that are causing this?

00:00:56   And then as a result, what of those pressures

00:00:59   could apply to myself or to someone else

00:01:02   or to anyone making software?

00:01:03   Because while it's interesting and certainly relevant

00:01:06   to talk about whether the quality of Apple's software

00:01:08   is diminishing over time, whether that is true

00:01:11   or whether it's not, the degree to which that is happening

00:01:14   and the reasons why it may be happening,

00:01:16   talking about the software quality of one

00:01:20   of the largest companies in the world

00:01:22   and how their giant, massive engineering organization,

00:01:25   compared at least to mine, is functioning, is interesting.

00:01:28   But that really won't help me make better software.

00:01:31   But I think there are a lot of lessons that we can be thinking

00:01:34   about that are the pressures and the reasons why

00:01:37   Apple is in the situation they are in that are relevant

00:01:40   and that are helpful more generally.

00:01:42   And so what I'm going to talk about today

00:01:45   is this concept that I've been-- all week,

00:01:46   I kept coming back to, which is the concept

00:01:49   of irresistible complexity, that over time, software,

00:01:53   it's like its natural state is to get more

00:01:58   and more complicated.

00:01:59   In some ways you could think about this almost as entropy,

00:02:02   which is like the concept in thermodynamics

00:02:03   where a system will always become more and more chaotic

00:02:06   unless it's acted on actively, and it takes energy

00:02:09   to keep something unchaotic.

00:02:11   Left to itself, it'll become more and more chaotic

00:02:14   just on its own.

00:02:16   And that same thing seems to always apply to software, which is in some ways kind of

00:02:21   weird.

00:02:22   You know, software shouldn't in some ways be simple and clean because it's completely

00:02:28   an artificial construct.

00:02:29   It's something that is entirely man-made and doesn't, in theory, then wouldn't have all

00:02:35   of the natural pressures of something like thermodynamics on it.

00:02:39   But the reality with software is that in many ways it is perhaps one of the most quickly

00:02:45   complexity gathering things I'm aware of.

00:02:48   It's incredibly hard to avoid complexity

00:02:51   in both the creation and the actual execution of software.

00:02:54   And I kind of ended up with three reasons

00:02:57   why I think that's the case.

00:02:59   And a lot of why, and I bring,

00:03:01   just to loop that back quickly to why I'm bringing it up

00:03:03   in relation to Apple's software quality questions,

00:03:06   is I think a lot of what Apple is facing

00:03:08   is this overwhelming increase in complexity over time

00:03:13   and how it becomes ever more difficult to keep ahead of it.

00:03:17   That there are processes you can put in place,

00:03:19   there are-- you can adjust your timeline,

00:03:22   or your engineering talent, or all these things

00:03:24   to potentially address that kind of a question.

00:03:26   But why are you having to fight back this giant wave

00:03:30   in the first place?

00:03:31   And so the first thing, of my three points about complexity,

00:03:34   the first kind of complexity that I

00:03:36   wanted to talk about is what I'm going

00:03:37   to call marketing complexity.

00:03:40   And this is something that isn't really

00:03:43   intrinsic to the software itself,

00:03:45   but is nevertheless one of the stronger driving forces

00:03:48   in the software's creation in the first place.

00:03:51   Because in the kind of software we're talking about,

00:03:53   whether it's software Apple's making,

00:03:54   or software I'm making to sell in a store,

00:03:56   or software you're making for a company or for yourself,

00:03:59   usually the motivation is that you are writing code

00:04:03   for the purposes of selling it.

00:04:05   The same thing applies even if you're a consultant.

00:04:07   Like what you are doing is selling something,

00:04:09   And you need to, therefore, have a market

00:04:13   who is willing to buy whatever it is that you are selling.

00:04:17   And so it's a little bit complicated,

00:04:19   because you build something.

00:04:21   You build version one.

00:04:22   It's awesome.

00:04:22   It's great.

00:04:23   Maybe you have a few bug fixes, and no one's perfect.

00:04:26   You end up with this nice, stable version.

00:04:28   You've got version 1.1, 1.2, whatever it is.

00:04:31   Nice and stable.

00:04:33   Now, the challenge becomes-- and this is in many ways driven

00:04:36   by the kind of business models that a lot of software

00:04:40   app products have is that you then

00:04:42   you start to think about, well, how can I sell this again?

00:04:46   It would be great if you could just

00:04:48   continue to sell version 1.2 over and over and over again

00:04:51   forever at the same rate and at the same quantity.

00:04:55   That'd be awesome.

00:04:56   But that's not the reality, because people look at it

00:04:58   and say, OK, that's great.

00:04:59   But if I'm going to pay more money,

00:05:01   if I'm going to do something else,

00:05:02   if you're stuck into that kind of the old school paid upgrades

00:05:05   model, then it gets really hard to be like, well, what have you given me recently?

00:05:11   Why is it worth me doing?

00:05:12   And so you kind of have to, in some ways, make something better.

00:05:17   And that definition of better is pretty loose because typically what ends up happening with

00:05:22   this is you keep adding features.

00:05:25   You know, you kind of want to be able to say, in this version we had this, and now in this

00:05:28   new version we have this plus all this other stuff.

00:05:31   We want to be able to put up a big slide that says it has 72 features and 1500 new APIs

00:05:36   or whatever.

00:05:37   Like this is the pressure that we always have because people like the thought of that.

00:05:42   Whether or not that's actually good for them.

00:05:44   I think most people like the thought of, "Ooh, this is better.

00:05:46   This is different.

00:05:47   This has stuff that I didn't have before."

00:05:50   And necessarily in some ways, everybody I know who uses something can always have an

00:05:55   opinion about it, right?

00:05:56   You always can think of like any piece of software, you probably have a list of things

00:06:00   that you would have done differently. And you're telling the software creator about

00:06:04   that. I get constant emails from people making suggestions about how I could improve my software.

00:06:11   And for that user, it may improve it for them. But in the scope of all of what I'm doing,

00:06:16   that may actually not make my software better. It just might make it more complicated. But

00:06:20   this is always the pressure. That there's always this pressure to, in order to keep

00:06:24   your business viable, that you have to add features. That selling the basic thing for

00:06:30   for the long term only sometimes works.

00:06:33   And that's a bit of a tragedy, but it's sort of the reality.

00:06:36   And you can fight it a bit.

00:06:37   Some of my products are kind of famous for their simplicity.

00:06:40   And that's great, but it gets really complicated for me

00:06:43   when my competitors are everything I do plus all

00:06:46   this other stuff.

00:06:47   And how do I compete with that necessarily?

00:06:49   Because not everybody would appreciate the fact

00:06:51   that using pedometer++, for example,

00:06:54   is probably easier to use than something

00:06:56   that did the basic pedometer functionality plus all

00:06:59   other stuff. And also of course, the simpler you are, the easier you are to be copied.

00:07:03   And so that has all kinds of other business implications. And so there's always this marketing

00:07:07   pressure to add more complexity, to add more features, to add more stuff to your software.

00:07:13   And that kind of dovetails into the next type of complexity, what I'm going to call intrinsic

00:07:17   complexity. And this is just something that I've always noted about software that makes

00:07:22   it kind of unique from a lot of other disciplines where you're making something, is that because

00:07:29   Because software isn't limited by the constraints of the physical world, where I'm not saying

00:07:36   that building a bridge is easy, building a bridge is ridiculously complicated, but the

00:07:40   bridge is operating in an environment where with a very known set of interactions and

00:07:46   there's only so many dimensions by which something can interact.

00:07:50   There's only so many physical dimensions that something can work with.

00:07:53   Whereas with software, you can build systems that interact in so many different directions

00:07:58   and in so many different ways at the same time.

00:08:00   And that sort of like multi-dimensionality of it,

00:08:03   it makes it really complicated as you add anything

00:08:06   to a piece of software.

00:08:07   It doesn't just sort of linearly grow

00:08:10   in terms of its overall complexity.

00:08:12   It tends to grow much quicker than that.

00:08:14   This reminds me in some ways of like the handshake problem,

00:08:17   which is that sort of math thing you may have done

00:08:20   in grade school where you kind of have to think about

00:08:22   if you have n people in a room

00:08:24   and they all want to shake hands with each other,

00:08:27   how many handshakes are exchanged, which by the way, in case you're curious, is n times

00:08:31   n minus one over two, I believe.

00:08:34   But that same thing applies to software, where if one more person walks into that room, the

00:08:41   number of handshakes doesn't go up just by one.

00:08:44   It goes up by the number of people who were already in that room before they walked in.

00:08:50   That is where it gets really, really complicated, because anything you add doesn't just exist

00:08:55   on an island on its own.

00:08:56   It has to interact with all kinds

00:08:58   of other parts of your system.

00:08:59   And so every little change you have,

00:09:02   it creates this ever-increasing volume of complexity

00:09:07   that you have to manage in your software methodology,

00:09:11   in your testing, in everything you do,

00:09:13   because now there's this other angle by which something

00:09:15   can fail.

00:09:15   Now there's this other angle by which

00:09:17   something can have a problem.

00:09:18   And this is difficult. This is just

00:09:20   sort of the intrinsic problem of if you take the marketing

00:09:22   complexity that you have to keep adding features as a given,

00:09:25   then your software will necessarily have this weight constantly growing on it and growing

00:09:31   in a way that isn't just sort of linear and easy to manage.

00:09:34   It grows, I'm probably using the term, but it grows exponentially.

00:09:38   It becomes harder and harder to do.

00:09:41   And this gets especially tricky too because hardware is also always advancing.

00:09:46   And at least in the way that most people deal with hardware compatibility is you can't just

00:09:51   say this new thing only works on the current version of whatever

00:09:55   it is.

00:09:55   Maybe you can get away with that if you don't have a huge

00:09:57   install base.

00:09:58   But if you're Apple or you're an app in the App Store,

00:10:01   you want to at least have fairly wide compatibility.

00:10:04   And so now there's this other dimension

00:10:05   by which you have to keep track of everything

00:10:07   and be doing all your testing and weird different stacks

00:10:10   and different devices.

00:10:11   And even if you didn't have the marketing complexity saying

00:10:15   that you needed to add features, you at least

00:10:17   have to support the new hardware.

00:10:18   That Moore's law is putting this new stuff into you

00:10:21   and you have to deal with that. And so there's this intrinsically difficult problem that

00:10:26   to solve in software that you there's just no way around. You can't if you add and add

00:10:30   anything to a piece of software, it is going to make it much harder down the road. If you

00:10:35   think about iOS, it's eight years now or something that that it's been under development, I'm

00:10:39   sure there is a tremendous volume of interaction and coupling between components on the system

00:10:44   matter how clearly and cleverly you engineer it, it is going to be really hard to add anything.

00:10:50   especially in recent times when the features that they've been adding are so pervasive.

00:10:53   And lastly, I wanted to talk about what I'm going to call personnel complexity or personal

00:10:58   complexity.

00:10:59   You know, either way, however one way you wanted to spell it is actually valid.

00:11:03   But this is the complexity that I see in myself and I see probably in a company like Apple

00:11:08   that you always have the difficulty of if you have good engineers, if you have good

00:11:13   people who are smart and talented and driven building your software, they are always going

00:11:18   to be wanting to solve not only hard problems, but interesting problems.

00:11:24   And making something stable is not necessarily an interesting problem, at least not for myself.

00:11:30   And I know a lot of really smart engineers for whom that's difficult.

00:11:34   I've met some amazing engineers who love maintenance.

00:11:37   And if you ever come across one of those, like just wrap your arms around them, give

00:11:40   them a big hug, because they are amazing and awesome and people who help make software

00:11:46   actually work in the real world.

00:11:47   But by and large, a lot of the best engineers

00:11:49   I've known in real life don't like maintaining software.

00:11:53   They like building something.

00:11:54   They like doing something new and interesting and awesome.

00:11:57   And so it gets really hard to then either retain talent

00:12:01   in an environment that is focused on stability, that

00:12:04   is focused on cleaning things up and making them bug free.

00:12:08   It's easier to keep those people engaged

00:12:10   when you're doing something new, when you're

00:12:12   working on something completely fresh and interesting,

00:12:14   a new feature, a new device, a new platform, whatever it is.

00:12:17   That's a much easier way to get and attract those people.

00:12:21   And so there's always this pressure inside of a software

00:12:24   engineering project to tackle a new problem, in some ways

00:12:27   just for the purpose of attacking the problem,

00:12:29   whether or not it's actually making the software better,

00:12:31   whether or not it's actually making it better.

00:12:33   And I see this all the time in my own software.

00:12:35   I often see myself wanting to add a feature,

00:12:38   not necessarily because-- or honestly, for me,

00:12:41   it's I want to create a new app because I

00:12:42   have a problem with that.

00:12:44   But you often-- I see I want to do that because it's

00:12:46   interesting.

00:12:47   Not, and it's much harder to keep myself motivated about saying, let's just keep working on making

00:12:52   this better and better, more and more bug free.

00:12:54   And you see this and you can kind of imagine it as I think about any large software company

00:12:57   and you know, for the purposes of today, I imagined at Apple, that the best engineers

00:13:01   at Apple at this point in the development cycle, once they probably locked down most

00:13:05   of the really grievous bugs in iOS, you know, I'm sure they're eager and you know, they're

00:13:11   they're chomping at the bit to get at the new features to get at the new stuff to work

00:13:15   on the hot new feature, whatever it's going to be in iOS 9.

00:13:18   And so it's really hard to keep that focus back

00:13:21   and to avoid this pressure to keep

00:13:25   adding new and more complicated things,

00:13:26   because that's the way the human mind works.

00:13:28   So these are the pressures.

00:13:30   These are the things that, in our software development,

00:13:32   are putting a pressure on us towards making

00:13:34   our software more and more complicated,

00:13:36   and with our complexity, likely bringing with it

00:13:38   more and more bugs, less and less stability.

00:13:40   And we have to fight these.

00:13:41   I think the first step, and why I wanted to do this show,

00:13:44   was to make sure that we're aware of this.

00:13:46   That the first step in being able to effectively fight

00:13:49   this complexity and this tendency in our software

00:13:51   is being aware of it and thinking about it critically.

00:13:54   That when I'm adding a feature, am I doing it

00:13:55   for a marketing reason?

00:13:56   Am I doing it in a way that is respectful

00:13:58   of the intrinsic complexity this will add to my software?

00:14:01   Am I doing it just for my own reasons,

00:14:02   for personal or personnel reasons,

00:14:04   to keep things interesting?

00:14:05   And if I am, I probably need to be thoughtful about that.

00:14:08   I mean, you need to be doubly cautious about it

00:14:10   to make sure that I'm taking these other considerations

00:14:12   into account as I'm doing it.

00:14:15   All right, that's it for today's show.

00:14:16   As always, if you have questions, comments, concerns,

00:14:18   or complaints, you can email me, david@developingperspective.com.

00:14:21   I'm on Twitter @_davidsmith.

00:14:23   Otherwise, I hope you have a great week.

00:14:24   Happy coding, and I will talk to you later.

00:14:27   Bye.