00:00:00 ◼ ► Welcome to Under the Radar, a show about independent iOS app development. I'm Marco Arment.
00:00:06 ◼ ► And I'm David Smith. Under the Radar is never longer than 30 minutes, so let's get started.
00:00:11 ◼ ► So today we wanted to dive into everybody's favorite pastime as a developer, and that is debugging.
00:00:20 ◼ ► It is a topic that I think we probably... it's kind of weird in some ways, because I think I probably spend
00:00:27 ◼ ► at least half, if not a majority of my time in development debugging code that I write.
00:00:33 ◼ ► Yet, in most of my actual formal preparation for being a software engineer, there was very, very little time and attention spent on it.
00:00:42 ◼ ► Almost all of my classes and things were all focused on the actual development and the architecture side of things,
00:00:49 ◼ ► whereas the reality is we spend most of our time fixing the problems and issues that arise in our apps.
00:00:55 ◼ ► And these can range from all kinds of different levels. There's the while you're developing debugging process,
00:01:02 ◼ ► where you write some code, you hit build and run, and you see if it works, and if it doesn't, you try and work out why.
00:01:10 ◼ ► There's the more subtle, something happens every now and then, and you're not really sure why, and you try and track that down.
00:01:19 ◼ ► And then there's the, you know, you're getting crash reports or bug reports from customers in the field, and then you have to try and track those down,
00:01:30 ◼ ► But in all of them, I think there's an element of, you know, debugging is essentially, you have... you sort of identify that there's some problem,
00:01:40 ◼ ► you then try and reproduce that problem in a way that you can then observe that problem,
00:01:47 ◼ ► then you try and isolate where in the actual program that is happening, and then you can try and fix it.
00:01:54 ◼ ► And you kind of go through this cycle of, you identify it, you reproduce it, you isolate it, and then you can fix it.
00:02:02 ◼ ► And you kind of cycle through that, or at least that's the process that I tend to take,
00:02:05 ◼ ► because if I don't go through each of those steps, I find that it's very hard to actually make progress.
00:02:10 ◼ ► You know, if you can't... you know, some bugs really are, by their nature, kind of hard to reproduce.
00:02:19 ◼ ► but if I get a bug report that I can't reproduce in any way, it's just as the same in some ways as not having that bug report in the first place,
00:02:31 ◼ ► So, you know, my first step is always, let's reproduce this, and then, you know, try and find the part of code.
00:02:37 ◼ ► And I think, having been a professional developer for enough years now, I feel like I have a better sort of sense for where the bugs might be hiding.
00:02:46 ◼ ► You know, it's like, I've seen every, you know, not every bug, but I've seen a lot of bugs before,
00:02:52 ◼ ► and you kind of get this sense of like, "Huh, this looks like this is an off-by-one error.
00:02:59 ◼ ► This looks like this is a cache invalidation error. This looks like I'm writing something incorrectly here,
00:03:05 ◼ ► or I forgot a case in a switch statement." Like, you kind of get this feeling for it, and then you can kind of go running around trying to find it.
00:03:12 ◼ ► And then, of course, there's also, I guess, the other kind of debugging, which is even super fun,
00:03:16 ◼ ► which is where there's a bug in the OS that you, you know, where your code is all doing the right thing,
00:03:27 ◼ ► And so, that's an even more fun line of debugging where you kind of get this sense of, at a certain,
00:03:32 ◼ ► you know, you keep going down and farther and farther down the rabbit hole until eventually you're like,
00:03:36 ◼ ► hit a brick wall at the end of it, and you're like, "All right, well, I guess I file a bug."
00:03:40 ◼ ► Yeah, and that's, you know, it's often easy for us to jump to, "Well, this must be a bug in the OS,"
00:03:49 ◼ ► or my favorite one, "This must be a bug in the compiler," or something like that, when we can't figure it out.
00:03:54 ◼ ► That is, for the vast majority of bugs that we face as programmers, that is almost never the case.
00:04:01 ◼ ► It, like, first of all, you are unlikely to ever have a bug in the compiler actually affect you,
00:04:06 ◼ ► and, you know, as you go up the stack of, you know, things that you're relying on, like, the OS does have bugs.
00:04:13 ◼ ► Libraries that you're using will have bugs, but if you run into something in your app that seems like a bug,
00:04:25 ◼ ► really make sure that you can't figure out that it's your bug first, because chances are very, very good.
00:04:32 ◼ ► It's probably your bug. The vast majority of the time of those cases, it will really be your bug.
00:04:38 ◼ ► It's only a question of whether you can find it, but don't just say, "Oh, well, it's a bug in the OS.
00:04:46 ◼ ► Really make sure it's not your bug first, because, again, the odds are strongly in favor of it being yours.
00:04:52 ◼ ► Yeah, because the reality is, like, most of our time, at least in my experience, dealing with bugs,
00:05:00 ◼ ► it's almost always something I forgot to do or a case that I forgot to consider when I was originally doing it.
00:05:07 ◼ ► Not necessarily out of laziness, even. Sometimes it's that. Sometimes you have the always classic,
00:05:14 ◼ ► like, you know, you see an example where it's like, if error doesn't equal nil, then do something with it.
00:05:21 ◼ ► And then if you don't actually do anything, when eventually you do run into the situation where the error is not nil,
00:05:29 ◼ ► But more often, it's just like you forgot to do something, and then it comes back to bite you.
00:05:46 ◼ ► oh, you get a bug report, and you're like, oh, no, that can't be right. I'm sure my code is good.
00:05:51 ◼ ► And then you're kind of like, oh, no, it's actually broken. And then you go to the, like, how did this ever work phase?
00:05:57 ◼ ► You're like, how on earth did this even work? Because it's amazing how, when you look back at your own code,
00:06:07 ◼ ► You put this into Xcode. You come back and look at it, and you're like, how on earth? What am I doing?
00:06:13 ◼ ► And I feel like that is the hardest part, or why debugging is so hard, is you have to recreate all of this sort of state
00:06:27 ◼ ► and then run it through a scenario that's happening, and to then see why it's doing something weird.
00:06:33 ◼ ► And very often, I find, like, the hardest thing is to try and understand even what the code is trying to do in the first place,
00:06:46 ◼ ► and you're like, huh, I wonder why I did that. I don't know anymore. And that makes debugging really, really hard.
00:06:53 ◼ ► - Yeah, and this is, it's also a major argument for keeping things simple and not that clever in the first place.
00:07:01 ◼ ► As we talked about, I think, a while ago now, I try to avoid overly clever code styles,
00:07:08 ◼ ► things like using really dense short hands or really clever mechanics, things like generics in Swift, that kind of thing.
00:07:19 ◼ ► You kind of use overly clever complex language features or constructs where it looks really cool,
00:07:27 ◼ ► and you feel really smart when you do it, but when you have to come back and debug code in six months,
00:07:33 ◼ ► and you have totally forgotten what led you to write that, and you've forgotten the cleverness behind it,
00:07:44 ◼ ► it's often very, very hard to do that. And you often, as a programmer, you might eventually hear the wisdom of,
00:07:52 ◼ ► like, write code for someone else to read, but really what you should be doing is writing code for yourself in three weeks to read,
00:08:05 ◼ ► So if you keep things simple from the beginning and not be too overly clever with the way you write things
00:08:11 ◼ ► and make things straightforward to read and to kind of self-document, then you will be doing yourself a huge favor.
00:08:19 ◼ ► Documentation, separate documentation, can help with some of these things, but I've found in practice that,
00:08:26 ◼ ► maybe it's just me, I don't document things that well. I try to just write things that are obvious when you look at them,
00:08:32 ◼ ► and that just have good names and straightforward approaches, and that tends to work very well.
00:08:41 ◼ ► And then for minimizing bugs to begin with, I think it's, again, very important not to be too clever and to realize,
00:08:51 ◼ ► you know, for me, many of my bugs come from quick workarounds or quick hacks that I do when I encounter some other bug.
00:09:06 ◼ ► whenever you are doing something in, like, a launch loop or in a UI, you know, response loop or something,
00:09:15 ◼ ► and there's some weird behavior that, or some weird precondition that you can't quite accommodate in a straightforward way.
00:09:32 ◼ ► And that's a really nice, quick way to avoid a lot of, like, you know, weird complexity or obvious bugs as you're developing.
00:09:42 ◼ ► But I have found in almost every case that using a dispatch async is almost always a bad idea,
00:09:54 ◼ ► Like, obviously there are uses where it's fine, but, like, if that's your approach to solve a problem,
00:09:59 ◼ ► then you get into concurrency bugs and weird conditions there that are really hard to fix and diagnose and reproduce.
00:10:10 ◼ ► And, again, it's like one of those areas where it seems like a clever thing when you're writing it,
00:10:18 ◼ ► and in the case of, like, bug avoidance, like, it seems like, "Oh, this will be a quick fix for this weird little bug.
00:10:28 ◼ ► But then you introduce all these weird other problems that might be non-obvious, and those become very hard to debug.
00:10:42 ◼ ► I also would strongly recommend avoiding concurrency if you can and avoiding playing with threads and queues
00:10:54 ◼ ► Yeah, because I feel like what you're doing there, I mean, and I'm very guilty of that.
00:10:57 ◼ ► Like, there have, I have done the, you know, dispatch async fix probably a hundred times,
00:11:06 ◼ ► And I cannot overstate how many bugs that I've had to fix end up being a poorly used dispatch async.
00:11:14 ◼ ► Yeah, because I think the reality is what you're doing, and this is maybe a maturity thing,
00:11:19 ◼ ► but it's the understanding that, like, when I do that now, I understand that I'm not fixing the problem.
00:11:33 ◼ ► And, you know, that's a problem, you know, one percent of users are hitting this issue.
00:11:46 ◼ ► but also incredibly difficult now to actually, like, properly fix and diagnose and reproduce, etc., etc.
00:11:54 ◼ ► Like, it is the kind of fix where it isn't actually solving it. It's just making it less likely.
00:11:59 ◼ ► Which sometimes, you know, pragmatically, I've, you know, I'm sure there are many places in, you know,
00:12:03 ◼ ► my shipping apps where I have that, and it's just one of, it's like a conscious choice that you're just like,
00:12:07 ◼ ► "Well, the actual fix is too hard or would require a massive refactoring or some kind of, you know,
00:12:22 ◼ ► I know that, you know, there is, this bug is, hasn't magically disappeared as a result of this,
00:12:31 ◼ ► But, yeah, ultimately, you know, most, when you're writing the code, you're kind of trying to do your future self a favor,
00:12:36 ◼ ► as much as you can. Because I think one of the things that I struggle with, too, when I'm writing code
00:12:41 ◼ ► is the realization that all of these things that seem obvious to me when I'm writing the code,
00:12:47 ◼ ► because I'm in a mindset where I'm focused entirely on whatever the solving this problem is.
00:12:52 ◼ ► Like, there's all these things that just seem obvious. Like, of course, you know, like, this is the way this works,
00:12:56 ◼ ► you know, that's why this is set up this way, this value will, you know, will always transform from this to this.
00:13:01 ◼ ► Like, there's all these things that seem obvious. And then, in the future, they're entirely non-obvious.
00:13:12 ◼ ► is that it's kind of impossible to know what your future self is going to want to know,
00:13:46 ◼ ► There's a reason for that, and if there is, like, you know, that's the kind of thing that you write down.
00:13:50 ◼ ► But otherwise, you end up with, like, writing your program twice if you're, you know, overly documenting it.
00:13:54 ◼ ► So, as much as when I'm debugging, I kind of wish my past self had written all this stuff down,
00:14:04 ◼ ► because if they did, they could have just written, you know, made the code more obvious or clear in the first place, probably.
00:14:21 ◼ ► You got a 14-day free trial, and when you use code RADAR at checkout, you got 20% off your first invoice.
00:14:30 ◼ ► They are focused on making the web faster and more reliable for everyone who has a site,
00:14:39 ◼ ► So, if you're a Pingdom user, you can monitor the availability and performance of your servers,
00:14:47 ◼ ► They test for more than 70 global test servers, and they can emulate visits to your site as often as every minute.
00:15:04 ◼ ► Maybe you depend on an external service, or maybe only the login screen does this one thing in your code.
00:15:09 ◼ ► And you need to check all these things, and you can do all that with Pingdom, and loads more in addition to that, too.
00:15:15 ◼ ► They can make it possible to monitor the availability of all these key interactions people have with your site.
00:15:20 ◼ ► Because, you know, a lot of times it's more complex than your site is up or your site is down.
00:15:32 ◼ ► So, regardless of whether you have a small website or you're managing a whole infrastructure,
00:15:47 ◼ ► It's so configurable. We've even used it in the past to monitor other people's websites.
00:15:52 ◼ ► Like, David and I have both independently used it to monitor the WWDC page for changes.
00:15:58 ◼ ► Like, in the past, when it was really important to know when WWDC tickets went on sale.
00:16:04 ◼ ► Check it out today and you will be the first to know when your site is down or being too slow.
00:16:10 ◼ ► So, go to Pingdom.com/Radar for a 14-day free trial and use code radar to get 20% off your first invoice.
00:16:21 ◼ ► So, the other thing that's probably worth diving into a little bit, too, is the way in which we actually do debugging.
00:16:29 ◼ ► Because maybe it's a shocking revelation, but I am, I believe, what is technically called a caveman debugger.
00:16:40 ◼ ► I do all of my debugging using, you know, nslog and the console, which I think is often disparagingly referred to as caveman debugging, which is fine.
00:16:53 ◼ ► I mean, every now and then I'll go to a WWDC session where they're talking about all the crazy things you can do in Xcode, where you can set conditional breakpoints and all this kind of really clever stuff.
00:17:04 ◼ ► And in the end, what I always do is just, I end up just nslogging tons of data to the console.
00:17:11 ◼ ► Because it's one of those things where most of what debugging is trying to do is, it's like you're trying to unwind the state machine that is your app.
00:17:20 ◼ ► Where you're trying to work out, it's like, your app is in this problem, you're in a good state, then something happens and you're in a bad state.
00:17:30 ◼ ► And what you're trying to do is identify that, you know, at what point it's going from that good state to that bad state.
00:17:52 ◼ ► Like, I can identify and find things fairly quickly this way because you kind of get the sense of, you know,
00:18:00 ◼ ► And so you'll end up putting nslogs, you know, in different parts of your app that you think might be part of it
00:18:11 ◼ ► And then something that I also find really helpful is when I format my nslogs as tab-delimited text.
00:18:25 ◼ ► And I nslog those as a tab-delimited set of strings because then I can take the output from the console
00:18:53 ◼ ► I'm taking something out of the console, looking at it, often in Excel, and then kind of finding the pattern,
00:19:28 ◼ ► And so the nice thing about a console log is that you can have those running without being attached to the debug server, for example.
00:19:56 ◼ ► But I'm very aware that by doing it that way, I'm not taking advantage of more sophisticated tools.
00:20:13 ◼ ► For example, NSLog debugging, which I do a lot myself, is not necessarily the best option if you have a nitty-gritty code level bug.
00:20:30 ◼ ► So I can see whenever this value is something out of whack, stop here and then I will inspect the stuff around it.
00:20:35 ◼ ► But NSLog debugging is a lot more helpful for things like larger systems interacting with each other,
00:20:43 ◼ ► any kind of large operations, concurrency, things where you're not really sure where the bugs might be,
00:20:54 ◼ ► NSLog debugging is also very, very helpful in situations in which the debugger itself is impractically slow to do things.
00:21:12 ◼ ► Because then the round trip between your computer running Xcode and the phone and the slow watch hardware,
00:21:19 ◼ ► trying to break watch apps with breakpoints and inspect local variables around and running things in the debugger command line,
00:21:27 ◼ ► is so slow when doing things on the watch that sometimes it's better off to just do something,
00:21:36 ◼ ► For me, I also debug very differently if I'm looking at a crash report versus some kind of behavior in the app that I'm catching live.
00:21:45 ◼ ► It's very important to look in iTunes Connect and to look at the, or rather I guess now it's in the organizer in Xcode,
00:22:17 ◼ ► but there are some types of crashes that only Apple's crash logs will generally, reliably be able to report to you.
00:22:24 ◼ ► That includes things like if iOS kills your app for some reason that isn't necessarily a crash,
00:22:30 ◼ ► but if your app is getting killed in the background or something, or if your extensions are getting killed,
00:22:49 ◼ ► I was just spending a lot of time debugging crashes in my today widget, or my widget I guess,
00:23:14 ◼ ► And there's a great technical note, TN-2151, Understanding and Analyzing Application Crash Reports.
00:23:25 ◼ ► So one of the things that I was facing is my extension was accessing the SQLite database
00:23:42 ◼ ► at the time that the OS suspends you, it will kill your app with a certain code that says deadlock in hex.
00:23:48 ◼ ► And if you didn't look at this document, and if you didn't find that crash report and find that code,
00:23:57 ◼ ► and it tells you right here, it's been terminated by the OS because it held onto a file lock
00:24:03 ◼ ► And before I found all this out, I just didn't know why my extension just kept crashing for everybody.
00:24:11 ◼ ► And people kept reporting things like, "Hey, your app is crashing in the background for some reason.
00:24:24 ◼ ► and if you look up what these different termination codes mean, you'll probably find out something useful.
00:25:04 ◼ ► And one of the reasons why I do also, as I said earlier, kind of recommend against unnecessary use of dispatch async
00:25:22 ◼ ► Some of the more advanced ones, like hockey, try to do this, and they have mixed degrees of success in my experience.
00:25:28 ◼ ► And so if your code is simpler, then any crash log you get will also be more useful to you.
00:25:41 ◼ ► And I feel like, in general, I will say that Xcode, as much as I don't use all of its tools,
00:25:46 ◼ ► some of the stuff that you're talking about, of going into the organizer, there's some really useful tools that we have at our disposal
00:25:59 ◼ ► It makes me think even of one of the other tools that I use a lot in debugging, in a strange way, is version control.
00:26:07 ◼ ► And specifically, get blame is often incredibly helpful in trying to track down weird situations in my apps,
00:26:18 ◼ ► where code is doing something weird, and I'm trying to work out why I structured it in the way I structured it.
00:26:23 ◼ ► And this is something that I love in Xcode, where you can go into the top right, where you go into the source control thing,
00:26:39 ◼ ► And so you often will end up happening there. By using that view, I can see, "Huh, that's weird.
00:26:45 ◼ ► Here's this one random line of code that is left over from an old version, that everything else was updated."
00:26:51 ◼ ► Or situations like that, and Xcode is really, and then you can, if you double click on it, it will show you that commit,
00:27:04 ◼ ► And that's another example of one of these little tools in Xcode where it's like, it helps you understand what's going on,
00:27:13 ◼ ► I mean, I love in the crash logs where it can help you, take you right to where the crash happened,
00:27:18 ◼ ► rather than sitting there and trying to work out which, you know, if this crash happened in this file on this line number,
00:27:27 ◼ ► like, well, that's useful, but it's also way more useful to just take me to that line number.
00:27:32 ◼ ► So there's definitely some great tools in Xcode that I think make a lot of debugging a lot simpler, and that I'm very grateful for.
00:27:38 ◼ ► I would also say, when you're running your app and you're trying to figure out maybe what's going on with the bug,
00:27:43 ◼ ► always watch that pane in Xcode that shows you the running CPU usage and memory of the app.
00:27:50 ◼ ► And of course, you can go into instruments and get, you know, like a more fine-grained version of this.
00:27:57 ◼ ► Like, the app just started behaving really weirdly when loading episodes of a certain podcast, and it, like, really weirdly.
00:28:09 ◼ ► And what led me to the solution was that I was, I happened to be glancing at that pane that was showing me the resource graphs in Xcode during one of the runs,
00:28:18 ◼ ► and I noticed that right before it crashed, memory usage skyrocketed all the way up to, like, a gig, and then everything freaked out and blew up.
00:28:31 ◼ ► Let me run it under the allocations instrument run that can show you where all your memory is being used."
00:28:47 ◼ ► And then within a half hour of noticing, "Hey, this thing is out of the ordinary on the memory meter here," I had to fix.
00:28:52 ◼ ► So, like, you know, use the tools that are available to you when appropriate, or just use nslog and printf debugging everywhere.
00:29:01 ◼ ► Yeah. And, I mean, the love of what you hit on there, though, is one thing that I will say that I love about debugging is that so often you get to the end,
00:29:08 ◼ ► and there's, like, a relatively straightforward solution that fixes the problem, and, like, that is one of the greatest joys of being a software development.
00:29:16 ◼ ► When you, like, you see the problem, you see the solution, and you can implement it, and, like, it's gone.
00:29:22 ◼ ► And this thing that was problematic and hurting your users is now just completely vanished.
00:29:30 ◼ ► It doesn't always happen, but when it does, it's absolutely awesome. So it's the one reason I love debugging.
00:29:35 ◼ ► That's why we do all this, right? That feeling when it finally works. We're like, "Yes! I love that feeling."