PodSearch

Under the Radar

298: The Storm

 

00:00:00   Welcome to Under the Radar, a show about independent iOS app development. I'm Marco Arment.

00:00:05   And I'm David Smith. Under the Radar is usually not longer than 30 minutes, so let's get started. Congratulations!

00:00:11   Thank you so much. So the Overcast rewrite has launched. First of all, I must thank you for many things.

00:00:20   One of which, at the top of my mind right now, is thank you for convincing me to do the phased rollout and not release it all at once.

00:00:28   I can say that it's something that I have learned the hard way.

00:00:33   Yes. So it turns out you are occasionally right about things. I'm glad I did that because the very first day, which was a few days ago now,

00:00:44   the very first day, there weren't any massive problems, but there were some small problems, and I was able to get a 0.1 update out later that day,

00:00:55   or actually the following morning, that addressed a bunch of those. So thank you for that.

00:01:00   But last episode we had the storm before the storm, and now I think we have the storm.

00:01:07   Because it's all going well. However, there is a lot to do. There is a lot to deal with. There's a huge amount of feedback.

00:01:16   So just for reference, right now, I did the phased rollout and paused it after 1% on day one because there were some bugs I wanted to fix.

00:01:25   Then on day two, I did the second update, I had that phased rollout as well. So that's still going out. I think I'm at the 10% mark or something like that now.

00:01:33   But just for reference, I already have something like 16% of my user base using the new version.

00:01:40   Part of this is because they just, maybe they saw it breeze by in my announcements. Like if they followed me in my social accounts or whatever,

00:01:49   they might have seen that, or there were a couple of good press articles, they might have seen those, they might have gone to manually update before the app store would have pushed it to them.

00:01:57   So I actually have a decent number of, you know, it's like 16 to 20%, depending on how it's measured, of my users are currently on one of these new versions.

00:02:06   The downside to that is that I currently, just unread in my inbox and Mastodon, have about 500 Mastodon replies to the Overcast account that I have not seen yet.

00:02:22   And I currently have 1200 emails that are mostly, and this is only from the last three days, that I'm just slowly falling behind further and further on.

00:02:35   So I'm definitely, and this is, again, this is only with 16 to 20% of the user base having this version of the app.

00:02:46   So I'm thinking a couple of things. Number one, I'm thinking I maybe should pause this rollout again today and let myself catch up on some of these common bugs and feedback items.

00:02:58   And number two, I'm kind of thinking I should find a better solution to sifting through all of these than just me, like, reading and deleting each one, one by one, and slowly adding them to Apple Notes when I discover new bugs and stuff like that.

00:03:12   So there's a lot going on here. But the good thing is that so far, the feedback has been either bug reports or mostly good and a little bit bad.

00:03:27   And that's, given how much I changed in this update, I think that's a pretty big win.

00:03:34   Like, I was very worried last episode that the feedback would be more negative.

00:03:40   Now, and again, in all fairness, because of who has it so far, I expect the feedback to trend more negative over the following couple of weeks.

00:03:48   Because as the phased rollout gets to everybody, it's going to reach people who don't follow me on social media, who are not ATP listeners, who are not reading, like, the websites that did press articles for me.

00:03:59   And I would expect those users to be less interested in my reasons for changing things and maybe have less context for them.

00:04:07   And therefore, I would expect those users to trend more negative in their reviews.

00:04:11   So I don't think I'm going to be out of the storm yet for a little while. But with as much as I changed, I really thought there would be way more negative feedback.

00:04:22   Things I thought would be small problems have been big problems, and things I thought would be big risks have been small problems.

00:04:31   So for instance, I made a few little feature removals here and there of the app, but only a couple of -- only one big one, I thought.

00:04:41   The big one I thought was streaming. And I explained why in ATP, and basically the gist is removing streaming was kind of necessary to deal with the realities of today's dynamic ad insertion in podcasts.

00:04:53   Because any two subsequent requests to the same file are not guaranteed to get the same file, and so if you start playing an episode, it starts buffering, the download fails in the middle, and then you, like, hit that boundary in the audio, and it's been resumed from that point.

00:05:07   The resumed download might have different ads of different lengths at different times in the file, and so you might have a weird audio splice at that point, and you might get doubled content or missing content.

00:05:16   It's a whole mess. So anyway, I removed streaming for lots of reasons. That's one of the big ones. But also, having the audio engine not support streaming at all allows me to both have some feature I couldn't do before, and also that allows me to pave the way to modernize the audio engine into the Swift and Async world in the future in ways that would be way, way technologically simpler and just less complicated.

00:05:42   Not easy, but less complicated. So there's big future reasons why I wanted to see if I could get away with that streaming.

00:05:49   And I thought the vast majority of the negative feedback would be about the streaming removal. And it turns out most people don't care. Even streaming users, some of them have said, "You know what? You're right. I've been using this, but it does have all these weird problems and bugs when dealing with DAI."

00:06:04   And so a lot of people have actually said, "Good. I'm glad you removed it." So that has proven so far to be not that bad of a removal.

00:06:12   Meanwhile, I removed a small preference that I haven't given much thought to over the years because it doesn't usually come up called "one-tap play."

00:06:21   And so the way in Overcast, the way the episode sells, when you tap them, they expand into that little button row, and then you can tap the play button from there.

00:06:29   Well, I had an option from a long time ago to make it so that if you tap the episode row, instead of expanding into the button row, it just starts to play back.

00:06:37   And then it swapped the info button on the right side for a drop-down button that would drop those tools down if you wanted that instead. So it kind of flipped the behavior of tapping the cell.

00:06:46   And I removed that thinking, "I had this a long time ago. I hardly ever test this. I don't want to write this entire UI twice. Let me remove that and see if anybody really cares."

00:06:57   Turns out a lot of people care. That, I think, has been a more controversial removal than streaming. So we'll see. I might bring that back. I haven't decided yet.

00:07:05   Right now I'm still in bug fix mode, so I'm not in feature expansion mode yet.

00:07:09   Sure.

00:07:10   And as for the design, I really thought I would get more pushback. And certainly, of course, I'm getting those emails saying, "I'm never using your app again. You ruined it. You have to bring back the old app."

00:07:23   Of course I get those. You get those with any kind of change. But the numbers of those have been substantially lower than I thought as a percentage of overall feedback.

00:07:32   So I think overall, I am pleasantly surprised with how well this has gone. There are still some bugs I'm trying to fix, the biggest of which is that for some reason continuous play is not working reliably in certain conditions, and I haven't yet figured out what those conditions are.

00:07:47   So that's what I'm working on later today, actually. Overall, this has actually gone surprisingly well. It's a little too early to judge things like server load, because again, most of the customer base doesn't have the new version yet.

00:08:05   But so far, that's looking okay. And I think overall, I am pleasantly surprised that it's going better than I expected. And I had kind of middling expectations. I didn't want to get my hopes up too high.

00:08:20   And I was a little afraid of certain risks. So far, the biggest risks have not played out. I didn't mess up the in-app purchase as far as I can tell. I didn't set my entire customer base on fire. I didn't break the servers.

00:08:32   Most people seem to like it. So I think I overall got out of this okay. And now I just have a mountain of bugs and tweak requests that I'm going to want to address and a mountain of feedback and things like that.

00:08:50   But it all seems very achievable. I expressed in the past, and I went into it a little bit more in ATP last week, I expressed in the past the feeling of just being overwhelmed and feeling like I could never get out.

00:09:05   When I was in the hard part of this rewrite about seven or eight months ago, it just felt like I was never going to get this done. It felt like I was not making progress. It was daunting. It was way too much ahead of me. I felt like I was never going to get there.

00:09:21   And now I got there. And yeah, there's still a lot to do, but I made it. I'm on the other side. The rewrite is out there. And now I'm able to actually really work on it more quickly and more easily.

00:09:37   And my time is more productive with this new foundation. So I'm finally on the other side of it. It feels fantastic.

00:09:44   Yeah, and it's awesome. I think it's important probably to take a moment before we get too into the where it happens from here to just have that moment of looking back and how impressive an accomplishment it is, what you've achieved here in the sense of you have essentially completely rebuilt the application

00:10:04   while without most of your users knowing or caring or having a problem about it from the ground up. They were going in the old app, and then you completely switcherood out this new app that's built with completely new underpinnings, a whole new database system, a whole new UI system,

00:10:24   all of these massive fundamental changes. And it's like, there's a little bump, and then now it's fine again. That is in and of itself a tremendous accomplishment that is no small thing.

00:10:37   And that there haven't been any kind of, you know, any time you completely overhaul and not just completely replace the fundamental data layer that undergirds the entire application, it's quite an impressive thing to do that and then have it not blow up or have some weird side issue or some person has some data set that causes some issue that you weren't aware of.

00:10:58   I did my big update to pedometer++ where I did a similar thing but mostly on the UI layer. I have not touched the data layer. I'm terrified of ever changing the data layer.

00:11:09   Only the deprecation of Objective-C I think would make me replace that data layer. So what you've done is a very impressive and awesome thing that I think is just good to linger on a moment like that is really cool.

00:11:22   I think it's easy to kind of take for granted that nothing terrible happened when nothing terrible happened. But I've done updates where things did not go well.

00:11:33   And it is not a given that any amount of preparation, any amount of testing will catch issues that could be potentially, you know, hopefully not broadly catastrophic, but catastrophic for particular users.

00:11:45   And so that is very cool and I think it's wonderful and I hope you took some time to celebrate reaching this point.

00:11:53   I know it's something. In some ways I'm kind of glad that we've been talking about this on Under the Radar for so long because we captured this journey from its very early stages.

00:12:04   And there have been episodes we've talked about of the despair you were just alluding to, of this like, "Oh my goodness, am I ever going to get out of this swamp?"

00:12:12   Like every step you take, you find two more steps you have to take and it's just, eventually that can be very daunting. But you've come through to the other side and the beautiful, brutal tragedy that's fabulous about software development is that you get to the other side and then there's just more work to do.

00:12:31   Right? You never get to the place where you're done. It's never the kind of work where, like I've always think it's kind of a great way to describe software. We describe when we actually take what we're working on and show it to the public as releasing it.

00:12:47   And I always have in my mind this vision of like, I don't know if you've ever done a farmer's caries when you're working out where you pick up something heavy, like a dumbbell, and you hold it and then you walk for a certain distance, essentially until you can't hold onto it anymore. And then you put it down. It's a way of working on your grip strength.

00:13:03   And it's like you get to this point, and in some ways with the software I feel like it's the same thing. It's like you just get to this point that you can't hold onto it anymore. You just have to get it out there.

00:13:10   And you let it go, and then the workout continues. You have this moment to take a deep breath, then you reach down, you pick it up again, and you just keep walking. And every now and then you release it and you have to put it down. But the work is never done. You never get to the end of this thing.

00:13:26   And that's part of what we love about what we do. It's kind of the tragedy of it, and it's the beauty of it as well.

00:13:32   I love this metaphor so much. That's exactly how it feels.

00:13:40   We actually take the deep breath, and we actually are like, "Yeah, we did it. That was really cool, what we just accomplished, that last section."

00:13:47   And sometimes the time between putting down the weights is short, and it's only a few steps. And sometimes it's things like this where you've walked around the block twice, and as you've gone it's getting harder and harder in some ways to keep going.

00:14:00   Because the longer those gaps, the more that's changed, the more the risk, the more the burden that you're carrying with you. But you did it, and I think that's really cool. We've come through that process. I'm very glad you did a slow rollout, because inevitably you've caught the fundamental truths that there is no amount of preparation that will find everything.

00:14:23   That's just the nature of the beast. And so you end up in a place that you'll find these bugs. But the nice thing about this is that the moment you did that first release is almost certainly the widest range of bugs that exist in this modern version of Overcast.

00:14:40   And so from here, you're just narrowing that down and narrowing that down and narrowing that down. And so finding a way to work through the bug reports and the issues and the feedback and all that, that is absolutely a useful thing.

00:14:54   But the nice thing is from here, you're narrowing that scope down. And I think fairly soon you'll have a sense of like, I always feel like there's this point in development where you get to this place and you're like, "Okay, this is good enough."

00:15:07   Not that there's no problems with it, but that it's good enough that there aren't any of these pressing issues that really have to be resolved. And hopefully, it'll be a week or two and then you'll be at that place. You have a couple of bug fixes, point releases out of Overcast.

00:15:22   You reach that point, and then it's like, "Okay, now what do I want to work on? Now what do I have to work on?"

00:15:28   I love the metaphor of just having to put it down after a while, because that is how it feels. The way I've always released software is basically that. I work on something for a while, and then at some point I'm like, "I just have to release this."

00:15:42   Either what I'm working on is so compelling I want to get it out there, or what I'm doing is so much better than what is out there that I want to get this out. I'll see somebody like my wife using the old version of the app that's currently in the App Store, and I'll be like, "Oh my God, that's so bad. What I'm working on is so much better. I have to get it out there."

00:16:04   I do have that kind of motivation. It is quite a feeling now to be on the other side of it, because I'm no longer rushing to hit a certain date. I hit the date I wanted to. It was the Overcast 10-year anniversary. I'm so happy I hit that date.

00:16:21   It meant so much to me. It was such a great story, both for my own mind and for the public. It really meant a lot to me to hit that date. I didn't think I could, and I did it. That's huge. Now I'm to the point where I don't have that massive time pressure.

00:16:35   Now the only pressure is trying to stop getting 1,200 emails so I can get back to other stuff. Now it's looking at what are the common issues people are complaining about or reporting, what are the common bugs I can fix. Even that, I'm working on that. That's getting better.

00:16:55   I will soon, probably within a few weeks, I will be at a point where the fires are all put out, even the smaller ones are mostly put out, and then I can start building. I can start taking advantage of this awesome foundation I now have to start doing real feature work.

00:17:13   That's something I haven't done in a long time, and I'm very much looking forward to that, being able to actually iterate, actually add more features, actually make cool stuff. I added this fairly little feature called Undo Seek that I mentioned before, where if you take a large seek in the app, you accidentally drag the bar or whatever, you can undo it within a short time.

00:17:38   People love that. That took me like half a day. People love it. And so I want to get to the point where I'm able to do more things like that, and I have so many ideas, but they were just not going to fit in this release time-wise.

00:17:54   This release was basically get what I already have, for the most part, rewritten and out the door. And I didn't even get everything. My Siri and Shortcut support is still totally broken. There are certain things that just didn't make it.

00:18:09   My storage management screen, OPML import/export, none of those made it. I'm going to have to add those in the coming weeks. But I have so many fun ideas for nice features people will enjoy that are in the vein of Undo Seek, that kind of quality of life or neat little improvement.

00:18:26   I have so many great ideas for that stuff, and I'm looking forward to finally, for the first time in years, being able to really do that.

00:18:33   We are brought to you this episode by Indeed. We're driven by the search for better, but when it comes to hiring, the best way to search for a candidate isn't to search at all. Don't search. Match with Indeed.

00:18:43   If you need to hire, you need Indeed. Indeed is your matching and hiring platform with over 350 million global monthly visitors, according to Indeed data, and a matching engine that helps you find quality candidates fast.

00:18:55   So ditch the busy work. Use Indeed for scheduling, screening, and messaging, so you can connect with candidates faster. And Indeed doesn't just help you hire faster. 93% of employers agree Indeed delivers the highest quality matches compared to other job sites, according to a recent Indeed survey.

00:19:10   Leveraging over 140 million qualifications and preferences every day, their matching engine is constantly learning from your preferences, so the more you use Indeed, the better it gets.

00:19:19   Join more than 3.5 million businesses worldwide that use Indeed to hire great talent fast. Listeners of this show will get a $75 sponsored job credit to get your jobs more visibility at Indeed.com/undertheradar.

00:19:32   Just go to Indeed.com/undertheradar right now to support our show and say you heard about Indeed on this podcast. Indeed.com/undertheradar.

00:19:44   Terms and conditions apply. Need to hire? You need Indeed. Our thanks to Indeed for their support of this show and Relay FM.

00:19:51   So I guess maybe something that's useful to talk through is just like from here, now that we've had this awesome accomplishment, you've reached this big milestone for Overcast, it's like, yeah, it's like how do you determine where to go from here?

00:20:04   And I think some of that you've really alluded to in terms of is responding to customer feedback, but I think certainly something that I've learned from my own experiences is that like customer feedback driven development is usually, it's very inefficient at the best.

00:20:22   And at its worst, it is pointing the app in the wrong direction because it's pointing it in like 20 directions all at once because every user is coming at the app with a different use case, a different mentality, a different sort of desire, a different reason why they use it.

00:20:36   And so if you're trying to sort of chase too many of those, you end up just not going anywhere.

00:20:41   And so it's a very challenging problem. I mean, I've been at the place where you are now where you have this, you know, whatever, you have 1200 emails. It's like reading through all of those is almost certainly not the most productive use of your time.

00:20:56   I mean, it's barely even a productive use of your time in some ways, at least in my experience, because, A, there's the sense of there's going to be a lot of duplication and a lot of repetition in that, that there'd be if there's a particular issue that everyone's running into, then they're going to run into the same issue over and over again.

00:21:14   And so you're going to have many reports of that. And especially even once you've fixed that issue, you know, and the issues that you fixed in the very first point release, you still likely are going to have emails from people who are opening that app version for the first time and having that issue because they got that version downloaded to their phone, but they haven't run the app yet.

00:21:33   And so then the next time they run it before it's done an update, they're going to see the issue that you've already fixed. And so it's just something that I would encourage you to be not necessarily view that 1200 emails or if you know, growing by the minute, as a particularly is something you need to get through as something that is worthwhile or important to get through that I think there is value to be gained there.

00:21:55   But it is very much almost like a statistical sampling approach, perhaps. I mean, I think another approach obviously is to have someone else tackle that for you. That is something that I found very helpful over the days, especially even just from a mental health perspective, even beyond the like technical aspects of that, that inevitably in there, there'll be people who say some very unkind things to you about the thing that you've spent the last year and a half of your life, working very difficult, you know, through very hard times to a certain extent.

00:22:24   And they're saying that it's not good and that a couple of those can really go the wrong way. But either way, if you give the other approach is to just take this statistical sampling approach, read 1% of those things expect that if you read a one random like 1% at random from those you will likely still capture all the issues or at least enough of the issues.

00:22:44   And then that if there are issues that persist, as you get fewer and fewer feedbacks, you can start to read more and more of them because it'll become less of a burden. But it is definitely something also where I think it's fun because now you're at the point where you can look at that list that you've almost certainly been keeping of features you want to implement, and you can start working on them.

00:23:05   And I think being focused on that and like that's where the actual impact comes from that there's one of you and you know, many, many more customers. And the biggest impact you can have is not responding to those customers or dealing with them individually, but doing things that will benefit them in aggregate and in bulk at once.

00:23:23   Yeah, exactly. And I'm trying to keep a healthy balance of that. I mean, I have found it worthwhile to skim the emails and the Mastodon replies and everything like that. I try to at least skim each one because sometimes there will be some detail in what somebody's reporting that actually helps me find the bug.

00:23:45   Like, you know, right now I mentioned the continuous play being broken for some people. I still have not figured out what that is because it works for me and I cannot replicate the problem, but it's pretty widespread.

00:23:55   So I can tell like, this is definitely a big problem. I'm trying to get logging out to these people as much as I can so I can, you know, I don't even have my logging infrastructure yet for the new version fully in place.

00:24:05   So like, it's kind of hard for me to debug certain things. So if somebody says something like, I was able to find one bug because somebody said some configuration detail, like they were using some preference in a way that I never use it, like a combination that I never use it, and that doesn't get widely tested because it's not very common.

00:24:21   And I'm like, wait a minute, in this case, they said this, I wonder if with these conditions, maybe this one thing can happen, and I looked at the code and found like a possible race condition. I was like, oh my god, that's like, that's it, like that's probably what it is.

00:24:35   And then I issued a build and that ended up fixing it. So sometimes finding a, sometimes reading those emails actually is extremely valuable.

00:24:44   But I think it is, you know, as you said, to a point, like when I'm getting the same seven or eight things reported over and over again, that is less useful to see each one.

00:24:56   It is more relevant to just recognize, oh, these eight things are what my users are having problems with. And that is the end of that information as far as I'm concerned.

00:25:06   Because when I'm hearing about it that much, I'm gonna have to fix all eight of them. Like that's, ultimately I'm going to have to address all of these problems.

00:25:14   So I don't even need to do things like, well, what percentage of users are being affected? Like, no, if I'm getting 1,200 emails and most of them are reporting the same eight issues, I probably need to address all eight of those issues.

00:25:25   And it's not super important to know, like, you know, all the details of exactly how and when to address those. No, if I'm getting that many emails in four days, you know, when only X percent of my users even have this release, I need to just do all of them.

00:25:41   But certainly as I go through the process and as I get closer to a stable point, I will have to be a little more disciplined about how I prioritize things and even what I decide to listen to at all.

00:25:52   A lot of times people will report a problem or a complaint that I recognize their complaint, but I think also, like, how should I solve this?

00:26:01   First of all, is this a need that I even need to solve? Should this person even necessarily be expecting my app to solve this problem?

00:26:07   And then secondly, what's the best way to address their need, even if it is not what they said? You know, the whole faster horse thing.

00:26:15   Like, if they're complaining that, like, for instance, if the streaming removal impacts somebody a lot because they keep thousands of podcast episodes on plate and they want to pick and choose what they're listening to, there's probably a better way to deal with that than bringing back the entire streaming infrastructure and all of the problems and risks and complexities that it brings.

00:26:34   Maybe there's smarter ways to help prioritize downloads or to automatically download or automatically delete things or, you know, things like that. There's so many other options there. So that's the kind of work I'm hoping to get back into over the coming months.

00:26:46   And of course, I mean, eventually, I guess I'll probably have to rewrite the Watch app. But that's down the road some more.

00:26:52   Honestly, the biggest reason I want to rewrite the Watch app is not that it so desperately needs it, although it does, because honestly, no one's even complaining about it. No one's even asking for this.

00:27:01   The biggest reason I want to rewrite the Watch app is that then I can delete all the old code out of my project. Because all of the old model code and sync code and everything is all still in the project because the Watch app uses it all.

00:27:12   And so that's the main reason. This is not a good reason to do something that will probably take me, like, a month or more. This is not a good reason to do it.

00:27:19   But that's what I have the whole next X months or years to look forward to, is that kind of debate. But it's more like I'm just so happy to be back doing active work on the app again that people can actually see and feel and use and enjoy.

00:27:36   I haven't been for a long time, and now I feel like I'm back.

00:27:40   Yeah, and I think even there, there is something like having the ability to feel a sense of improvement in the app is going to be a tangible thing that I think your users experience.

00:27:53   And in some ways, I remember, I think I was talking about this, where sometimes I will intentionally hold back a feature that I think is a good feature from a release so that it can go into point one, so that it gives a sense of inertia and a sense of, like, that the app is continuing to get better in a more tangible, concrete way.

00:28:12   Rather than waiting until both features were finished, releasing one and then releasing the other one somewhat shortly thereafter is, I think, potentially a nice feel that your users can get.

00:28:22   And I think it's kind of nice that you now have the ability to do that kind of thing, that you could imagine a world where between now and the end of this year, there are meaningful feature releases and updates and things that are happening to the app, because you feel empowered and technically capable of doing that.

00:28:38   Like, when iOS 18 comes out in, whatever, a month and a half from now or something, like, you will have something there most likely ready to go and being available that fits in with the modern stuff and it's not this weird hacky thing that you have to do, because you're very swift and very async.

00:28:54   So anything that relates to that, you're ready for. And I think that inertia is going to feel really good, both from your sense as a developer and I imagine as on the customer side as well.

00:29:04   That's literally what I've been waiting years to accomplish. Like, every single WVDC, every single new iOS release, I've been thinking, "Oh, how the heck am I going to wedge this in?"

00:29:13   And now I'm just in such a better place to do that. This is what I've been working towards. And it feels just amazing to finally be here and to finally have this giant, daunting task mostly behind me. It's a great feeling.

00:29:29   Yeah. Congratulations. And I just couldn't be happier that we were able to close that chapter and open a new one about where Overcast is going from here.

00:29:38   Thank you so much. And thanks everybody for listening. And we'll talk to you in two weeks.

00:29:44   Bye.

00:29:45   [ Silence ]