Under the Radar

00:00:00 ◼   ► Welcome to Under the Radar, a show about independent iOS app development.

00:00:04 ◼   ► I'm Marco Arment.

00:00:05 ◼   ► And I'm David Smith.

00:00:06 ◼   ► Under the Radar is never longer than 30 minutes, so let's get started.

00:00:10 ◼   ► So today we wanted to dive a little bit into the weeds,

00:00:14 ◼   ► go all the way down the stack, you know, as you kind of, you know,

00:00:18 ◼   ► if you were drawing one of those silly, what is it, OPML?

00:00:21 ◼   ► No, there's a fancy name for those kind of charts that you learn about in school,

00:00:26 ◼   ► but they never actually draw ever again.

00:00:28 ◼   ► How do you not know this?

00:00:30 ◼   ► I mean, I don't know what you're talking about, but I figured you are the master of charts.

00:00:33 ◼   ► You should know this.

00:00:34 ◼   ► Well, it's not a chart. It's like drawing. I'm terrible at drawing.

00:00:38 ◼   ► You're like a flow chart?

00:00:39 ◼   ► Well, I'm thinking of the one, you know, you go all the way down,

00:00:42 ◼   ► there's like, the database was always that cylinder that you would draw.

00:00:47 ◼   ► I don't know why it was a cylinder.

00:00:48 ◼   ► Like, is this back from the days when you had tape disks or something?

00:00:51 ◼   ► It was probably to represent like a hard disk.

00:00:53 ◼   ► Maybe. So, you have, when you draw your infrastructure diagram,

00:00:59 ◼   ► you always end up in the back with a little cylinder, and that's your database.

00:01:03 ◼   ► And it's, you know, an important part of applications in certain ways,

00:01:10 ◼   ► but I think the role that persistence has played, and even as you just noticed there,

00:01:15 ◼   ► I switched from using database to persistence,

00:01:17 ◼   ► is the way in which we think about storing data to then get back later

00:01:21 ◼   ► has certainly evolved over the last, you know, the last eight years of iOS

00:01:26 ◼   ► and of the App Store.

00:01:28 ◼   ► And I think it seems something that's worth getting into,

00:01:30 ◼   ► because there's such a variety of approaches that you can take for this,

00:01:34 ◼   ► where you go all the way on the one extreme to things like raw SQLite.

00:01:39 ◼   ► You can then take wrappers around SQLite.

00:01:43 ◼   ► You can have things that are like core data using SQLite,

00:01:47 ◼   ► which are Realm or all kinds of other kind of middlemen that are wrapper,

00:01:53 ◼   ► like very sophisticated wrappers.

00:01:55 ◼   ► You can start to get into systems where you have your own persistence system

00:02:01 ◼   ► that is writing files to disk.

00:02:03 ◼   ► You can start to get into just using NSUserDefaults and Keychain

00:02:08 ◼   ► and very lightweight things,

00:02:10 ◼   ► and then you can go all the way to the other extreme of having no persistence at all,

00:02:14 ◼   ► and I know of a variety of apps that are entirely unpersisted,

00:02:18 ◼   ► that other than like a user token, the app is entirely unpersisted.

00:02:23 ◼   ► And that flexibility is, I think, in many ways a direct result

00:02:29 ◼   ► of the increasing capability and connectivity of the devices

00:02:33 ◼   ► that we're writing apps for,

00:02:35 ◼   ► where I think if you imagine an app as this thing all on its own,

00:02:41 ◼   ► it's going to need a fairly sophisticated persistence system

00:02:44 ◼   ► because that is where the apps--

00:02:47 ◼   ► it needs to be able to function on its own,

00:02:49 ◼   ► and it needs to be able to do it in a way that,

00:02:51 ◼   ► if we go back in the day when you only had a 3G connection

00:02:54 ◼   ► connected to your iPhone,

00:02:56 ◼   ► you really couldn't expect the user to be able to download information very often,

00:02:59 ◼   ► so it has to be fairly self-contained,

00:03:01 ◼   ► whereas you can imagine now there are certain kind of applications

00:03:03 ◼   ► that are essentially just displaying a JSON blob to the user,

00:03:10 ◼   ► and maybe the data is in some ways also time sensitive.

00:03:15 ◼   ► Showing the data that you had cached a week ago would be kind of irrelevant.

00:03:20 ◼   ► So say, for example, like a weather app.

00:03:22 ◼   ► It's an example of an app I've written that has essentially no persistence

00:03:25 ◼   ► because the app's data is completely useless.

00:03:30 ◼   ► But I think it's an interesting thing to think about.

00:03:32 ◼   ► Where we--

00:03:34 ◼   ► Hi, I'm David Smith. My app's data is completely useless.

00:03:37 ◼   ► Hey, you know, sometimes-- I'm saying it's useless as soon as it's old.

00:03:43 ◼   ► Yeah, it's immediately out of date, basically.

00:03:45 ◼   ► It's immediately out of date. It's necessary.

00:03:47 ◼   ► As soon as I have my blob of weather data,

00:03:51 ◼   ► it's already starting to get out of date and fall out.

00:03:55 ◼   ► But anyway, getting back on my train of thought,

00:03:58 ◼   ► the other thing that has made persistence I think so interesting now

00:04:01 ◼   ► is the rise of extensions,

00:04:04 ◼   ► and this is something that I've been struggling with a lot this summer

00:04:07 ◼   ► of trying to keep all the various parts of our applications in sync.

00:04:12 ◼   ► So, Podometer++, for example, has an iPhone app,

00:04:16 ◼   ► it has a widget extension,

00:04:18 ◼   ► it has a watch extension, which actually is two parts.

00:04:22 ◼   ► One of them is just the UI, and the other part's the actual watch extension.

00:04:26 ◼   ► It has a SiriKit extension, and it has an iMessage extension.

00:04:30 ◼   ► And so I have five different sort of things that I have to keep data in sync now

00:04:35 ◼   ► that isn't syncing to a server, that is just syncing locally.

00:04:39 ◼   ► And so persistence is a really interesting and tricky problem now,

00:04:42 ◼   ► and I think what I find myself typically doing now

00:04:45 ◼   ► is I'm trying to simplify as much as I can my persistence

00:04:51 ◼   ► to a point that essentially most of my persistence now is just dictionaries

00:04:56 ◼   ► and sort of having these blobs of data that I can very easily move around

00:05:01 ◼   ► in a very lightweight way because I need to get data

00:05:04 ◼   ► so that when the watch app updates its step count,

00:05:08 ◼   ► that watch, that step count needs to, via watch connectivity,

00:05:11 ◼   ► find its way down into my phone,

00:05:14 ◼   ► which needs to find itself into the iPhone app,

00:05:16 ◼   ► it needs to find its way into the iMessage app,

00:05:18 ◼   ► it needs to find itself into the widget so that everything can stay in sync.

00:05:21 ◼   ► And so having some big heavyweight persistence scheme

00:05:24 ◼   ► that is doing some kind of really heavy database-oriented transactional things

00:05:30 ◼   ► or things just doesn't work, it's too heavyweight.

00:05:33 ◼   ► And so I'm starting to more and more just shift myself away from that

00:05:36 ◼   ► into dictionaries and that kind of stores.

00:05:40 ◼   ► And so far I've found that to be pretty effective.

00:05:43 ◼   ► Have you had to run into any of these kind of problems with Overcast

00:05:47 ◼   ► in terms of keeping various parts of the app in sync?

00:05:49 ◼   ► Oh, absolutely.

00:05:50 ◼   ► And in fact I've kind of gone the opposite direction as you over the years.

00:05:53 ◼   ► Well, sort of.

00:05:54 ◼   ► So with Instapaper early on, I was just using raw SQLite API in C

00:06:01 ◼   ► because I thought the speed and performance would matter.

00:06:04 ◼   ► And maybe at the time it did.

00:06:06 ◼   ► That was back on the iPhone 1, so it might have mattered back then.

00:06:09 ◼   ► I've slowly moved first into using FMDB

00:06:14 ◼   ► and then later on building on top of FMDB my own FC model system,

00:06:19 ◼   ► which is kind of a lighter weight than core data persistence layer

00:06:24 ◼   ► built on top of SQLite.

00:06:26 ◼   ► And for different apps that were not like my big app,

00:06:31 ◼   ► things like the magazine and Bugshot,

00:06:34 ◼   ► I've used different things there, either no persistence

00:06:37 ◼   ► or in the case of the magazine I first started out with just using

00:06:40 ◼   ► a bunch of plists that I would just write to disk.

00:06:43 ◼   ► And having dictionaries in plists is enough for a lot of apps.

00:06:49 ◼   ► And as you mentioned, kind of going towards that kind of area

00:06:53 ◼   ► with a lot of your apps, there's a lot to be said for that.

00:06:56 ◼   ► Just having very simple dictionaries in files,

00:07:00 ◼   ► there's a lot to be said to that.

00:07:02 ◼   ► The setup cost, the amount of effort it takes to start working on that

00:07:07 ◼   ► is extremely low.

00:07:09 ◼   ► The amount of complexity in the app is extremely low.

00:07:12 ◼   ► The bug potential in most cases, but not all, is extremely low.

00:07:17 ◼   ► And it's also just really fast. There's a lot to be said for that.

00:07:21 ◼   ► However, when you go to the more sophisticated database-backed systems,

00:07:26 ◼   ► things like core data, then you get a lot, you view it as heavy

00:07:33 ◼   ► or as maybe over-designed for your needs,

00:07:36 ◼   ► depending on what your needs are for the particular app.

00:07:38 ◼   ► But database systems give you a lot for free

00:07:42 ◼   ► that if you are using a file-backed system,

00:07:45 ◼   ► or a more manual or simple system,

00:07:47 ◼   ► that you have to either go without and hope it doesn't bite you

00:07:51 ◼   ► or you have to write it yourself, which is probably not a good idea.

00:07:55 ◼   ► So things like concurrent access to the same file not corrupting the data.

00:08:01 ◼   ► Things like, in your extension contexts,

00:08:05 ◼   ► having the data be able to be updated by one process

00:08:07 ◼   ► and then notifying the others that might be accessing it at the same time,

00:08:11 ◼   ► notifying them of the changes.

00:08:13 ◼   ► There are things like this that the good persistence layers will do already for you.

00:08:19 ◼   ► You don't have to write it.

00:08:21 ◼   ► And if you use them even for simple tasks,

00:08:23 ◼   ► it might be worth getting those features.

00:08:25 ◼   ► Now, they do bring complexity, for sure.

00:08:28 ◼   ► Core data, I'm not a huge expert in core data

00:08:31 ◼   ► because I only used it for basically the second half of the magazine

00:08:34 ◼   ► after I stopped using my P-list system.

00:08:36 ◼   ► So I used it very little and not recently.

00:08:39 ◼   ► So all this with a grain of salt.

00:08:41 ◼   ► But there are obviously complexities.

00:08:43 ◼   ► There's things like concurrent access from different threads

00:08:45 ◼   ► that you have to really be careful with.

00:08:47 ◼   ► Although, again, same thing.

00:08:49 ◼   ► You have to be careful when you're writing files, too.

00:08:51 ◼   ► There's things like dealing with schema migrations

00:08:55 ◼   ► that a lot of these layers do not make easy.

00:08:58 ◼   ► Possible crashes, like if the migration fails on an upgrade or whatever else,

00:09:03 ◼   ► or the database gets corrupt.

00:09:05 ◼   ► A lot of apps can't or don't handle that very well.

00:09:08 ◼   ► So there's lots of complexity if you take on some of the bigger systems

00:09:13 ◼   ► that you have to deal with.

00:09:14 ◼   ► But I would posit that in most cases for most apps,

00:09:18 ◼   ► if you're doing things like extensions or concurrent threads in one app,

00:09:24 ◼   ► you probably want those big systems

00:09:26 ◼   ► because they will take care of problems

00:09:29 ◼   ► that you would have to worry about if you were doing it yourself

00:09:32 ◼   ► with plists and stuff in your own app.

00:09:35 ◼   ► Does that make sense?

00:09:36 ◼   ► Yeah, it is such an interesting tension

00:09:39 ◼   ► that I feel like you find with these types of questions.

00:09:42 ◼   ► The thing that I hate,

00:09:44 ◼   ► one of the things I really don't like when I'm developing an app

00:09:47 ◼   ► is whenever I make a decision that is setting me on a path

00:09:51 ◼   ► where it will be difficult to change down the road.

00:09:54 ◼   ► So if I go file a new project in Xcode,

00:09:59 ◼   ► and it says, "Would you like to check the box to use Core Data?"

00:10:03 ◼   ► Now obviously that checkbox itself is not the--

00:10:07 ◼   ► I can obviously take it out or put it back in later.

00:10:09 ◼   ► But conceptually, I agonize over whether I'm going to push that button

00:10:14 ◼   ► because it feels like I'm setting myself--

00:10:17 ◼   ► I'm making a choice that has a lot of implications down the road.

00:10:20 ◼   ► And it's like I'm choosing which adventure I'm going to be going on.

00:10:26 ◼   ► Am I going to be going on the Core Data adventure

00:10:29 ◼   ► of finding weird threading issues

00:10:31 ◼   ► and dealing with merging in the background?

00:10:34 ◼   ► Whenever I've done Core Data, and I've done a lot of Core Data,

00:10:36 ◼   ► that seems to be the adventure you're on,

00:10:38 ◼   ► where the bugs you're going to have are related to that.

00:10:41 ◼   ► Or am I going to go down the road of a letter-weight file-based solution

00:10:46 ◼   ► where my bugs and issues are going to be cache invalidation or freshness,

00:10:51 ◼   ► those types of issues.

00:10:52 ◼   ► I'm just choosing which one I'm doing.

00:10:55 ◼   ► And really awkward is if your app involves any amount of user data,

00:10:59 ◼   ► as soon as you have that first version out in the world,

00:11:03 ◼   ► suddenly you are going to be responsible for maintaining that--

00:11:07 ◼   ► in some ways indefinitely, not truly indefinitely.

00:11:10 ◼   ► But you then have to start dealing with,

00:11:13 ◼   ► "Well, if I ever change my mind and want to do something different,

00:11:16 ◼   ► I'm going to have to build some kind of big migration.

00:11:19 ◼   ► I'm going to have to test it like crazy.

00:11:21 ◼   ► And I'm going to have to work really hard

00:11:23 ◼   ► to make sure that I don't break any user's data."

00:11:26 ◼   ► Because that's the nature of persistence.

00:11:30 ◼   ► You're trying to take something, put it on your phone,

00:11:32 ◼   ► and have it last indefinitely.

00:11:35 ◼   ► And so I really hate these kind of decisions

00:11:38 ◼   ► that we have to make in development, where it's not just like,

00:11:40 ◼   ► "Oh, Core Data isn't really working out for me right here.

00:11:43 ◼   ► I think I want to go to SQLite. Let me just do that."

00:11:46 ◼   ► It's like, well, there goes three months of my life,

00:11:48 ◼   ► transitioning from pulling the data out

00:11:51 ◼   ► and switching it into something else.

00:11:53 ◼   ► And that kind of a heavyweight decision, I think, is so awkward.

00:11:56 ◼   ► And also, it's probably also a fair thing to say,

00:11:58 ◼   ► that I think it is good to have the awareness that

00:12:01 ◼   ► because these decisions tend to have such high switching costs,

00:12:06 ◼   ► the discussion about which one you should use.

00:12:10 ◼   ► So if you are talking online,

00:12:12 ◼   ► or you're reading documentation or blog posts about these things,

00:12:15 ◼   ► people often become very passionate about them.

00:12:19 ◼   ► It's the old ViEmacs kind of problem,

00:12:22 ◼   ► where you become very passionate about your thing

00:12:25 ◼   ► and very defensive of it,

00:12:27 ◼   ► because the thought of having--

00:12:29 ◼   ► If someone else is actually correct,

00:12:31 ◼   ► in terms of, oh, some new persistent scheme comes out

00:12:34 ◼   ► that is objectively better than what you have

00:12:37 ◼   ► and what you've been using,

00:12:39 ◼   ► you have a problem, and you have a serious problem,

00:12:41 ◼   ► and you have this weird issue that you then feel bad about.

00:12:44 ◼   ► And so discussions with this are always kind of tinged in that way.

00:12:48 ◼   ► And so it's just a good disclaimer to kind of talk about,

00:12:50 ◼   ► because it's just such a high switching cost.

00:12:53 ◼   ► You can't just be like, "Yeah, you know, maybe I will use that.

00:12:56 ◼   ► Maybe I will drop this and switch to that."

00:12:58 ◼   ► I've done migrations like that,

00:13:00 ◼   ► where I've gone from one kind of data system to another.

00:13:03 ◼   ► And it is usually-- That is the worst part of my job ever,

00:13:07 ◼   ► and I try as best I can.

00:13:09 ◼   ► Once I make the choice, it's like, I'm just going to go with it,

00:13:12 ◼   ► and I'm just going to do the best I can.

00:13:14 ◼   ► I have a recipe book manager,

00:13:16 ◼   ► which is the most terrifying of my persistence needs,

00:13:19 ◼   ► because people are literally putting their family recipes,

00:13:23 ◼   ► things that are very emotionally important to them, into my app,

00:13:29 ◼   ► and they expect them to be there.

00:13:31 ◼   ► And I've had issues where I do a bad data migration in Core Data,

00:13:35 ◼   ► which is what I use for that app.

00:13:37 ◼   ► I do a bad data migration, and it corrupts some recipes.

00:13:43 ◼   ► That is terrible.

00:13:46 ◼   ► And so you have to be really thoughtful about these kinds of things.

00:13:50 ◼   ► And I think in some ways, as I go forward,

00:13:53 ◼   ► I'm getting less and less, wanting more and more lightweight things.

00:13:57 ◼   ► I'm almost in some ways trying to make fewer and fewer promises

00:14:00 ◼   ► and trying to have fewer situations

00:14:05 ◼   ► where I am really responsible inside of my app code

00:14:09 ◼   ► for taking care of something that's important to somebody,

00:14:11 ◼   ► that as soon as I can, like in my recipe book now,

00:14:15 ◼   ► I added a free sync system to it that takes all the--

00:14:19 ◼   ► essentially, I call it sync, most people just use it with one app.

00:14:23 ◼   ► It's just backup.

00:14:25 ◼   ► So I have a live backup of all their user data out into a database

00:14:29 ◼   ► that I can keep for them,

00:14:31 ◼   ► so that now when weird things happen to users' devices

00:14:34 ◼   ► and the persistence scheme goes crazy and Core Data gets--

00:14:37 ◼   ► it's not uncommon for me to get a request from somebody

00:14:40 ◼   ► that when I ask them for the crash log and it comes back

00:14:43 ◼   ► and I get the thing and it's just like,

00:14:45 ◼   ► it's a Core Data error that just says the SQLite file has become--

00:14:48 ◼   ► is malformed and cannot be opened.

00:14:51 ◼   ► And that happens all the time on iOS devices.

00:14:53 ◼   ► Like it happens-- one of the biggest culprits on iOS devices

00:14:57 ◼   ► is if the disk runs out of space,

00:14:59 ◼   ► then weird things can happen with your persistence layer,

00:15:01 ◼   ► depending on how it's done and what its settings are,

00:15:04 ◼   ► and that very frequently can corrupt SQLite databases.

00:15:08 ◼   ► It shouldn't, but in practice it does.

00:15:10 ◼   ► Or things like, you know, sometimes like,

00:15:13 ◼   ► when writing to the app container,

00:15:15 ◼   ► the share container between extensions,

00:15:17 ◼   ► sometimes those writes will just fail for some reason.

00:15:20 ◼   ► Or they'll be denied in some weird permissions error or something.

00:15:24 ◼   ► And it's like, well, then like,

00:15:26 ◼   ► if a write fails to your persistence layer,

00:15:28 ◼   ► then like that can also potentially corrupt the data that's there or something.

00:15:31 ◼   ► And it's like, this stuff happens all the time,

00:15:33 ◼   ► and it's so nice if you have the luxury

00:15:35 ◼   ► to be able to have in your app code, as I do in Overcast,

00:15:37 ◼   ► if the database file is corrupt,

00:15:39 ◼   ► just delete it and resync everything from the server.

00:15:41 ◼   ► Yeah, exactly.

00:15:43 ◼   ► And that's the place where persistence

00:15:45 ◼   ► and the really industrial-grade persistence stuff comes into play.

00:15:50 ◼   ► I love that-- I've been using-- I'm a Postgres guy.

00:15:54 ◼   ► I mean, you know you're a MySQL person,

00:15:56 ◼   ► but like, functionally it doesn't really matter.

00:15:58 ◼   ► Well, it doesn't matter. This is the thing. I don't care.

00:16:00 ◼   ► They're both great. I've just been using Postgres for a long time,

00:16:03 ◼   ► and so I know how to do it.

00:16:05 ◼   ► But the thing is, Postgres, like,

00:16:07 ◼   ► I've done some horrible things to Postgres databases,

00:16:10 ◼   ► and I've never lost data as a result.

00:16:13 ◼   ► You know, I've had weird situations where, you know,

00:16:15 ◼   ► I accidentally powered off the wrong server in the middle of something,

00:16:18 ◼   ► and like, it was in the middle of doing a migration,

00:16:21 ◼   ► and it got canceled halfway.

00:16:24 ◼   ► Like, those types of systems are designed for this kind of recovery

00:16:28 ◼   ► in a way that SQLite, which is, at the end,

00:16:32 ◼   ► what most of the persistence schemes on device use, aren't.

00:16:36 ◼   ► And so it's definitely something that I think now I'm leaning so much towards the,

00:16:40 ◼   ► "Let's get this all onto a nice, safe server somewhere

00:16:43 ◼   ► with some really proper, you know, backup,

00:16:46 ◼   ► whereas I don't have to worry about, like,

00:16:48 ◼   ► it's not the users doing backups and restores.

00:16:50 ◼   ► It's like, I'm doing backup and restore. I'm versioning the database.

00:16:53 ◼   ► I'm making sure that everything is nice and safe

00:16:55 ◼   ► in a way that, on device, like you said,

00:16:58 ◼   ► if something goes wrong, it's like, just say,

00:17:00 ◼   ► "Oh, I'm sorry, I need you to connect to the network

00:17:02 ◼   ► so I can go grab the most recent version of this."

00:17:05 ◼   ► And that, like, coding with that in mind,

00:17:10 ◼   ► I think, makes so many of these problems just go,

00:17:12 ◼   ► either go away or be so much less stressful.

00:17:14 ◼   ► - Yeah, because, and, you know, the reality is,

00:17:16 ◼   ► like, your customers will have, you know,

00:17:19 ◼   ► what if their phone, well, before the iPhone 7,

00:17:22 ◼   ► what if it falls in a toilet or something?

00:17:24 ◼   ► You know, like, they lose everything on their phone,

00:17:26 ◼   ► or their phone gets stolen, or whatever, their phone fails,

00:17:29 ◼   ► and they have to get a new one,

00:17:30 ◼   ► and they have to restore from backup.

00:17:31 ◼   ► Well, do they have a backup? How old is that backup?

00:17:33 ◼   ► And, like, the answer to those questions might be bad

00:17:36 ◼   ► for the data, and so if you just always have everything

00:17:40 ◼   ► synced to a server somewhere that you can always restore

00:17:43 ◼   ► back to the phone as soon as the user logs in,

00:17:45 ◼   ► and whether it's iCloud or your own stuff,

00:17:47 ◼   ► for this purpose, it doesn't really matter,

00:17:48 ◼   ► that is such a better place to be than trying to rely

00:17:51 ◼   ► on storing user data indefinitely,

00:17:53 ◼   ► just locally in the app, as you said.

00:17:55 ◼   ► - Oh, sure, 'cause I mean, even for,

00:17:57 ◼   ► and there's a couple of situations that I run into

00:17:59 ◼   ► that are a bit trickier for me,

00:18:00 ◼   ► because some of my apps use store health data,

00:18:04 ◼   ► and there's some very specific and complicated rules

00:18:07 ◼   ► around storing health-related data.

00:18:10 ◼   ► I mean, it's not like health in the sense of diagnosing diseases,

00:18:13 ◼   ► but, you know, steps, sleep data, activity data,

00:18:16 ◼   ► like, these are things that fall broadly

00:18:18 ◼   ► into the category of health data,

00:18:20 ◼   ► and so I currently, I don't store any of those

00:18:23 ◼   ► anywhere off-device, because if I had a server,

00:18:26 ◼   ► that had stored health data, suddenly there's all kinds

00:18:29 ◼   ► of regulatory things that I don't really know about,

00:18:32 ◼   ► that I'm sure exist, and I really don't want to know about,

00:18:35 ◼   ► that would start to come into play,

00:18:37 ◼   ► and there's rules about not using iCloud, necessarily,

00:18:40 ◼   ► but as a result, I do a lot of my backup and restore

00:18:43 ◼   ► using the either iTunes or iCloud backup system,

00:18:47 ◼   ► and it sometimes doesn't work.

00:18:49 ◼   ► Like, there's just no two ways around it,

00:18:51 ◼   ► where sometimes someone doesn't update.

00:18:53 ◼   ► You know, I've had a couple of these

00:18:54 ◼   ► this last couple weeks, where someone updates to iOS 10,

00:18:58 ◼   ► something goes weird in the update,

00:19:00 ◼   ► they restore from the backup, and the app state is just missing.

00:19:04 ◼   ► And there's nothing I can do.

00:19:06 ◼   ► - That's horrible.

00:19:08 ◼   ► - There's nothing that I can do for that person,

00:19:11 ◼   ► and, you know, and so, but it's a situation

00:19:14 ◼   ► where I wish I did have a means to back that up externally,

00:19:18 ◼   ► because there is nothing better when you have,

00:19:21 ◼   ► like, this has happened dozens of times with my recipe app,

00:19:24 ◼   ► where something happens to someone's device,

00:19:27 ◼   ► and they send an email that is just the, like, the panicked,

00:19:30 ◼   ► like, I have all of my family recipes in this app,

00:19:33 ◼   ► I've been using it, I love it, you know,

00:19:35 ◼   ► my kid threw the iPad out the window and it shattered,

00:19:39 ◼   ► and I didn't have a backup of the device.

00:19:42 ◼   ► Like, is there anything that can be done?

00:19:44 ◼   ► And there is the, it is just absolute,

00:19:46 ◼   ► it is an absolute joy to be able to go into this,

00:19:49 ◼   ► like, go into the admin panel, verify that, you know,

00:19:51 ◼   ► when I last, they last synced their system,

00:19:54 ◼   ► I'd be able to say to them, it's like, you know what,

00:19:56 ◼   ► I have 267 recipes, safe and sound,

00:19:59 ◼   ► whenever you get your new device,

00:20:00 ◼   ► log in with your credentials, and they'll all be back to you.

00:20:03 ◼   ► Like, that is amazing, that is, like,

00:20:05 ◼   ► you just made someone's day, and for what was otherwise,

00:20:08 ◼   ► it was gonna be a really, really bad day.

00:20:10 ◼   ► - I love that we chose to do this episode

00:20:13 ◼   ► on, like, the local persistence schemes that you use

00:20:16 ◼   ► in your app, and meanwhile, our conclusion is,

00:20:18 ◼   ► don't rely on them for anything. (laughs)

00:20:21 ◼   ► - That's true, but I think that is in some ways the point.

00:20:23 ◼   ► This is where I've been going to more and more.

00:20:25 ◼   ► It's like, there's, you can so easily get focused

00:20:28 ◼   ► on what is the best local scheme for doing this,

00:20:31 ◼   ► and the more and more of this I do,

00:20:34 ◼   ► the less and less I think that storing data on device

00:20:37 ◼   ► is a good idea for most apps, that I think most apps

00:20:40 ◼   ► benefit from instead of having their data locally,

00:20:43 ◼   ► pushing it somewhere else, where they, like you said,

00:20:45 ◼   ► whether that's iCloud, whether that's your own servers,

00:20:47 ◼   ► whatever, the more that you can push your data somewhere else

00:20:51 ◼   ► and then be pulling it down as needed,

00:20:53 ◼   ► the better you're gonna be, because there's just

00:20:55 ◼   ► so many issues that you're gonna run into otherwise.

00:20:59 ◼   ► - And if you do wanna run your own servers,

00:21:01 ◼   ► our sponsor this week is perfectly timed, it's Linode.

00:21:05 ◼   ► So Linode, Linode is a wonderful web host

00:21:08 ◼   ► with high performance Linux SSD servers,

00:21:10 ◼   ► spread across eight data centers around the world.

00:21:12 ◼   ► They're a fantastic solution for your server infrastructure.

00:21:15 ◼   ► Linode offers servers that you can get up and running

00:21:17 ◼   ► in just under a minute, with plans starting at just $10

00:21:20 ◼   ► a month, and that now gets you two gigs of RAM

00:21:22 ◼   ► for 10 bucks a month, so that could be, I mean,

00:21:24 ◼   ► look, these servers are powerful, these are really fast,

00:21:26 ◼   ► we use them, I have, I think 12 Linode instances

00:21:29 ◼   ► that I run Overcast on, plus one for market.org,

00:21:31 ◼   ► plus one for some accessory stuff, I have lots of Linodes.

00:21:34 ◼   ► I bet most apps could get away with the $10 a month plan

00:21:38 ◼   ► for all of their user data.

00:21:40 ◼   ► It's really, you get a lot for the money here.

00:21:42 ◼   ► Anyway, Linode is great for other tasks.

00:21:44 ◼   ► Also, things like if you wanna run your own private

00:21:46 ◼   ► Git server, if you wanna host large databases,

00:21:49 ◼   ► like 267 recipes, if you wanna run a mail server,

00:21:52 ◼   ► if you wanna operate powerful apps, and so much more.

00:21:55 ◼   ► With industry-leading native SSD storage

00:21:58 ◼   ► and access to a 40 gigabit network,

00:22:00 ◼   ► you will have all the power you need

00:22:01 ◼   ► to get your tasks done at Linode.

00:22:03 ◼   ► Now, if you're a listener of this show,

00:22:05 ◼   ► sign up at Linode.com/Radar.

00:22:08 ◼   ► You'll not only be supporting us,

00:22:09 ◼   ► but you'll also get $20 towards any Linode plan.

00:22:12 ◼   ► You can also use code radar20 at checkout

00:22:15 ◼   ► to get that same deal.

00:22:16 ◼   ► So, there's a seven-day money-back guarantee.

00:22:17 ◼   ► There's nothing to lose.

00:22:18 ◼   ► Go to Linode.com, spelled like Linux, but Linode.

00:22:21 ◼   ► Linode.com/Radar to learn more,

00:22:24 ◼   ► sign up and take advantage of that $20 credit

00:22:26 ◼   ► or use promo code radar20 at checkout.

00:22:29 ◼   ► Thank you so much to Linode for supporting this show.

00:22:32 ◼   ► So, I think it's worth also mentioning,

00:22:35 ◼   ► you mentioned at the end of the last segment

00:22:37 ◼   ► about how it's actually becoming more and more useful

00:22:39 ◼   ► to not store anything on the device

00:22:41 ◼   ► and just always fetch it from the server.

00:22:43 ◼   ► Back when I first started developing,

00:22:45 ◼   ► which was Instapaper back then,

00:22:47 ◼   ► offline access was a big deal, especially for that app.

00:22:50 ◼   ► That was kind of the point of it,

00:22:51 ◼   ► and that was a major feature of it.

00:22:54 ◼   ► And I think over time, we've slowly had times

00:22:58 ◼   ► in which we are offline have slowly eroded

00:23:01 ◼   ► out of people's lives, and they're still there.

00:23:03 ◼   ► There are still occasions where people are offline,

00:23:06 ◼   ► but that is getting really few and far between.

00:23:09 ◼   ► I mean, now, you can now even use devices on planes

00:23:13 ◼   ► during takeoff and landing.

00:23:15 ◼   ► You can be online for the entire middle part

00:23:17 ◼   ► of the flight if you want to,

00:23:19 ◼   ► and that's only gonna get more and more common over time.

00:23:21 ◼   ► A lot of subways, like underground transit systems,

00:23:25 ◼   ► where there was no reception 10 years ago,

00:23:29 ◼   ► they're starting to get cell reception, cell coverage

00:23:31 ◼   ► in a lot of them now.

00:23:32 ◼   ► There's more and more areas where people

00:23:34 ◼   ► have more and more connectivity.

00:23:36 ◼   ► Offline access is one of the only reasons

00:23:40 ◼   ► why you would store data locally

00:23:42 ◼   ► for a lot of server-backed apps,

00:23:44 ◼   ► and I think it's very reasonable for a lot of apps today

00:23:48 ◼   ► to just be like, "Sorry, we don't work offline,"

00:23:50 ◼   ► because the actual demand for getting the app

00:23:54 ◼   ► to work offline is just shrinking dramatically over time.

00:23:58 ◼   ► - Yeah, and I think there's something, too.

00:24:01 ◼   ► It's changing the user's expectations,

00:24:05 ◼   ► and it changes from a persistence problem

00:24:07 ◼   ► to a caching problem, because I think of even an app,

00:24:10 ◼   ► say like the app like Instagram, right,

00:24:12 ◼   ► or Tweetbot, I think is another app sort of like this,

00:24:14 ◼   ► where you have these apps that are essentially server-based.

00:24:18 ◼   ► What you're showing by its nature is somewhat time-bound,

00:24:22 ◼   ► and so you're always going to want to get the latest stuff.

00:24:25 ◼   ► And my expectation for that app, though,

00:24:27 ◼   ► is that when I open it, if I'm offline,

00:24:30 ◼   ► that something reasonable happens.

00:24:32 ◼   ► It's not just like frowny face, "Sorry."

00:24:36 ◼   ► Instead, what I see in both of those apps

00:24:38 ◼   ► is they show some kind of cache of whatever it is.

00:24:42 ◼   ► It's the last data that they showed you.

00:24:44 ◼   ► It's some version of that,

00:24:47 ◼   ► and I think it's a difference in framing,

00:24:50 ◼   ► but I think that is a significant difference

00:24:52 ◼   ► in user expectation, where if you can change the mindset

00:24:56 ◼   ► from true persistence, from it being a peer with your server

00:25:01 ◼   ► to being just a sort of a dumb client that does caching,

00:25:05 ◼   ► and it's not persistence, really, it's just caching.

00:25:08 ◼   ► You know, caching is certainly a problem.

00:25:10 ◼   ► It's what they always say.

00:25:12 ◼   ► It's one of the three biggest problems in computer science

00:25:14 ◼   ► is cache invalidation.

00:25:16 ◼   ► It's not an easy problem, necessarily,

00:25:17 ◼   ► but it's something that is definitely more straightforward,

00:25:22 ◼   ► where if you always assume that the server is in charge

00:25:25 ◼   ► and you can reasonably have access to it most of the time,

00:25:29 ◼   ► you get away with it, you cache and show

00:25:31 ◼   ► whatever you last had otherwise.

00:25:33 ◼   ► And I've definitely built apps where sometimes this can even get

00:25:36 ◼   ► really, really simple, and I love when an app can--

00:25:39 ◼   ► like, there's a really easy, elegant solution

00:25:41 ◼   ► for something like this, where if your app just persists--

00:25:44 ◼   ► you know, downloads JSON and shows it to the user,

00:25:48 ◼   ► well, why don't you just go ahead and cache

00:25:50 ◼   ► your JSON requests as you do them,

00:25:52 ◼   ► and you can use exactly the same logic if you're offline

00:25:55 ◼   ► when you're online to just load the last JSON blob

00:25:58 ◼   ► that you had and display that to your user.

00:26:00 ◼   ► Like, you don't even need persistence in a way of,

00:26:03 ◼   ► you know, there's some kind of complicated mapping

00:26:05 ◼   ► that's turning it into something else.

00:26:07 ◼   ► It's like, nope, it all just starts at the end of the day.

00:26:09 ◼   ► It's like it all is just these plain text files at the end,

00:26:13 ◼   ► which maybe it's another analog to where, like, blogging

00:26:16 ◼   ► or any of these systems where if you can ultimately end up

00:26:20 ◼   ► just with plain text, you're almost always on the right path.

00:26:23 ◼   ► - Well, also, like, you know, like, that approach

00:26:25 ◼   ► of just using cached things from a server,

00:26:28 ◼   ► you can often get exactly what you need from that

00:26:31 ◼   ► with just the built-in NSURL cache system

00:26:33 ◼   ► that you can access from every network request in iOS.

00:26:37 ◼   ► Like, there is this entire caching layer, like, you know,

00:26:40 ◼   ► if people who make network requests, you ever wonder

00:26:42 ◼   ► what that caching policy option does?

00:26:44 ◼   ► Well, it turns out if you dive deeply into that,

00:26:46 ◼   ► there is a lot of control you have.

00:26:48 ◼   ► You can basically customize the entire layer,

00:26:50 ◼   ► customize every request.

00:26:52 ◼   ► You don't even have to control the API.

00:26:54 ◼   ► You can modify the cache behavior of requests

00:26:56 ◼   ► as they come in.

00:26:57 ◼   ► Like, you can, there is so much you can do

00:26:59 ◼   ► with just effectively basic HTTP caching

00:27:02 ◼   ► and, you know, basic cache and validation headers,

00:27:05 ◼   ► like, you know, the expires headers and ETags

00:27:07 ◼   ► and stuff like that, and that's all built in.

00:27:09 ◼   ► So you don't even necessarily have to think about where,

00:27:13 ◼   ► you know, do I need to write this file to disk

00:27:15 ◼   ► in a certain spot?

00:27:16 ◼   ► You know, how do I expire this data?

00:27:18 ◼   ► No, you can let the system handle it for you.

00:27:20 ◼   ► There's tons of functionality built in

00:27:22 ◼   ► just by using the built-in URL loading system.

00:27:24 ◼   ► Sure, and that's even better.

00:27:25 ◼   ► I mean, that sounds like a wonderful thing.

00:27:27 ◼   ► I mean, now that you're saying that, I'm like,

00:27:29 ◼   ► man, I've been doing this the wrong way for a little bit

00:27:31 ◼   ► because I tend to write the files to disk.

00:27:33 ◼   ► But, yeah, like, the less that you can do, the better.

00:27:39 ◼   ► Yeah, if you can just say, you know, hey,

00:27:41 ◼   ► keep around the last version of a request you made

00:27:44 ◼   ► or, like you said, when you start to get into ETags

00:27:46 ◼   ► and expires headers and things that are, like,

00:27:48 ◼   ► purpose-built for this purpose,

00:27:50 ◼   ► that you're not having to build anything for it.

00:27:52 ◼   ► Like, yours, if you have a well-behaved web service,

00:27:56 ◼   ► it probably is doing a lot of this for you anyway.

00:28:00 ◼   ► The ability to do this kind of stuff,

00:28:01 ◼   ► that you can then just not worry about it,

00:28:03 ◼   ► and your app just makes a request,

00:28:05 ◼   ► and if you're offline, you get the cached data,

00:28:07 ◼   ► and if you're online, you get the fresh data,

00:28:10 ◼   ► and, you know, don't worry about it.

00:28:12 ◼   ► And you've got to be careful, certainly,

00:28:14 ◼   ► as with all these things.

00:28:15 ◼   ► Like, I notice this when I travel internationally.

00:28:18 ◼   ► It's usually when I run into this kind of issue,

00:28:21 ◼   ► where you start to really notice which apps do a good job

00:28:23 ◼   ► of handling poor connectivity situations

00:28:26 ◼   ► or no connectivity situations.

00:28:29 ◼   ► But the reality is, if you worry about those edge cases too much,

00:28:35 ◼   ► you'll end up making your app worse overall, I think.

00:28:38 ◼   ► And so being careful about not getting too worried.

00:28:41 ◼   ► It's like, well, what if someone's going on an epic expedition

00:28:45 ◼   ► and they're going to be not connected to the internet

00:28:47 ◼   ► for a week at a time, should they still be able to use the app?

00:28:51 ◼   ► It's like, well, maybe. I don't know.

00:28:54 ◼   ► It's like, if you have a really compelling reason, great.

00:28:57 ◼   ► But if you don't, don't over-engineer it,

00:28:59 ◼   ► just because you think you maybe should or could.

00:29:02 ◼   ► Right, because any of that engineering,

00:29:04 ◼   ► not only does it take a lot of time and energy

00:29:06 ◼   ► that you could be spending on features,

00:29:07 ◼   ► but it also introduces tons of bug potential.

00:29:10 ◼   ► So the least persistence you can do, the better.

00:29:14 ◼   ► And I think your example there is great.

00:29:16 ◼   ► It's like, if someone's going to be out away from connectivity for a while,

00:29:19 ◼   ► well, is your app really that useful to somebody

00:29:22 ◼   ► if everything in it is stale anyway?

00:29:24 ◼   ► It depends on the app.

00:29:25 ◼   ► If it's something like a podcast player, yes,

00:29:28 ◼   ► people can load up on a podcast and live off them for weeks.

00:29:30 ◼   ► But if it's something like a Twitter client, no,

00:29:32 ◼   ► that's not going to be useful to them.

00:29:34 ◼   ► So it depends on the app, but I think the answer for most apps

00:29:37 ◼   ► is probably you don't really need to worry about it.

00:29:39 ◼   ► Exactly.

00:29:40 ◼   ► All right, well, we're out of time this week.

00:29:42 ◼   ► Thank you, everybody, for listening, and we'll talk to you next week.

00:29:45 ◼   ► Bye!

00:29:46 ◼   ► [BLANK_AUDIO]