Under the Radar

47: Data Persistence


00:00:00   Welcome to Under the Radar, a show about independent iOS app development.

00:00:04   I'm Marco Arment.

00:00:05   And I'm David Smith.

00:00:06   Under the Radar is never longer than 30 minutes, so let's get started.

00:00:10   So today we wanted to dive a little bit into the weeds,

00:00:14   go all the way down the stack, you know, as you kind of, you know,

00:00:18   if you were drawing one of those silly, what is it, OPML?

00:00:21   No, there's a fancy name for those kind of charts that you learn about in school,

00:00:26   but they never actually draw ever again.

00:00:28   How do you not know this?

00:00:30   I mean, I don't know what you're talking about, but I figured you are the master of charts.

00:00:33   You should know this.

00:00:34   Well, it's not a chart. It's like drawing. I'm terrible at drawing.

00:00:38   You're like a flow chart?

00:00:39   Well, I'm thinking of the one, you know, you go all the way down,

00:00:42   there's like, the database was always that cylinder that you would draw.

00:00:47   I don't know why it was a cylinder.

00:00:48   Like, is this back from the days when you had tape disks or something?

00:00:51   It was probably to represent like a hard disk.

00:00:53   Maybe. So, you have, when you draw your infrastructure diagram,

00:00:59   you always end up in the back with a little cylinder, and that's your database.

00:01:03   And it's, you know, an important part of applications in certain ways,

00:01:10   but I think the role that persistence has played, and even as you just noticed there,

00:01:15   I switched from using database to persistence,

00:01:17   is the way in which we think about storing data to then get back later

00:01:21   has certainly evolved over the last, you know, the last eight years of iOS

00:01:26   and of the App Store.

00:01:28   And I think it seems something that's worth getting into,

00:01:30   because there's such a variety of approaches that you can take for this,

00:01:34   where you go all the way on the one extreme to things like raw SQLite.

00:01:39   You can then take wrappers around SQLite.

00:01:43   You can have things that are like core data using SQLite,

00:01:47   which are Realm or all kinds of other kind of middlemen that are wrapper,

00:01:53   like very sophisticated wrappers.

00:01:55   You can start to get into systems where you have your own persistence system

00:02:01   that is writing files to disk.

00:02:03   You can start to get into just using NSUserDefaults and Keychain

00:02:08   and very lightweight things,

00:02:10   and then you can go all the way to the other extreme of having no persistence at all,

00:02:14   and I know of a variety of apps that are entirely unpersisted,

00:02:18   that other than like a user token, the app is entirely unpersisted.

00:02:23   And that flexibility is, I think, in many ways a direct result

00:02:29   of the increasing capability and connectivity of the devices

00:02:33   that we're writing apps for,

00:02:35   where I think if you imagine an app as this thing all on its own,

00:02:41   it's going to need a fairly sophisticated persistence system

00:02:44   because that is where the apps--

00:02:47   it needs to be able to function on its own,

00:02:49   and it needs to be able to do it in a way that,

00:02:51   if we go back in the day when you only had a 3G connection

00:02:54   connected to your iPhone,

00:02:56   you really couldn't expect the user to be able to download information very often,

00:02:59   so it has to be fairly self-contained,

00:03:01   whereas you can imagine now there are certain kind of applications

00:03:03   that are essentially just displaying a JSON blob to the user,

00:03:10   and maybe the data is in some ways also time sensitive.

00:03:15   Showing the data that you had cached a week ago would be kind of irrelevant.

00:03:20   So say, for example, like a weather app.

00:03:22   It's an example of an app I've written that has essentially no persistence

00:03:25   because the app's data is completely useless.

00:03:30   But I think it's an interesting thing to think about.

00:03:32   Where we--

00:03:34   Hi, I'm David Smith. My app's data is completely useless.

00:03:37   Hey, you know, sometimes-- I'm saying it's useless as soon as it's old.

00:03:43   Yeah, it's immediately out of date, basically.

00:03:45   It's immediately out of date. It's necessary.

00:03:47   As soon as I have my blob of weather data,

00:03:51   it's already starting to get out of date and fall out.

00:03:55   But anyway, getting back on my train of thought,

00:03:58   the other thing that has made persistence I think so interesting now

00:04:01   is the rise of extensions,

00:04:04   and this is something that I've been struggling with a lot this summer

00:04:07   of trying to keep all the various parts of our applications in sync.

00:04:12   So, Podometer++, for example, has an iPhone app,

00:04:16   it has a widget extension,

00:04:18   it has a watch extension, which actually is two parts.

00:04:22   One of them is just the UI, and the other part's the actual watch extension.

00:04:26   It has a SiriKit extension, and it has an iMessage extension.

00:04:30   And so I have five different sort of things that I have to keep data in sync now

00:04:35   that isn't syncing to a server, that is just syncing locally.

00:04:39   And so persistence is a really interesting and tricky problem now,

00:04:42   and I think what I find myself typically doing now

00:04:45   is I'm trying to simplify as much as I can my persistence

00:04:51   to a point that essentially most of my persistence now is just dictionaries

00:04:56   and sort of having these blobs of data that I can very easily move around

00:05:01   in a very lightweight way because I need to get data

00:05:04   so that when the watch app updates its step count,

00:05:08   that watch, that step count needs to, via watch connectivity,

00:05:11   find its way down into my phone,

00:05:14   which needs to find itself into the iPhone app,

00:05:16   it needs to find its way into the iMessage app,

00:05:18   it needs to find itself into the widget so that everything can stay in sync.

00:05:21   And so having some big heavyweight persistence scheme

00:05:24   that is doing some kind of really heavy database-oriented transactional things

00:05:30   or things just doesn't work, it's too heavyweight.

00:05:33   And so I'm starting to more and more just shift myself away from that

00:05:36   into dictionaries and that kind of stores.

00:05:40   And so far I've found that to be pretty effective.

00:05:43   Have you had to run into any of these kind of problems with Overcast

00:05:47   in terms of keeping various parts of the app in sync?

00:05:49   Oh, absolutely.

00:05:50   And in fact I've kind of gone the opposite direction as you over the years.

00:05:53   Well, sort of.

00:05:54   So with Instapaper early on, I was just using raw SQLite API in C

00:06:01   because I thought the speed and performance would matter.

00:06:04   And maybe at the time it did.

00:06:06   That was back on the iPhone 1, so it might have mattered back then.

00:06:09   I've slowly moved first into using FMDB

00:06:14   and then later on building on top of FMDB my own FC model system,

00:06:19   which is kind of a lighter weight than core data persistence layer

00:06:24   built on top of SQLite.

00:06:26   And for different apps that were not like my big app,

00:06:31   things like the magazine and Bugshot,

00:06:34   I've used different things there, either no persistence

00:06:37   or in the case of the magazine I first started out with just using

00:06:40   a bunch of plists that I would just write to disk.

00:06:43   And having dictionaries in plists is enough for a lot of apps.

00:06:49   And as you mentioned, kind of going towards that kind of area

00:06:53   with a lot of your apps, there's a lot to be said for that.

00:06:56   Just having very simple dictionaries in files,

00:07:00   there's a lot to be said to that.

00:07:02   The setup cost, the amount of effort it takes to start working on that

00:07:07   is extremely low.

00:07:09   The amount of complexity in the app is extremely low.

00:07:12   The bug potential in most cases, but not all, is extremely low.

00:07:17   And it's also just really fast. There's a lot to be said for that.

00:07:21   However, when you go to the more sophisticated database-backed systems,

00:07:26   things like core data, then you get a lot, you view it as heavy

00:07:33   or as maybe over-designed for your needs,

00:07:36   depending on what your needs are for the particular app.

00:07:38   But database systems give you a lot for free

00:07:42   that if you are using a file-backed system,

00:07:45   or a more manual or simple system,

00:07:47   that you have to either go without and hope it doesn't bite you

00:07:51   or you have to write it yourself, which is probably not a good idea.

00:07:55   So things like concurrent access to the same file not corrupting the data.

00:08:01   Things like, in your extension contexts,

00:08:05   having the data be able to be updated by one process

00:08:07   and then notifying the others that might be accessing it at the same time,

00:08:11   notifying them of the changes.

00:08:13   There are things like this that the good persistence layers will do already for you.

00:08:19   You don't have to write it.

00:08:21   And if you use them even for simple tasks,

00:08:23   it might be worth getting those features.

00:08:25   Now, they do bring complexity, for sure.

00:08:28   Core data, I'm not a huge expert in core data

00:08:31   because I only used it for basically the second half of the magazine

00:08:34   after I stopped using my P-list system.

00:08:36   So I used it very little and not recently.

00:08:39   So all this with a grain of salt.

00:08:41   But there are obviously complexities.

00:08:43   There's things like concurrent access from different threads

00:08:45   that you have to really be careful with.

00:08:47   Although, again, same thing.

00:08:49   You have to be careful when you're writing files, too.

00:08:51   There's things like dealing with schema migrations

00:08:55   that a lot of these layers do not make easy.

00:08:58   Possible crashes, like if the migration fails on an upgrade or whatever else,

00:09:03   or the database gets corrupt.

00:09:05   A lot of apps can't or don't handle that very well.

00:09:08   So there's lots of complexity if you take on some of the bigger systems

00:09:13   that you have to deal with.

00:09:14   But I would posit that in most cases for most apps,

00:09:18   if you're doing things like extensions or concurrent threads in one app,

00:09:24   you probably want those big systems

00:09:26   because they will take care of problems

00:09:29   that you would have to worry about if you were doing it yourself

00:09:32   with plists and stuff in your own app.

00:09:35   Does that make sense?

00:09:36   Yeah, it is such an interesting tension

00:09:39   that I feel like you find with these types of questions.

00:09:42   The thing that I hate,

00:09:44   one of the things I really don't like when I'm developing an app

00:09:47   is whenever I make a decision that is setting me on a path

00:09:51   where it will be difficult to change down the road.

00:09:54   So if I go file a new project in Xcode,

00:09:59   and it says, "Would you like to check the box to use Core Data?"

00:10:03   Now obviously that checkbox itself is not the--

00:10:07   I can obviously take it out or put it back in later.

00:10:09   But conceptually, I agonize over whether I'm going to push that button

00:10:14   because it feels like I'm setting myself--

00:10:17   I'm making a choice that has a lot of implications down the road.

00:10:20   And it's like I'm choosing which adventure I'm going to be going on.

00:10:26   Am I going to be going on the Core Data adventure

00:10:29   of finding weird threading issues

00:10:31   and dealing with merging in the background?

00:10:34   Whenever I've done Core Data, and I've done a lot of Core Data,

00:10:36   that seems to be the adventure you're on,

00:10:38   where the bugs you're going to have are related to that.

00:10:41   Or am I going to go down the road of a letter-weight file-based solution

00:10:46   where my bugs and issues are going to be cache invalidation or freshness,

00:10:51   those types of issues.

00:10:52   I'm just choosing which one I'm doing.

00:10:55   And really awkward is if your app involves any amount of user data,

00:10:59   as soon as you have that first version out in the world,

00:11:03   suddenly you are going to be responsible for maintaining that--

00:11:07   in some ways indefinitely, not truly indefinitely.

00:11:10   But you then have to start dealing with,

00:11:13   "Well, if I ever change my mind and want to do something different,

00:11:16   I'm going to have to build some kind of big migration.

00:11:19   I'm going to have to test it like crazy.

00:11:21   And I'm going to have to work really hard

00:11:23   to make sure that I don't break any user's data."

00:11:26   Because that's the nature of persistence.

00:11:30   You're trying to take something, put it on your phone,

00:11:32   and have it last indefinitely.

00:11:35   And so I really hate these kind of decisions

00:11:38   that we have to make in development, where it's not just like,

00:11:40   "Oh, Core Data isn't really working out for me right here.

00:11:43   I think I want to go to SQLite. Let me just do that."

00:11:46   It's like, well, there goes three months of my life,

00:11:48   transitioning from pulling the data out

00:11:51   and switching it into something else.

00:11:53   And that kind of a heavyweight decision, I think, is so awkward.

00:11:56   And also, it's probably also a fair thing to say,

00:11:58   that I think it is good to have the awareness that

00:12:01   because these decisions tend to have such high switching costs,

00:12:06   the discussion about which one you should use.

00:12:10   So if you are talking online,

00:12:12   or you're reading documentation or blog posts about these things,

00:12:15   people often become very passionate about them.

00:12:19   It's the old ViEmacs kind of problem,

00:12:22   where you become very passionate about your thing

00:12:25   and very defensive of it,

00:12:27   because the thought of having--

00:12:29   If someone else is actually correct,

00:12:31   in terms of, oh, some new persistent scheme comes out

00:12:34   that is objectively better than what you have

00:12:37   and what you've been using,

00:12:39   you have a problem, and you have a serious problem,

00:12:41   and you have this weird issue that you then feel bad about.

00:12:44   And so discussions with this are always kind of tinged in that way.

00:12:48   And so it's just a good disclaimer to kind of talk about,

00:12:50   because it's just such a high switching cost.

00:12:53   You can't just be like, "Yeah, you know, maybe I will use that.

00:12:56   Maybe I will drop this and switch to that."

00:12:58   I've done migrations like that,

00:13:00   where I've gone from one kind of data system to another.

00:13:03   And it is usually-- That is the worst part of my job ever,

00:13:07   and I try as best I can.

00:13:09   Once I make the choice, it's like, I'm just going to go with it,

00:13:12   and I'm just going to do the best I can.

00:13:14   I have a recipe book manager,

00:13:16   which is the most terrifying of my persistence needs,

00:13:19   because people are literally putting their family recipes,

00:13:23   things that are very emotionally important to them, into my app,

00:13:29   and they expect them to be there.

00:13:31   And I've had issues where I do a bad data migration in Core Data,

00:13:35   which is what I use for that app.

00:13:37   I do a bad data migration, and it corrupts some recipes.

00:13:43   That is terrible.

00:13:46   And so you have to be really thoughtful about these kinds of things.

00:13:50   And I think in some ways, as I go forward,

00:13:53   I'm getting less and less, wanting more and more lightweight things.

00:13:57   I'm almost in some ways trying to make fewer and fewer promises

00:14:00   and trying to have fewer situations

00:14:05   where I am really responsible inside of my app code

00:14:09   for taking care of something that's important to somebody,

00:14:11   that as soon as I can, like in my recipe book now,

00:14:15   I added a free sync system to it that takes all the--

00:14:19   essentially, I call it sync, most people just use it with one app.

00:14:23   It's just backup.

00:14:25   So I have a live backup of all their user data out into a database

00:14:29   that I can keep for them,

00:14:31   so that now when weird things happen to users' devices

00:14:34   and the persistence scheme goes crazy and Core Data gets--

00:14:37   it's not uncommon for me to get a request from somebody

00:14:40   that when I ask them for the crash log and it comes back

00:14:43   and I get the thing and it's just like,

00:14:45   it's a Core Data error that just says the SQLite file has become--

00:14:48   is malformed and cannot be opened.

00:14:51   And that happens all the time on iOS devices.

00:14:53   Like it happens-- one of the biggest culprits on iOS devices

00:14:57   is if the disk runs out of space,

00:14:59   then weird things can happen with your persistence layer,

00:15:01   depending on how it's done and what its settings are,

00:15:04   and that very frequently can corrupt SQLite databases.

00:15:08   It shouldn't, but in practice it does.

00:15:10   Or things like, you know, sometimes like,

00:15:13   when writing to the app container,

00:15:15   the share container between extensions,

00:15:17   sometimes those writes will just fail for some reason.

00:15:20   Or they'll be denied in some weird permissions error or something.

00:15:24   And it's like, well, then like,

00:15:26   if a write fails to your persistence layer,

00:15:28   then like that can also potentially corrupt the data that's there or something.

00:15:31   And it's like, this stuff happens all the time,

00:15:33   and it's so nice if you have the luxury

00:15:35   to be able to have in your app code, as I do in Overcast,

00:15:37   if the database file is corrupt,

00:15:39   just delete it and resync everything from the server.

00:15:41   Yeah, exactly.

00:15:43   And that's the place where persistence

00:15:45   and the really industrial-grade persistence stuff comes into play.

00:15:50   I love that-- I've been using-- I'm a Postgres guy.

00:15:54   I mean, you know you're a MySQL person,

00:15:56   but like, functionally it doesn't really matter.

00:15:58   Well, it doesn't matter. This is the thing. I don't care.

00:16:00   They're both great. I've just been using Postgres for a long time,

00:16:03   and so I know how to do it.

00:16:05   But the thing is, Postgres, like,

00:16:07   I've done some horrible things to Postgres databases,

00:16:10   and I've never lost data as a result.

00:16:13   You know, I've had weird situations where, you know,

00:16:15   I accidentally powered off the wrong server in the middle of something,

00:16:18   and like, it was in the middle of doing a migration,

00:16:21   and it got canceled halfway.

00:16:24   Like, those types of systems are designed for this kind of recovery

00:16:28   in a way that SQLite, which is, at the end,

00:16:32   what most of the persistence schemes on device use, aren't.

00:16:36   And so it's definitely something that I think now I'm leaning so much towards the,

00:16:40   "Let's get this all onto a nice, safe server somewhere

00:16:43   with some really proper, you know, backup,

00:16:46   whereas I don't have to worry about, like,

00:16:48   it's not the users doing backups and restores.

00:16:50   It's like, I'm doing backup and restore. I'm versioning the database.

00:16:53   I'm making sure that everything is nice and safe

00:16:55   in a way that, on device, like you said,

00:16:58   if something goes wrong, it's like, just say,

00:17:00   "Oh, I'm sorry, I need you to connect to the network

00:17:02   so I can go grab the most recent version of this."

00:17:05   And that, like, coding with that in mind,

00:17:10   I think, makes so many of these problems just go,

00:17:12   either go away or be so much less stressful.

00:17:14   - Yeah, because, and, you know, the reality is,

00:17:16   like, your customers will have, you know,

00:17:19   what if their phone, well, before the iPhone 7,

00:17:22   what if it falls in a toilet or something?

00:17:24   You know, like, they lose everything on their phone,

00:17:26   or their phone gets stolen, or whatever, their phone fails,

00:17:29   and they have to get a new one,

00:17:30   and they have to restore from backup.

00:17:31   Well, do they have a backup? How old is that backup?

00:17:33   And, like, the answer to those questions might be bad

00:17:36   for the data, and so if you just always have everything

00:17:40   synced to a server somewhere that you can always restore

00:17:43   back to the phone as soon as the user logs in,

00:17:45   and whether it's iCloud or your own stuff,

00:17:47   for this purpose, it doesn't really matter,

00:17:48   that is such a better place to be than trying to rely

00:17:51   on storing user data indefinitely,

00:17:53   just locally in the app, as you said.

00:17:55   - Oh, sure, 'cause I mean, even for,

00:17:57   and there's a couple of situations that I run into

00:17:59   that are a bit trickier for me,

00:18:00   because some of my apps use store health data,

00:18:04   and there's some very specific and complicated rules

00:18:07   around storing health-related data.

00:18:10   I mean, it's not like health in the sense of diagnosing diseases,

00:18:13   but, you know, steps, sleep data, activity data,

00:18:16   like, these are things that fall broadly

00:18:18   into the category of health data,

00:18:20   and so I currently, I don't store any of those

00:18:23   anywhere off-device, because if I had a server,

00:18:26   that had stored health data, suddenly there's all kinds

00:18:29   of regulatory things that I don't really know about,

00:18:32   that I'm sure exist, and I really don't want to know about,

00:18:35   that would start to come into play,

00:18:37   and there's rules about not using iCloud, necessarily,

00:18:40   but as a result, I do a lot of my backup and restore

00:18:43   using the either iTunes or iCloud backup system,

00:18:47   and it sometimes doesn't work.

00:18:49   Like, there's just no two ways around it,

00:18:51   where sometimes someone doesn't update.

00:18:53   You know, I've had a couple of these

00:18:54   this last couple weeks, where someone updates to iOS 10,

00:18:58   something goes weird in the update,

00:19:00   they restore from the backup, and the app state is just missing.

00:19:04   And there's nothing I can do.

00:19:06   - That's horrible.

00:19:08   - There's nothing that I can do for that person,

00:19:11   and, you know, and so, but it's a situation

00:19:14   where I wish I did have a means to back that up externally,

00:19:18   because there is nothing better when you have,

00:19:21   like, this has happened dozens of times with my recipe app,

00:19:24   where something happens to someone's device,

00:19:27   and they send an email that is just the, like, the panicked,

00:19:30   like, I have all of my family recipes in this app,

00:19:33   I've been using it, I love it, you know,

00:19:35   my kid threw the iPad out the window and it shattered,

00:19:39   and I didn't have a backup of the device.

00:19:42   Like, is there anything that can be done?

00:19:44   And there is the, it is just absolute,

00:19:46   it is an absolute joy to be able to go into this,

00:19:49   like, go into the admin panel, verify that, you know,

00:19:51   when I last, they last synced their system,

00:19:54   I'd be able to say to them, it's like, you know what,

00:19:56   I have 267 recipes, safe and sound,

00:19:59   whenever you get your new device,

00:20:00   log in with your credentials, and they'll all be back to you.

00:20:03   Like, that is amazing, that is, like,

00:20:05   you just made someone's day, and for what was otherwise,

00:20:08   it was gonna be a really, really bad day.

00:20:10   - I love that we chose to do this episode

00:20:13   on, like, the local persistence schemes that you use

00:20:16   in your app, and meanwhile, our conclusion is,

00:20:18   don't rely on them for anything. (laughs)

00:20:21   - That's true, but I think that is in some ways the point.

00:20:23   This is where I've been going to more and more.

00:20:25   It's like, there's, you can so easily get focused

00:20:28   on what is the best local scheme for doing this,

00:20:31   and the more and more of this I do,

00:20:34   the less and less I think that storing data on device

00:20:37   is a good idea for most apps, that I think most apps

00:20:40   benefit from instead of having their data locally,

00:20:43   pushing it somewhere else, where they, like you said,

00:20:45   whether that's iCloud, whether that's your own servers,

00:20:47   whatever, the more that you can push your data somewhere else

00:20:51   and then be pulling it down as needed,

00:20:53   the better you're gonna be, because there's just

00:20:55   so many issues that you're gonna run into otherwise.

00:20:59   - And if you do wanna run your own servers,

00:21:01   our sponsor this week is perfectly timed, it's Linode.

00:21:05   So Linode, Linode is a wonderful web host

00:21:08   with high performance Linux SSD servers,

00:21:10   spread across eight data centers around the world.

00:21:12   They're a fantastic solution for your server infrastructure.

00:21:15   Linode offers servers that you can get up and running

00:21:17   in just under a minute, with plans starting at just $10

00:21:20   a month, and that now gets you two gigs of RAM

00:21:22   for 10 bucks a month, so that could be, I mean,

00:21:24   look, these servers are powerful, these are really fast,

00:21:26   we use them, I have, I think 12 Linode instances

00:21:29   that I run Overcast on, plus one for market.org,

00:21:31   plus one for some accessory stuff, I have lots of Linodes.

00:21:34   I bet most apps could get away with the $10 a month plan

00:21:38   for all of their user data.

00:21:40   It's really, you get a lot for the money here.

00:21:42   Anyway, Linode is great for other tasks.

00:21:44   Also, things like if you wanna run your own private

00:21:46   Git server, if you wanna host large databases,

00:21:49   like 267 recipes, if you wanna run a mail server,

00:21:52   if you wanna operate powerful apps, and so much more.

00:21:55   With industry-leading native SSD storage

00:21:58   and access to a 40 gigabit network,

00:22:00   you will have all the power you need

00:22:01   to get your tasks done at Linode.

00:22:03   Now, if you're a listener of this show,

00:22:05   sign up at Linode.com/Radar.

00:22:08   You'll not only be supporting us,

00:22:09   but you'll also get $20 towards any Linode plan.

00:22:12   You can also use code radar20 at checkout

00:22:15   to get that same deal.

00:22:16   So, there's a seven-day money-back guarantee.

00:22:17   There's nothing to lose.

00:22:18   Go to Linode.com, spelled like Linux, but Linode.

00:22:21   Linode.com/Radar to learn more,

00:22:24   sign up and take advantage of that $20 credit

00:22:26   or use promo code radar20 at checkout.

00:22:29   Thank you so much to Linode for supporting this show.

00:22:32   So, I think it's worth also mentioning,

00:22:35   you mentioned at the end of the last segment

00:22:37   about how it's actually becoming more and more useful

00:22:39   to not store anything on the device

00:22:41   and just always fetch it from the server.

00:22:43   Back when I first started developing,

00:22:45   which was Instapaper back then,

00:22:47   offline access was a big deal, especially for that app.

00:22:50   That was kind of the point of it,

00:22:51   and that was a major feature of it.

00:22:54   And I think over time, we've slowly had times

00:22:58   in which we are offline have slowly eroded

00:23:01   out of people's lives, and they're still there.

00:23:03   There are still occasions where people are offline,

00:23:06   but that is getting really few and far between.

00:23:09   I mean, now, you can now even use devices on planes

00:23:13   during takeoff and landing.

00:23:15   You can be online for the entire middle part

00:23:17   of the flight if you want to,

00:23:19   and that's only gonna get more and more common over time.

00:23:21   A lot of subways, like underground transit systems,

00:23:25   where there was no reception 10 years ago,

00:23:29   they're starting to get cell reception, cell coverage

00:23:31   in a lot of them now.

00:23:32   There's more and more areas where people

00:23:34   have more and more connectivity.

00:23:36   Offline access is one of the only reasons

00:23:40   why you would store data locally

00:23:42   for a lot of server-backed apps,

00:23:44   and I think it's very reasonable for a lot of apps today

00:23:48   to just be like, "Sorry, we don't work offline,"

00:23:50   because the actual demand for getting the app

00:23:54   to work offline is just shrinking dramatically over time.

00:23:58   - Yeah, and I think there's something, too.

00:24:01   It's changing the user's expectations,

00:24:05   and it changes from a persistence problem

00:24:07   to a caching problem, because I think of even an app,

00:24:10   say like the app like Instagram, right,

00:24:12   or Tweetbot, I think is another app sort of like this,

00:24:14   where you have these apps that are essentially server-based.

00:24:18   What you're showing by its nature is somewhat time-bound,

00:24:22   and so you're always going to want to get the latest stuff.

00:24:25   And my expectation for that app, though,

00:24:27   is that when I open it, if I'm offline,

00:24:30   that something reasonable happens.

00:24:32   It's not just like frowny face, "Sorry."

00:24:36   Instead, what I see in both of those apps

00:24:38   is they show some kind of cache of whatever it is.

00:24:42   It's the last data that they showed you.

00:24:44   It's some version of that,

00:24:47   and I think it's a difference in framing,

00:24:50   but I think that is a significant difference

00:24:52   in user expectation, where if you can change the mindset

00:24:56   from true persistence, from it being a peer with your server

00:25:01   to being just a sort of a dumb client that does caching,

00:25:05   and it's not persistence, really, it's just caching.

00:25:08   You know, caching is certainly a problem.

00:25:10   It's what they always say.

00:25:12   It's one of the three biggest problems in computer science

00:25:14   is cache invalidation.

00:25:16   It's not an easy problem, necessarily,

00:25:17   but it's something that is definitely more straightforward,

00:25:22   where if you always assume that the server is in charge

00:25:25   and you can reasonably have access to it most of the time,

00:25:29   you get away with it, you cache and show

00:25:31   whatever you last had otherwise.

00:25:33   And I've definitely built apps where sometimes this can even get

00:25:36   really, really simple, and I love when an app can--

00:25:39   like, there's a really easy, elegant solution

00:25:41   for something like this, where if your app just persists--

00:25:44   you know, downloads JSON and shows it to the user,

00:25:48   well, why don't you just go ahead and cache

00:25:50   your JSON requests as you do them,

00:25:52   and you can use exactly the same logic if you're offline

00:25:55   when you're online to just load the last JSON blob

00:25:58   that you had and display that to your user.

00:26:00   Like, you don't even need persistence in a way of,

00:26:03   you know, there's some kind of complicated mapping

00:26:05   that's turning it into something else.

00:26:07   It's like, nope, it all just starts at the end of the day.

00:26:09   It's like it all is just these plain text files at the end,

00:26:13   which maybe it's another analog to where, like, blogging

00:26:16   or any of these systems where if you can ultimately end up

00:26:20   just with plain text, you're almost always on the right path.

00:26:23   - Well, also, like, you know, like, that approach

00:26:25   of just using cached things from a server,

00:26:28   you can often get exactly what you need from that

00:26:31   with just the built-in NSURL cache system

00:26:33   that you can access from every network request in iOS.

00:26:37   Like, there is this entire caching layer, like, you know,

00:26:40   if people who make network requests, you ever wonder

00:26:42   what that caching policy option does?

00:26:44   Well, it turns out if you dive deeply into that,

00:26:46   there is a lot of control you have.

00:26:48   You can basically customize the entire layer,

00:26:50   customize every request.

00:26:52   You don't even have to control the API.

00:26:54   You can modify the cache behavior of requests

00:26:56   as they come in.

00:26:57   Like, you can, there is so much you can do

00:26:59   with just effectively basic HTTP caching

00:27:02   and, you know, basic cache and validation headers,

00:27:05   like, you know, the expires headers and ETags

00:27:07   and stuff like that, and that's all built in.

00:27:09   So you don't even necessarily have to think about where,

00:27:13   you know, do I need to write this file to disk

00:27:15   in a certain spot?

00:27:16   You know, how do I expire this data?

00:27:18   No, you can let the system handle it for you.

00:27:20   There's tons of functionality built in

00:27:22   just by using the built-in URL loading system.

00:27:24   Sure, and that's even better.

00:27:25   I mean, that sounds like a wonderful thing.

00:27:27   I mean, now that you're saying that, I'm like,

00:27:29   man, I've been doing this the wrong way for a little bit

00:27:31   because I tend to write the files to disk.

00:27:33   But, yeah, like, the less that you can do, the better.

00:27:39   Yeah, if you can just say, you know, hey,

00:27:41   keep around the last version of a request you made

00:27:44   or, like you said, when you start to get into ETags

00:27:46   and expires headers and things that are, like,

00:27:48   purpose-built for this purpose,

00:27:50   that you're not having to build anything for it.

00:27:52   Like, yours, if you have a well-behaved web service,

00:27:56   it probably is doing a lot of this for you anyway.

00:28:00   The ability to do this kind of stuff,

00:28:01   that you can then just not worry about it,

00:28:03   and your app just makes a request,

00:28:05   and if you're offline, you get the cached data,

00:28:07   and if you're online, you get the fresh data,

00:28:10   and, you know, don't worry about it.

00:28:12   And you've got to be careful, certainly,

00:28:14   as with all these things.

00:28:15   Like, I notice this when I travel internationally.

00:28:18   It's usually when I run into this kind of issue,

00:28:21   where you start to really notice which apps do a good job

00:28:23   of handling poor connectivity situations

00:28:26   or no connectivity situations.

00:28:29   But the reality is, if you worry about those edge cases too much,

00:28:35   you'll end up making your app worse overall, I think.

00:28:38   And so being careful about not getting too worried.

00:28:41   It's like, well, what if someone's going on an epic expedition

00:28:45   and they're going to be not connected to the internet

00:28:47   for a week at a time, should they still be able to use the app?

00:28:51   It's like, well, maybe. I don't know.

00:28:54   It's like, if you have a really compelling reason, great.

00:28:57   But if you don't, don't over-engineer it,

00:28:59   just because you think you maybe should or could.

00:29:02   Right, because any of that engineering,

00:29:04   not only does it take a lot of time and energy

00:29:06   that you could be spending on features,

00:29:07   but it also introduces tons of bug potential.

00:29:10   So the least persistence you can do, the better.

00:29:14   And I think your example there is great.

00:29:16   It's like, if someone's going to be out away from connectivity for a while,

00:29:19   well, is your app really that useful to somebody

00:29:22   if everything in it is stale anyway?

00:29:24   It depends on the app.

00:29:25   If it's something like a podcast player, yes,

00:29:28   people can load up on a podcast and live off them for weeks.

00:29:30   But if it's something like a Twitter client, no,

00:29:32   that's not going to be useful to them.

00:29:34   So it depends on the app, but I think the answer for most apps

00:29:37   is probably you don't really need to worry about it.

00:29:39   Exactly.

00:29:40   All right, well, we're out of time this week.

00:29:42   Thank you, everybody, for listening, and we'll talk to you next week.

00:29:45   Bye!

00:29:46   [BLANK_AUDIO]