Developing Perspective

00:00:00 ◼   ► Hello and welcome to Developing Perspective.

00:00:02 ◼   ► Developing Perspective is a podcast discussing news of note in iOS development, Apple, and

00:00:06 ◼   ► the like.

00:00:07 ◼   ► I'm your host, David Smith.

00:00:08 ◼   ► I'm an independent iOS developer based in Herna, Virginia.

00:00:11 ◼   ► This is show number 179.

00:00:13 ◼   ► Today is Thursday, April 3rd.

00:00:16 ◼   ► Developing Perspective is never longer than 15 minutes, so let's get started.

00:00:19 ◼   ► All right, so I'm finally back from all of my travels and conference speaking, which

00:00:24 ◼   ► all I'd say went pretty well for some of my first times out.

00:00:28 ◼   ► But I'm glad to finally be able to button down in April, probably be focusing more on

00:00:33 ◼   ► building things rather than just talking about building things.

00:00:35 ◼   ► Which I guess is somewhat ironic, even that I'm starting off the month talking on a podcast.

00:00:40 ◼   ► But anyway, so unless you live under a rock or in under a rock that only has internet

00:00:46 ◼   ► such that you can download 15 minute podcasts, you probably heard that this week WWDC was

00:00:53 ◼   ► announced, which is a delightful break

00:00:56 ◼   ► from their typical pattern, or by typical,

00:00:58 ◼   ► I mean what they've done the last couple of years.

00:01:00 ◼   ► They announced in late April, where they've

00:01:02 ◼   ► announced now nice and early.

00:01:04 ◼   ► We have the dates.

00:01:05 ◼   ► It's the first week in June.

00:01:06 ◼   ► And they're doing something different this year.

00:01:08 ◼   ► They're distributing tickets through a kind

00:01:11 ◼   ► of more of a lottery, or a random selection,

00:01:13 ◼   ► I think is how Apple called it every time.

00:01:15 ◼   ► And I'm pretty excited about that.

00:01:17 ◼   ► The more I thought about it, the more it kind of became obvious

00:01:19 ◼   ► that this is where they were going to have to go,

00:01:22 ◼   ► that the old system that was largely based on either knowing

00:01:27 ◼   ► when tickets were going to be in terms of having alert systems

00:01:29 ◼   ► and things, then rushing and doing it,

00:01:31 ◼   ► what they tried last year with the stampede,

00:01:34 ◼   ► those are all kind of-- they just don't scale well.

00:01:37 ◼   ► And the only system that scales once you have too high--

00:01:41 ◼   ► way too high demand for the number of tickets

00:01:43 ◼   ► you have available, if you don't want to or can't increase

00:01:46 ◼   ► the number of tickets, is to go with something like a lottery.

00:01:49 ◼   ► And so that's what they did.

00:01:50 ◼   ► We'll all put our names in the hat.

00:01:53 ◼   ► You have until, I believe it is 10 AM on Monday to do it.

00:01:56 ◼   ► So if you have a chance between now and then

00:01:58 ◼   ► and you're wanting to try, by all means do.

00:02:02 ◼   ► But then through some process, randomly

00:02:05 ◼   ► select 5,000 names out of that hat.

00:02:09 ◼   ► And if you get one, great.

00:02:10 ◼   ► And if you don't, I'm sorry.

00:02:13 ◼   ► But by and large, I think this is a great improvement.

00:02:15 ◼   ► And hopefully it'll go well.

00:02:17 ◼   ► And this is where they'll be heading going forward.

00:02:20 ◼   ► would expect. And I think it's generally good because while no system is perfect,

00:02:24 ◼   ► there's no way that they can build, you know, build a structure for getting

00:02:29 ◼   ► tickets where everyone who wants a ticket will get one. That's impossible,

00:02:33 ◼   ► unless they're able to drive down interest in developing for Apple's platforms to such a

00:02:38 ◼   ► degree that there's no longer 5,000 people who want to do it, which seems very unlikely.

00:02:42 ◼   ► Or they would have to do something like, how would you determine who are the, you know,

00:02:49 ◼   ► the 5,000 correct people to get it,

00:02:50 ◼   ► in terms of for some definition of correctness.

00:02:52 ◼   ► Some people would say, like, oh, I want the old guard

00:02:54 ◼   ► to always get their tickets.

00:02:55 ◼   ► The most successful people should get their tickets.

00:02:57 ◼   ► The newest people should get their tickets

00:02:59 ◼   ► so we can build the platform.

00:03:00 ◼   ► The youngest, the oldest.

00:03:01 ◼   ► I mean, you could imagine whatever you want.

00:03:03 ◼   ► The reality is you just want to do it randomly.

00:03:05 ◼   ► You just want to give everybody a chance, an even chance,

00:03:08 ◼   ► an honest chance to get a ticket.

00:03:10 ◼   ► And then from there, just hope everyone

00:03:14 ◼   ► gets seen as sort of hope things--

00:03:16 ◼   ► people understand if they don't get one.

00:03:18 ◼   ► And on the flip side, I hope the people who get tickets, use them, enjoy them, take them

00:03:23 ◼   ► to full advantage.

00:03:25 ◼   ► I'll be out in San Francisco either way.

00:03:26 ◼   ► I've already booked my plane and hotel for the week.

00:03:29 ◼   ► I'll be out there either way because it's such a wonderful week.

00:03:33 ◼   ► Even if I can get a ticket, I'd still want to be out there to meet, sort of see the people

00:03:36 ◼   ► who I only typically see once a year at DubDub, as well as just being around.

00:03:42 ◼   ► It's a really kind of jazzing energy that is always kind of a motivating thing for the

00:03:45 ◼   ► middle of my year.

00:03:46 ◼   ► So that's kind of what I'm doing.

00:03:48 ◼   ► I said make sure you put your name in the hat if you want.

00:03:51 ◼   ► WVDC is really cool.

00:03:52 ◼   ► If you've never been, I highly recommend having the experience at least once.

00:03:57 ◼   ► You know, getting up early, waiting in line, going to some of the kind of special events

00:04:01 ◼   ► that there aren't videos for.

00:04:02 ◼   ► It's fun.

00:04:03 ◼   ► Obviously it's kind of a bit of an expensive vacation, but it's definitely useful and fun.

00:04:09 ◼   ► And I think there's something about it that is kind of a bit infectious.

00:04:14 ◼   ► And so that's kind of cool.

00:04:15 ◼   ► All right, so I'm going to be diving into the main topic.

00:04:18 ◼   ► I'm going to be talking for the rest of the show.

00:04:20 ◼   ► And it's a slight break for my App Store series.

00:04:23 ◼   ► I'll be picking that up.

00:04:24 ◼   ► I have a few more topics I want to talk up through.

00:04:26 ◼   ► But with W3C popping up today, I wanted

00:04:29 ◼   ► to make sure I had a full episode the next time I

00:04:31 ◼   ► did the App Store thing.

00:04:32 ◼   ► And so W3C took about a third of it.

00:04:34 ◼   ► So I was going to do something a little bit shorter.

00:04:36 ◼   ► And so it's going to be what I want to talk about,

00:04:38 ◼   ► is server hosting.

00:04:40 ◼   ► And this is going to be maybe a bit of a funny thing

00:04:42 ◼   ► to talk about.

00:04:42 ◼   ► But it really got prompted from a series of podcasts and conversations back and forth.

00:04:49 ◼   ► I think it got started with Justin Williams was talking about it, Bart Garment, Brent

00:04:53 ◼   ► Simmons, you know, we've done a lot of these back and forths about what level of abstraction

00:04:58 ◼   ► is the appropriate place to find for your own application.

00:05:02 ◼   ► So in the one extreme, obviously, you could build your back ends, you know, your and honestly,

00:05:07 ◼   ► it's probably even taken even step further and say, almost every utility application

00:05:11 ◼   ► these days uses some form of online backend.

00:05:14 ◼   ► Maybe if you're a game, you can get away with it.

00:05:16 ◼   ► But even then, you're probably using something like Game Center.

00:05:19 ◼   ► Very few apps now work well as just the app in itself

00:05:24 ◼   ► in complete isolation.

00:05:26 ◼   ► You're always interacting outwardly,

00:05:28 ◼   ► because that's often even what makes apps useful.

00:05:31 ◼   ► And so if you're going to interact with something,

00:05:34 ◼   ► you have to decide what you're going to interact with.

00:05:36 ◼   ► Are you going to, on the one end,

00:05:38 ◼   ► to kind of purchase very high level, very large building blocks, kind of building your

00:05:43 ◼   ► backend with Duplo.

00:05:44 ◼   ► You know, are you going to be sitting there and just kind of snapping some big blocks

00:05:48 ◼   ► together to build your backend?

00:05:50 ◼   ► You know, using things that are very simple, very well defined interfaces, kind of these

00:05:55 ◼   ► big blocks.

00:05:56 ◼   ► Or are you going to go farther and farther down the stack?

00:05:57 ◼   ► Maybe you're going to go from Duplo to Lego and then at some point you're going to be

00:06:00 ◼   ► building your own Lego, but you buy an injection molder to make it.

00:06:04 ◼   ► And then you're going to be buying the machine that's necessary to build the injection molder.

00:06:09 ◼   ► You could imagine if you brutalized that metaphor to go all the way from the Duplo all the way

00:06:13 ◼   ► down.

00:06:14 ◼   ► But at every level, you're making different choices.

00:06:18 ◼   ► And there's always trade-offs between the different levels as you're going.

00:06:22 ◼   ► And the reality is-- and I've worked at all of those levels.

00:06:27 ◼   ► And so I feel like I have some authority to talk about this.

00:06:32 ◼   ► But the reality is at every level, you're making trade-offs.

00:06:35 ◼   ► And you're just choosing which trade-offs

00:06:37 ◼   ► make the most sense for you and your product.

00:06:40 ◼   ► And those are going to differ for every different--

00:06:42 ◼   ► every project is going to have a different set of trade-offs

00:06:45 ◼   ► that make the most sense for it.

00:06:47 ◼   ► But here's some of the kind of things that I think about

00:06:49 ◼   ► that will hopefully-- maybe I thought

00:06:50 ◼   ► it would be helpful to kind of get you thinking as you're

00:06:52 ◼   ► trying to plan out your back end,

00:06:54 ◼   ► as you're trying to decide what makes sense for you.

00:06:56 ◼   ► First is to say, every abstraction choice

00:06:59 ◼   ► you make in your hosting stack is a leaky abstraction.

00:07:05 ◼   ► And by that I mean, if you take the core functionality,

00:07:08 ◼   ► say whatever it was, if the functionality is,

00:07:11 ◼   ► I want you to take this picture and I need to put it somewhere

00:07:15 ◼   ► and then retrieve it later, say it's a very simple operation

00:07:19 ◼   ► conceptually, not simple to implement,

00:07:21 ◼   ► but simple conceptually, I have these bytes

00:07:25 ◼   ► that I need to put somewhere and then get back.

00:07:28 ◼   ► Now I could use a service on the one end that

00:07:30 ◼   ► was very abstracted, that basically just said,

00:07:33 ◼   ► save picture, retrieve picture.

00:07:36 ◼   ► That's all it did.

00:07:37 ◼   ► It's very, very high level, very abstract.

00:07:40 ◼   ► And that's great.

00:07:42 ◼   ► Or I could go all the way to the other extent,

00:07:45 ◼   ► I could build a web application who

00:07:48 ◼   ► takes in some binary data, who then packages it up,

00:07:51 ◼   ► and then take that binary data and I distribute it out

00:07:54 ◼   ► to a content distribution network or whatever.

00:07:57 ◼   ► And you could build and imagine systems all the way

00:08:01 ◼   ► in between of those two things.

00:08:02 ◼   ► Like, from a very, very low level,

00:08:04 ◼   ► dealing with the raw bytes, putting them on disks,

00:08:07 ◼   ► to something that's just like a web endpoint that you

00:08:10 ◼   ► hit with a post method.

00:08:12 ◼   ► Like, you could very easily kind of go in either direction.

00:08:16 ◼   ► And understand that the very abstract version,

00:08:18 ◼   ► you know, the very high level version, it's nice.

00:08:22 ◼   ► It's easy to use.

00:08:22 ◼   ► You're just like, here's my thing.

00:08:24 ◼   ► I'm going to put it in there.

00:08:25 ◼   ► And that's usually fine.

00:08:28 ◼   ► But every now and then, little implementation details,

00:08:30 ◼   ► little other things that are being hidden away

00:08:32 ◼   ► are going to leak through.

00:08:33 ◼   ► And so you have to understand what those are likely.

00:08:36 ◼   ► What are those problems?

00:08:37 ◼   ► What are those issues that are going to come back to bite you

00:08:40 ◼   ► so that you can understand if those trade-offs make sense

00:08:44 ◼   ► to you and those trade-offs are worthwhile to you?

00:08:47 ◼   ► And inevitably, they will.

00:08:48 ◼   ► Inevitably, they'll be things or cases or situations

00:08:52 ◼   ► where you try using that backend in a way that it falls down.

00:08:57 ◼   ► Say, for example, you have a very, very large image.

00:09:00 ◼   ► You'd have a 10 gig image, and you try and send it

00:09:03 ◼   ► to your very, very high level thing,

00:09:05 ◼   ► and it's just the whole thing falls apart.

00:09:07 ◼   ► Well, that's why I did it.

00:09:08 ◼   ► Well, it's a leaky abstraction.

00:09:10 ◼   ► Maybe it just isn't built to handle that thing.

00:09:12 ◼   ► Or if you give it a very small file,

00:09:16 ◼   ► you give it a zero byte file, maybe weird things will happen.

00:09:20 ◼   ► Things that are just intrinsic in the way it was implemented

00:09:24 ◼   ► that you won't necessarily find out right away.

00:09:27 ◼   ► Now, obviously, those are contrived examples,

00:09:29 ◼   ► but hopefully they kind of get you

00:09:30 ◼   ► thinking of what I'm saying.

00:09:31 ◼   ► No matter how abstract the front end interface is,

00:09:34 ◼   ► there's always going to be back end implementation details that

00:09:37 ◼   ► are going to affect you.

00:09:38 ◼   ► So the first question you always have to ask yourself

00:09:40 ◼   ► when you're doing this is, what can you live with?

00:09:45 ◼   ► What are the things that you feel comfortable having

00:09:47 ◼   ► to-- the constraints and things that you're

00:09:49 ◼   ► have to place in your application as a result of making these choices.

00:09:54 ◼   ► And also, whatever choice you-- the deeper down in the stack you go, the more flexibility

00:10:01 ◼   ► you have, the more things that you can do, the fewer constraints you have overtly on

00:10:05 ◼   ► your application, that you can do whatever you want.

00:10:08 ◼   ► And that's awesome, and that's terrifying.

00:10:12 ◼   ► If you can do anything, that means you can do anything.

00:10:14 ◼   ► You can do really bad, horrible decisions that you then have to maintain and deal with

00:10:19 ◼   ► forever. Or you can build awesome solutions to not problems that nobody else can solve

00:10:24 ◼   ► because no one else has built the great back end. So you have to be thoughtful about what

00:10:29 ◼   ► constraints do you want to work in? How big of a sandbox do you want to get messy in?

00:10:36 ◼   ► Generally what I find, and this is kind of just a personality thing I think in a large

00:10:39 ◼   ► degree, is that the more and more I do this, the deeper into the stack I go. And that's

00:10:46 ◼   ► interesting because I suspect, I suspect there'll be a curve or a couple of curves, I guess,

00:10:51 ◼   ► into this where as I keep going, I'll go deeper and deeper into the stack. And at some point,

00:10:57 ◼   ► I'll likely sort of start coming back up again. And I think a lot of that has to do with the

00:11:03 ◼   ► role that learning plays in this. Because as a developer, I love learning new things. I love

00:11:08 ◼   ► learning new platforms and technologies. That's something that I just really enjoy. And so part

00:11:14 ◼   ► of what's made feed wrangler, for example, really interesting

00:11:17 ◼   ► from a personal level is that I'm learning

00:11:20 ◼   ► a lot of great new stuff.

00:11:21 ◼   ► I'm understanding now, feed wrangler

00:11:25 ◼   ► is built from pretty small pieces.

00:11:27 ◼   ► It uses Postgres, obviously, as a database, which is pretty big.

00:11:31 ◼   ► But exactly how that's deployed and managed and orchestrated

00:11:35 ◼   ► is all done by me.

00:11:36 ◼   ► I'm SSHing into servers and editing config files

00:11:38 ◼   ► and doing all this very low level stuff.

00:11:41 ◼   ► And some of it was out of necessity

00:11:44 ◼   ► because it's a very data-intensive application.

00:11:46 ◼   ► But a lot of it, too, is just because it's really interesting to learn.

00:11:49 ◼   ► And I can do things now that I wouldn't have been able to otherwise.

00:11:52 ◼   ► And the number of times things that I can learn from implementing a UI table

00:11:56 ◼   ► view again isn't as fun for me right now as it

00:11:59 ◼   ► is for doing this really cool server networking stuff.

00:12:03 ◼   ► And so that's kind of just really fun.

00:12:05 ◼   ► And even beyond just being fun, I mean, it's cool.

00:12:10 ◼   ► And it's weird to say, but it's just cool.

00:12:12 ◼   ► Like this last week, one of my main database servers

00:12:15 ◼   ► for Feed Wrangler was running out of hard drive space

00:12:18 ◼   ► because I kind of poorly planned it.

00:12:20 ◼   ► And so it's like, OK, I'll go and provision a new server,

00:12:23 ◼   ► have it physically built in my data center,

00:12:26 ◼   ► have that deployed out, set it up,

00:12:28 ◼   ► configure all the security and the permission stuff,

00:12:31 ◼   ► go and do all that, deploy Postgres onto it,

00:12:33 ◼   ► set it up from a backup into a live streaming binary copy

00:12:38 ◼   ► of the main primary master server that was starting to run out of space,

00:12:44 ◼   ► get it set up so that the two are perfectly in sync and running perfectly forward,

00:12:47 ◼   ► and then dynamically switch everything over to that being the new master,

00:12:52 ◼   ► which is crazy.

00:12:53 ◼   ► Like, the fact that I was able to-- and it actually worked.

00:12:57 ◼   ► It's a funny thing when you roll out a server change and everything goes well.

00:13:01 ◼   ► And that's almost scarier than things going poorly.

00:13:04 ◼   ► But I've done this now a couple times.

00:13:07 ◼   ► I know what I'm doing in a way that is really fun.

00:13:10 ◼   ► And so I kind of would be sad if I

00:13:12 ◼   ► was using a service that didn't-- I just hit a button

00:13:14 ◼   ► and says, like, expand database size.

00:13:17 ◼   ► Brrp!

00:13:18 ◼   ► Like, that'd be kind of sad.

00:13:19 ◼   ► I wouldn't be learning.

00:13:20 ◼   ► And so it's kind of fun.

00:13:23 ◼   ► Another reason I tend to go low level

00:13:25 ◼   ► is I enjoy being able to optimize things

00:13:28 ◼   ► for my application.

00:13:29 ◼   ► I can work through a lot of bottlenecks

00:13:31 ◼   ► that I wouldn't be able to if I didn't have access

00:13:34 ◼   ► to much lower level stuff.

00:13:35 ◼   ► things where I'm tweaking kernel values,

00:13:37 ◼   ► when I'm changing the memory configurations on Linux servers.

00:13:40 ◼   ► You can get really low level, and you can increase performance

00:13:43 ◼   ► in tangible ways that you just couldn't do otherwise.

00:13:46 ◼   ► It's pretty cool.

00:13:48 ◼   ► One of the things is the realization

00:13:50 ◼   ► that no matter what level I do it,

00:13:52 ◼   ► I'm going to have to monitor the system.

00:13:53 ◼   ► A lot of people say, well, I use an off-the-shelf system.

00:13:57 ◼   ► I use Azure.

00:13:57 ◼   ► I use AWS.

00:13:59 ◼   ► I use something higher level because I

00:14:01 ◼   ► want to have someone-- if something goes wrong,

00:14:03 ◼   ► I want someone else to be trying to fix it.

00:14:05 ◼   ► In some ways, that's better.

00:14:06 ◼   ► But the reality is, if it goes down, it goes down.

00:14:08 ◼   ► And I'm going to have to worry about that either way, which

00:14:11 ◼   ► kind of sucks, but it is how it is.

00:14:14 ◼   ► And lastly, it's often cheaper.

00:14:17 ◼   ► And really, that's the bottom line for an indie like me.

00:14:21 ◼   ► A lot of these services, I look at them like Azure.

00:14:24 ◼   ► And it's just expensive.

00:14:26 ◼   ► It's compared to me just running some raw Linux servers.

00:14:30 ◼   ► It just doesn't really scale.

00:14:32 ◼   ► And so you have to make that choice.

00:14:33 ◼   ► You have to think about your application.

00:14:35 ◼   ► Think what trade-offs you want to live with,

00:14:37 ◼   ► what abstractions are leaky that you'd be OK with.

00:14:40 ◼   ► And just find what works best for you.

00:14:42 ◼   ► All right, that's it for today's show.

00:14:43 ◼   ► As always, if you have questions, comments, concerns,

00:14:45 ◼   ► or complaints, I'm on Twitter @_davidsmith,

00:14:47 ◼   ► david@devlaucomperspective.com.

00:14:49 ◼   ► Have a good week.

00:14:49 ◼   ► Good luck on Monday.