Under the Radar

00:00:00 ◼   ► welcome to under the radar a show about

00:00:02 ◼   ► independent iOS app development I'm mark

00:00:04 ◼   ► Worman and I'm David Smith

00:00:05 ◼   ► under-the-radar is never longer than 30

00:00:07 ◼   ► minutes so let's get started this week

00:00:09 ◼   ► we wanted to talk about when you're

00:00:11 ◼   ► forced to do something for outside

00:00:15 ◼   ► reasons or market demand or pragmatic

00:00:18 ◼   ► reasons that may be that like maybe you

00:00:21 ◼   ► don't you wish you wouldn't have to do

00:00:23 ◼   ► or you think you shouldn't have to do

00:00:25 ◼   ► but you're forced to do it anyway and

00:00:28 ◼   ► this was brought about the reason I was

00:00:31 ◼   ► talked about this initially is that I I

00:00:33 ◼   ► had a number of things happen recently

00:00:34 ◼   ► with overcast that were of this nature

00:00:36 ◼   ► and where I had to basically adopt to

00:00:39 ◼   ► the way people actually are or the way

00:00:41 ◼   ► things actually work rather than hold on

00:00:44 ◼   ► to my you know long-standing you know

00:00:47 ◼   ► belief that it should be a different way

00:00:49 ◼   ► or my wish that it was a different way

00:00:52 ◼   ► because it would save me some work she's

00:00:54 ◼   ► often up in the case yeah the first the

00:00:58 ◼   ► the big one that kind of drove all this

00:01:00 ◼   ► was overcast recently had to implement

00:01:04 ◼   ► feed redirect support and/or permanent

00:01:07 ◼   ► feed change support and I talked about

00:01:09 ◼   ► this a little bit on ATP last week so

00:01:11 ◼   ► I'm not gonna go too far into it but the

00:01:13 ◼   ► basic idea here is that podcast feeds

00:01:17 ◼   ► change over time and podcasters when

00:01:22 ◼   ► they the podcast is often will move the

00:01:24 ◼   ► feed to a different service or to a

00:01:26 ◼   ► different host or tweeted for an ad

00:01:27 ◼   ► tracking platform or whatever else they

00:01:30 ◼   ► just move the feet and and they don't

00:01:31 ◼   ► like if you move a website you usually

00:01:34 ◼   ► set up a redirect from the old URL to

00:01:36 ◼   ► the new URLs and you usually leave that

00:01:38 ◼   ► redirect there forever and even if you

00:01:41 ◼   ► give it the the permanent redirect code

00:01:43 ◼   ► of 301 rather than the temporary one of

00:01:45 ◼   ► 302 you still leave up the redirect

00:01:48 ◼   ► forever because you know that like

00:01:50 ◼   ► there's old links the point to it

00:01:52 ◼   ► there's old search engine ranking that

00:01:54 ◼   ► might point to it and so you you know

00:01:56 ◼   ► that like it's a good practice on the

00:01:57 ◼   ► web to to basically make redirects work

00:02:00 ◼   ► forever so if you're writing something

00:02:02 ◼   ► that that keeps URLs in a database and

00:02:05 ◼   ► and crawls them every so often on the

00:02:08 ◼   ► web you can pretty much never process

00:02:11 ◼   ► permanent redirect codes and be fine

00:02:13 ◼   ► because almost all the redirects will

00:02:16 ◼   ► keep working indefinitely into the

00:02:18 ◼   ► future

00:02:19 ◼   ► and when I made overcast this is how I

00:02:21 ◼   ► set up the database and and this was you

00:02:23 ◼   ► know I mean I made the database in 2013

00:02:25 ◼   ► that's when I made the schema I started

00:02:27 ◼   ► crawling and and started building up my

00:02:30 ◼   ► database of feeds and and the app

00:02:31 ◼   ► launched about a year later and so for

00:02:34 ◼   ► me at the time I made this decision of

00:02:37 ◼   ► how to structure my feeds table and my

00:02:39 ◼   ► feed items table and and how users

00:02:42 ◼   ► interact with feeds and feed items I

00:02:44 ◼   ► designed this whole schema back then and

00:02:46 ◼   ► there was one big flaw in the schema

00:02:49 ◼   ► which was that feeds couldn't have their

00:02:52 ◼   ► URLs changed if a if a feed was created

00:02:56 ◼   ► with a new URL it would be a whole new

00:02:58 ◼   ► feed entry and then there was no

00:03:00 ◼   ► mechanism in place to do things like you

00:03:03 ◼   ► know if if a new URL is created and in

00:03:05 ◼   ► the and then one of the old ones

00:03:06 ◼   ► redirects to that one how do I can I

00:03:09 ◼   ► merge those entries do I copy the old

00:03:12 ◼   ► one to the new one do what does the newt

00:03:14 ◼   ► does the old one have some kind of an

00:03:15 ◼   ► alias list that also points to the new

00:03:17 ◼   ► one I had no setup like that in place

00:03:20 ◼   ► and so the entire app and everything and

00:03:22 ◼   ► the entire database were built up

00:03:23 ◼   ► without that consideration or without

00:03:25 ◼   ► that possibility and this started to

00:03:28 ◼   ► become a problem pretty soon after

00:03:31 ◼   ► launch that a couple of webs a couple of

00:03:34 ◼   ► podcaster at switching their feed URLs

00:03:36 ◼   ► and for awhile I thought well you know

00:03:38 ◼   ► people just subscribe to the new one the

00:03:40 ◼   ► old one will still work forever because

00:03:41 ◼   ► it'll redirect and it'll be fine there

00:03:43 ◼   ► were a number of problems with the set

00:03:44 ◼   ► up in reality one of the big ones is

00:03:46 ◼   ► that when I would do things like count

00:03:49 ◼   ► subscribers to do things like search

00:03:51 ◼   ► ranking the two different entries would

00:03:54 ◼   ► count as independent feeds I would have

00:03:57 ◼   ► to do search filtering so things like

00:03:59 ◼   ► you know I filter I don't show search

00:04:00 ◼   ► results that don't have an iTunes ID

00:04:02 ◼   ► associated so when the iTunes ID would

00:04:04 ◼   ► move it would basically lose the search

00:04:08 ◼   ► ranking and the entire history of the

00:04:09 ◼   ► previous one etc so there were all sorts

00:04:11 ◼   ► of problems of that but it wasn't a big

00:04:13 ◼   ► problem because not a lot of podcasts

00:04:16 ◼   ► were moving their feeds in 2014 and then

00:04:19 ◼   ► over the following years a lot of

00:04:23 ◼   ► podcasts switched from HTTP to HTTPS

00:04:27 ◼   ► and because a lot of websites did as we

00:04:30 ◼   ► all kind of learn with that would we

00:04:31 ◼   ► probably should be doing that that

00:04:33 ◼   ► created a whole lot of duplicate entries

00:04:36 ◼   ► additionally during that time a certain

00:04:39 ◼   ► podcast networks shut down or moved or

00:04:42 ◼   ► changed their names change their domains

00:04:43 ◼   ► and that caused even more extra or

00:04:46 ◼   ► removed entries and then the big one was

00:04:50 ◼   ► over the last year or two a lot of major

00:04:54 ◼   ► podcast publishers have switched to

00:04:57 ◼   ► dynamic ad insertion platforms which

00:04:59 ◼   ► hosts the feet of their own our URL for

00:05:01 ◼   ► who knows what reason and so tons of

00:05:05 ◼   ► major podcasts are now switching their

00:05:07 ◼   ► feeds if they didn't already do it with

00:05:09 ◼   ► HTTPS like two years ago in the last

00:05:12 ◼   ► year they're now doing it with ad

00:05:13 ◼   ► tracking platforms so I've had a like it

00:05:17 ◼   ► basically went from a small problem in

00:05:19 ◼   ► year one to a medium-size problem in

00:05:22 ◼   ► year two and a really big problem in

00:05:24 ◼   ► year three and so I finally had to

00:05:27 ◼   ► implement feed redirects and I had you

00:05:30 ◼   ► know all this time I've been thinking

00:05:32 ◼   ► like I can just follow redirects I can

00:05:34 ◼   ► feel like they'll be fine they'll be

00:05:35 ◼   ► they'll be good forever and the reality

00:05:38 ◼   ► was just different the reality was

00:05:39 ◼   ► podcasters don't do things the way web

00:05:41 ◼   ► web developers do things they break

00:05:43 ◼   ► redirects like they consider iTunes the

00:05:45 ◼   ► authoritative source and if iTunes moves

00:05:48 ◼   ► the feed for them which they can do

00:05:50 ◼   ► through multiple different means either

00:05:52 ◼   ► a 301 or putting a special XML tag in

00:05:56 ◼   ► the feed that was returned with a 200

00:05:57 ◼   ► response or going through like an iTunes

00:06:00 ◼   ► customer support representative learning

00:06:02 ◼   ► the podcast directory and like just like

00:06:04 ◼   ► emailing somebody and saying sorry we

00:06:05 ◼   ► messed up our feet can you fix it and

00:06:07 ◼   ► none of those things would carry over to

00:06:09 ◼   ► overcast so the reality of the market

00:06:12 ◼   ► was very very different from how I

00:06:13 ◼   ► wanted it to be I really badly wanted it

00:06:16 ◼   ► to work in this one way that websites

00:06:18 ◼   ► always work which is like you know set

00:06:19 ◼   ► up a redirect and keep it up forever but

00:06:22 ◼   ► the reality was very different the

00:06:23 ◼   ► reality was podcasters had to move their

00:06:24 ◼   ► feeds frequently they would break the

00:06:26 ◼   ► old URLs and redirects most of the time

00:06:29 ◼   ► and even if they didn't I still had this

00:06:31 ◼   ► problem of like split feed in the

00:06:34 ◼   ► database so that's no good so finally

00:06:37 ◼   ► over the last couple of weeks I finally

00:06:39 ◼   ► wrote redirect support and be

00:06:41 ◼   ► cause of my database schema being so

00:06:42 ◼   ► unfriendly to it it's a huge hack and

00:06:45 ◼   ► it's not perfect the way I do it is that

00:06:48 ◼   ► basically like copy your old entries

00:06:51 ◼   ► onto the new feed and delete your old

00:06:53 ◼   ► ones and that's not great like there's a

00:06:56 ◼   ► whole lot of downsides to that one of

00:06:57 ◼   ► the big ones is that the app seeing a

00:07:00 ◼   ► new feed item ID for all these episodes

00:07:02 ◼   ► the APRI downloads them all that's no

00:07:05 ◼   ► good either so you like you it maintains

00:07:08 ◼   ► your history but you have to download

00:07:09 ◼   ► like all of a sudden your phone is

00:07:11 ◼   ► downloading like all your saved episodes

00:07:13 ◼   ► of a certain show that you hadn't

00:07:14 ◼   ► listened to yet and that's kind of weird

00:07:16 ◼   ► to cut to happen unexpectedly so that's

00:07:19 ◼   ► not a great solution but I had to be

00:07:22 ◼   ► pragmatic I had to address this problem

00:07:25 ◼   ► somehow because the market was very

00:07:29 ◼   ► different than what I wanted it to be

00:07:31 ◼   ► yeah and I I think I also hear in what

00:07:35 ◼   ► you're saying it's this thing that is so

00:07:37 ◼   ► happens to me this happened to me so

00:07:39 ◼   ► many times is it's as much as we would

00:07:42 ◼   ► like I think we would like to be able to

00:07:44 ◼   ► predict the future and when we're

00:07:46 ◼   ► initially building our applications and

00:07:49 ◼   ► we're designing them and we're

00:07:50 ◼   ► structuring them we would make choices

00:07:53 ◼   ► that you know our future selves will

00:07:56 ◼   ► thank us for right and conceptually that

00:08:00 ◼   ► sounds great that okay yeah of course

00:08:01 ◼   ► like I should engine you I should struct

00:08:04 ◼   ► structure these things you know with

00:08:05 ◼   ► maximum flexibility and I should be able

00:08:07 ◼   ► to I should a head of time I should

00:08:09 ◼   ► consider all these possibilities the

00:08:12 ◼   ► reality is I've done this enough to know

00:08:14 ◼   ► that you can't like that is impossible

00:08:18 ◼   ► and things will change and break and

00:08:21 ◼   ► become problems in ways that you could

00:08:24 ◼   ► never have predicted maybe you get

00:08:27 ◼   ► slightly better at this you know maybe

00:08:28 ◼   ► you don't make the same mistake twice

00:08:29 ◼   ► like those types of things may be

00:08:32 ◼   ► improving but I feel like the situation

00:08:34 ◼   ► you find yourself in is something that

00:08:36 ◼   ► is universal to app development or just

00:08:38 ◼   ► you know software engineering in general

00:08:40 ◼   ► that you will at some point make be

00:08:44 ◼   ► forced to either you know you either

00:08:45 ◼   ► have to make an assumption or you have

00:08:48 ◼   ► to explode complexity like those are the

00:08:51 ◼   ► two choice one of those two things you

00:08:53 ◼   ► have to do at

00:08:54 ◼   ► hundreds of points in your application

00:08:57 ◼   ► as you're developing it you either have

00:08:59 ◼   ► so you can make an assumption you can

00:09:00 ◼   ► say in this case you're going to assume

00:09:02 ◼   ► you know that mapping a feed URL to the

00:09:06 ◼   ► fee a ID directly will be a stable link

00:09:09 ◼   ► or you have to explode complexity and

00:09:12 ◼   ► like you had build you'd have to do some

00:09:14 ◼   ► very complicated things to make that

00:09:17 ◼   ► work you know flexibly and in this case

00:09:20 ◼   ► you know you chose to make the to start

00:09:22 ◼   ► with the assumption but like that's the

00:09:23 ◼   ► same thing that we do all the time

00:09:25 ◼   ► there's so much so much of you know soft

00:09:27 ◼   ► development is about getting good at

00:09:31 ◼   ► making that decision between making an

00:09:33 ◼   ► assumption and expanding complexity and

00:09:35 ◼   ► you could apply this stuff so many thing

00:09:37 ◼   ► it's like how complicated is your class

00:09:38 ◼   ► hierarchy you know what do kind of

00:09:41 ◼   ► choices do you do for data no data

00:09:43 ◼   ► persistence or or so many different

00:09:45 ◼   ► things come down to this base this very

00:09:47 ◼   ► simple situation or do you are you going

00:09:49 ◼   ► to make an assumption or are you going

00:09:51 ◼   ► to explode complexity and like and that

00:09:55 ◼   ► means that you're stuck like now you

00:09:56 ◼   ► have to go and do be forced to do this

00:09:58 ◼   ► thing because your assumption proved to

00:10:01 ◼   ► be wrong or proved to be in wrong enough

00:10:04 ◼   ► that continuing with that assumption

00:10:07 ◼   ► becomes problematic and it's so much

00:10:10 ◼   ► more complex to is like like as I've

00:10:12 ◼   ► mentioned this on Twitter here and there

00:10:13 ◼   ► the last couple of weeks I've gotten a

00:10:15 ◼   ► number of people who are suggesting like

00:10:17 ◼   ► oh why don't you why didn't you just do

00:10:18 ◼   ► this with your database or why didn't

00:10:19 ◼   ► you just make this migration happen and

00:10:21 ◼   ► a lot of them a lot of the the ideas or

00:10:24 ◼   ► suggestions don't really work or aren't

00:10:26 ◼   ► as simple as they sound

00:10:27 ◼   ► because overcast is not just a local app

00:10:29 ◼   ► with a local database it is a service

00:10:32 ◼   ► with a server back-end that syncs

00:10:34 ◼   ► between multiple copies of your app on

00:10:35 ◼   ► multiple devices so any kind of like

00:10:38 ◼   ► one-way migration either can't happen or

00:10:41 ◼   ► is way more complex than you would think

00:10:43 ◼   ► and also because it is a server back tap

00:10:46 ◼   ► the the complexity of the schema has a

00:10:49 ◼   ► direct relationship to how scalable it

00:10:52 ◼   ► is and how efficient it is and how

00:10:54 ◼   ► expensive it is to host you know if if

00:10:57 ◼   ► every look up to the feeds table has to

00:11:00 ◼   ► then make another look up to some kind

00:11:02 ◼   ► of like URL aliases table or something

00:11:05 ◼   ► like that like to to map the URL to the

00:11:07 ◼   ► feeds arm fetching

00:11:08 ◼   ► that's a pretty significant hit in

00:11:10 ◼   ► performance that's that's much more

00:11:11 ◼   ► complexity and that could be way more

00:11:14 ◼   ► load on the database servers and that

00:11:15 ◼   ► could really make scaling more difficult

00:11:17 ◼   ► and if you have multi if you have many

00:11:19 ◼   ► of those things in your schema for

00:11:21 ◼   ► flexibility and everything then you

00:11:24 ◼   ► start running into pretty big scaling

00:11:26 ◼   ► problems and big challenges that either

00:11:28 ◼   ► just make your life hard or make things

00:11:30 ◼   ► cost way more to hosts and that could

00:11:32 ◼   ► break your business model so like

00:11:33 ◼   ► there's there's all sorts of other

00:11:35 ◼   ► considerations here that make things

00:11:37 ◼   ► that this sound like a hard problem

00:11:38 ◼   ► anyway so that was you know one problem

00:11:42 ◼   ► of many that I've had and I kind of

00:11:45 ◼   ► wanted to talk for the show about a few

00:11:47 ◼   ► more of those but before we do speaking

00:11:50 ◼   ► of scaling your web servers we were a

00:11:52 ◼   ► sponsor this week by Linode Linode has

00:11:55 ◼   ► fast powerful web hosting options that

00:11:57 ◼   ► you can get set up in just seconds it's

00:11:59 ◼   ► very very easy to understand other tools

00:12:00 ◼   ► and they could let you choose the

00:12:02 ◼   ► resources and the next distribution that

00:12:04 ◼   ► you want giving you the power and

00:12:05 ◼   ► flexibility that you need all this

00:12:06 ◼   ► starts at just five dollars a month that

00:12:08 ◼   ► gets you a server with one gigabyte or

00:12:13 ◼   ► very very fast if industry-leading

00:12:15 ◼   ► performance they have native SSD storage

00:12:17 ◼   ► and all their servers access to a 200

00:12:20 ◼   ► gigabit network eat intel xeon e5

00:12:23 ◼   ► processors my favorite line of

00:12:24 ◼   ► processors these are the fastest

00:12:26 ◼   ► processors in the cloud market leader

00:12:28 ◼   ► has over 400,000 customers who are all

00:12:31 ◼   ► serviced by the friendly 24/7 support

00:12:34 ◼   ► team you can email them if you need any

00:12:36 ◼   ► help you can chat over IRC if you need

00:12:37 ◼   ► any help and they have all sorts of

00:12:39 ◼   ► wonderful documentation that's all

00:12:41 ◼   ► public you can see it right now you and

00:12:43 ◼   ► in fact a lot of times if you search for

00:12:45 ◼   ► Linux server administration topics a lot

00:12:47 ◼   ► of times the search results you get from

00:12:48 ◼   ► Google are just Linode support articles

00:12:50 ◼   ► because they are so easy to use to

00:12:52 ◼   ► understand and so helpful so litterin

00:12:54 ◼   ► has fantastic pricing options available

00:12:56 ◼   ► again you can get a server with one

00:12:57 ◼   ► gigabyte us five bucks a month you can

00:13:00 ◼   ► get a server with 16 gigs of ram for

00:13:02 ◼   ► just 60 bucks a month and a whole lot of

00:13:04 ◼   ► levels above and below it in between

00:13:05 ◼   ► across the board this is twice the

00:13:07 ◼   ► amount of RAM that you'll get at most

00:13:08 ◼   ► other places as a listener of the show

00:13:10 ◼   ► if you sign up at lynda.com a slash of

00:13:12 ◼   ► radar you'll be supporting us and you

00:13:14 ◼   ► look at $20 towards any Linode plan and

00:13:16 ◼   ► with a 7 day money-back guarantee

00:13:18 ◼   ► there's nothing to lose so go to

00:13:20 ◼   ► lynda.com slash

00:13:22 ◼   ► rate

00:13:22 ◼   ► are to learn more sign up and take

00:13:24 ◼   ► advantage about $20 credit or use promo

00:13:25 ◼   ► code radar 2017 at checkout thank you so

00:13:29 ◼   ► much to Linode for running all of my

00:13:30 ◼   ► servers and supporting the show so

00:13:33 ◼   ► another one I had with overcast a small

00:13:36 ◼   ► one before I go to another big one is

00:13:38 ◼   ► password protected feeds this is one

00:13:42 ◼   ► that you know podcasts sometimes we'll

00:13:44 ◼   ► have paid memberships or other kind of

00:13:47 ◼   ► like you know meant private feeds that

00:13:48 ◼   ► they protect with some kind of password

00:13:50 ◼   ► authentication scheme usually you know

00:13:52 ◼   ► HTTP basic auth it's compatible with

00:13:54 ◼   ► with a lot of apps and this becomes a

00:13:57 ◼   ► problem when you have a server-side

00:13:58 ◼   ► crawling like overcast does and in a

00:14:00 ◼   ► couple of a couple of competitors do

00:14:01 ◼   ► because it the question becomes like

00:14:04 ◼   ► where do you do you retain the password

00:14:07 ◼   ► and the entry on the server side and

00:14:09 ◼   ► then do you crawl with someone's

00:14:10 ◼   ► password do you how do you protect that

00:14:13 ◼   ► from being used by multiple people and

00:14:16 ◼   ► then if you have a whole lot of people

00:14:18 ◼   ► who all have separate usernames and

00:14:20 ◼   ► passwords they're calling the feed with

00:14:21 ◼   ► do you have to then crawl the feed from

00:14:22 ◼   ► like you know a hundred thousand

00:14:24 ◼   ► different users from like hundred

00:14:26 ◼   ► thousand different copies of the feed

00:14:27 ◼   ► like you have to keep all those in your

00:14:29 ◼   ► database and have those clogging up your

00:14:31 ◼   ► crawler so it's kind of a big problem so

00:14:33 ◼   ► when I launched overcast I made a

00:14:35 ◼   ► decision I'm just not gonna support

00:14:37 ◼   ► those and I this was kind of like a

00:14:39 ◼   ► political decision like I don't like

00:14:41 ◼   ► that I have to support these so I'm just

00:14:43 ◼   ► gonna see if I can not support them and

00:14:45 ◼   ► and just hope that nobody uses password

00:14:48 ◼   ► feeds anymore after a while and that

00:14:51 ◼   ► turned out not to be the case at all it

00:14:52 ◼   ► turned out that password feeds are

00:14:55 ◼   ► getting more and more popular as more

00:14:58 ◼   ► and more podcasts exist and many of them

00:15:00 ◼   ► choose to have paid tiers or private

00:15:02 ◼   ► feeds or something like that and what a

00:15:05 ◼   ► lot of people figured out also is that

00:15:07 ◼   ► when I switch my crawlers to use go when

00:15:11 ◼   ► I switch to it to a go offered crawler

00:15:12 ◼   ► goes URL library if you supply it a URL

00:15:18 ◼   ► that has the username and password

00:15:19 ◼   ► prefixed on it the way you could do like

00:15:21 ◼   ► username colon password at hostname it

00:15:24 ◼   ► that will just work correctly in the go

00:15:28 ◼   ► crawling library and people figured this

00:15:30 ◼   ► out about overcast a couple years ago

00:15:31 ◼   ► and so people have been adding user name

00:15:34 ◼   ► and password feeds to overcast

00:15:36 ◼   ► anyway and just doing it you know just

00:15:38 ◼   ► like the nerds who figure that out no to

00:15:40 ◼   ► just add that manually as a URL and so

00:15:43 ◼   ► I'm already now having my database

00:15:45 ◼   ► beefed look with lots of these copies of

00:15:47 ◼   ► these things and and lots of different

00:15:49 ◼   ► feeds and and the good thing is that

00:15:50 ◼   ► like you know concerns like whether it's

00:15:53 ◼   ► you know kept private or not that just

00:15:55 ◼   ► kind of works with overcast because I

00:15:56 ◼   ► filter by weather feeds have iTunes IDs

00:15:59 ◼   ► and if they don't have iTunes IDs they

00:16:01 ◼   ► don't show up in search or in many other

00:16:03 ◼   ► places and so these copies of these

00:16:07 ◼   ► private fees that have these using the

00:16:08 ◼   ► passwords prefixed on them don't have

00:16:10 ◼   ► iTunes entries so they don't show up and

00:16:12 ◼   ► search so that that part ends up working

00:16:14 ◼   ► but I am seeing my database slowly

00:16:16 ◼   ► filled with these anyway and now there's

00:16:19 ◼   ► so many of them that if I would like

00:16:21 ◼   ► close that hole and make those not work

00:16:23 ◼   ► anymore that would be a pretty big

00:16:25 ◼   ► problem for me I have a lot of very

00:16:27 ◼   ► angry people and that would probably be

00:16:29 ◼   ► like you know a pretty nasty move so I'm

00:16:31 ◼   ► not gonna break these now instead of

00:16:34 ◼   ► decided you know what actually the right

00:16:36 ◼   ► move here is to just make an interface

00:16:39 ◼   ► like on the add URL screen

00:16:42 ◼   ► I should just detect whether it's given

00:16:45 ◼   ► me a 401 response and prompt people to

00:16:48 ◼   ► enter a username and password and just

00:16:50 ◼   ► store that for them in the URL for the

00:16:52 ◼   ► feed because again this is like

00:16:54 ◼   ► pragmatic versus ideal like in the ideal

00:16:57 ◼   ► world I shouldn't have to deal with

00:16:58 ◼   ► these in the real world turns out I do

00:17:01 ◼   ► and so I might as well make it pleasant

00:17:04 ◼   ► for people who use them and I might as

00:17:06 ◼   ► well and and you know the reality is I

00:17:08 ◼   ► just have to design my crawlers and

00:17:10 ◼   ► scale my crawlers in such a way that

00:17:13 ◼   ► having a whole bunch of these feeds in

00:17:15 ◼   ► the database is not going to be a

00:17:17 ◼   ► problem and that's just the reality

00:17:19 ◼   ► because the reality is you have to do

00:17:22 ◼   ► like it's really hard to impose your

00:17:26 ◼   ► will on your users in that way like if

00:17:29 ◼   ► people want to you want to do this thing

00:17:31 ◼   ► and they want to use your app to do it

00:17:32 ◼   ► they're going to do it mm-hmm like it

00:17:35 ◼   ► would doesn't really get you very far if

00:17:38 ◼   ► like when someone put like the

00:17:39 ◼   ► alternative I suppose is like if someone

00:17:41 ◼   ► went to the ad URL screen and you

00:17:43 ◼   ► detected they there was a you know a

00:17:45 ◼   ► username and password prefix to the URL

00:17:47 ◼   ► you know you could throw up a big nasty

00:17:49 ◼   ► mess

00:17:49 ◼   ► a chance a no no no this is not what we

00:17:52 ◼   ► do and it was with a wagging finger bad

00:17:56 ◼   ► URL that's really that otherwise this is

00:17:58 ◼   ► going to happen like people are going to

00:18:00 ◼   ► realize it is happening yeah and if

00:18:02 ◼   ► anything you could you view it it's like

00:18:04 ◼   ► okay well you were able to without

00:18:07 ◼   ► without really doing any extra work you

00:18:09 ◼   ► were able to see how big of a market and

00:18:11 ◼   ► need and interest there is in this so at

00:18:15 ◼   ► least it's a slightly positive in that

00:18:17 ◼   ► sense like rather than building this

00:18:18 ◼   ► feature ahead of time you're able to

00:18:19 ◼   ► build it now knowing that it will be

00:18:22 ◼   ► used that you know roughly how many

00:18:24 ◼   ► people are going to use it and then you

00:18:25 ◼   ► can you know make changes to the way you

00:18:28 ◼   ► do crawling or the way that you manage

00:18:30 ◼   ► these things accordingly to take

00:18:32 ◼   ► advantage of it and so like it's a good

00:18:34 ◼   ► thing in that sense but yeah like you

00:18:36 ◼   ► can't

00:18:36 ◼   ► I mean you this is this is a kind of

00:18:38 ◼   ► thing that just sort of always happens I

00:18:39 ◼   ► find that you know you expect people to

00:18:42 ◼   ► be using your app in one way and so

00:18:44 ◼   ► often it is the way that you want to use

00:18:47 ◼   ► it or the way that you you know you have

00:18:48 ◼   ► it in your mind and then you get a

00:18:50 ◼   ► support request you get an email from

00:18:51 ◼   ► somebody and they show you something

00:18:54 ◼   ► about how they were using your app that

00:18:56 ◼   ► can kind of sum it's like it's like

00:18:57 ◼   ► you're doing what exactly what woke okay

00:19:02 ◼   ► like and sometimes I see those things

00:19:05 ◼   ► and it's like that's genius like that's

00:19:07 ◼   ► amazing I need to you know like

00:19:08 ◼   ► percolate this up into this actual

00:19:10 ◼   ► proper feature and sometimes you're just

00:19:12 ◼   ► like um I'm amazed that works it's not

00:19:17 ◼   ► intended to work I don't know if it will

00:19:20 ◼   ► work in the future

00:19:20 ◼   ► good luck but like it is just one of

00:19:24 ◼   ► those things that you know users are

00:19:26 ◼   ► shockingly creative about with the ways

00:19:28 ◼   ► that they'll make something work

00:19:29 ◼   ► especially I mean inviting thing it's a

00:19:31 ◼   ► compliment to Overcash that people even

00:19:33 ◼   ► though this isn't a feature of the app

00:19:35 ◼   ► they took the effort and did the work to

00:19:37 ◼   ► make it you know to somehow make it work

00:19:40 ◼   ► because they want to use the app anyway

00:19:42 ◼   ► that they don't want to switch to

00:19:43 ◼   ► another app that mase you know support

00:19:45 ◼   ► this at a at a official level because

00:19:49 ◼   ► that you know they like the app and they

00:19:51 ◼   ► want to stay with it anyway and my final

00:19:54 ◼   ► example here that I wanted to talk about

00:19:55 ◼   ► today was when the iPhone 10 came out it

00:19:59 ◼   ► you know it has this new OLED screen and

00:20:01 ◼   ► it turns out that completely black

00:20:03 ◼   ► looks really good on an OLED screen the

00:20:06 ◼   ► problem is that completely black doesn't

00:20:08 ◼   ► look so good if you have like a totally

00:20:11 ◼   ► black background and you know other

00:20:13 ◼   ► elements that are like you know lighter

00:20:15 ◼   ► and go totally black does not look as

00:20:16 ◼   ► good on LCD screens the screens that

00:20:19 ◼   ► we've had an iPhone since the first

00:20:20 ◼   ► iPhone before this but true black looks

00:20:23 ◼   ► really good on OLED and and all those

00:20:25 ◼   ► like dark Gray's that we've been using

00:20:28 ◼   ► on LCDs because that looks better there

00:20:30 ◼   ► don't look good on the iPhone 10 so what

00:20:34 ◼   ► as soon as Efrain 10 came out there was

00:20:35 ◼   ► immediate demand from users who want all

00:20:38 ◼   ► their apps to have true black themes

00:20:41 ◼   ► available designers haven't been

00:20:43 ◼   ► accustomed to LCD screens for the last

00:20:45 ◼   ► forever designers have all have all

00:20:47 ◼   ► basically said like you know looked it

00:20:49 ◼   ► doesn't look as good it's hard and it is

00:20:51 ◼   ► harder in many ways like there's certain

00:20:53 ◼   ► certain design themes and standards that

00:20:56 ◼   ► just either don't work or have to be

00:20:57 ◼   ► severely modified to work with a true

00:20:59 ◼   ► black background and design ocean argue

00:21:03 ◼   ► like maybe they don't think it looks as

00:21:05 ◼   ► good but the reality is a lot of people

00:21:09 ◼   ► think it looks good and a lot of people

00:21:10 ◼   ► want it and one of the things they

00:21:12 ◼   ► started doing when the iPhone 10 shipped

00:21:15 ◼   ► I got a massive spike in requests for

00:21:18 ◼   ► supporting the accessibility smart

00:21:20 ◼   ► invert colors feature because what had

00:21:23 ◼   ► happened was people seeking more dark

00:21:26 ◼   ► themes in their apps with iOS apps that

00:21:28 ◼   ► so often are like bright white they

00:21:30 ◼   ► there have been numerous articles posted

00:21:32 ◼   ► online and telling people hey neat trick

00:21:34 ◼   ► you can turn on this smarter invert

00:21:37 ◼   ► colors feature in accessibility settings

00:21:38 ◼   ► and all your apps turn black and so they

00:21:41 ◼   ► were doing that and overcast didn't

00:21:43 ◼   ► support it correctly with like it like

00:21:44 ◼   ► it was overcast was not excluding the

00:21:46 ◼   ► images from being inverted so all the

00:21:48 ◼   ► artwork would look weird so I got this

00:21:49 ◼   ► huge spike in requests when the iPhone

00:21:52 ◼   ► 10 came out to support smart invert

00:21:55 ◼   ► colors the reality was these were not

00:21:57 ◼   ► people who needed this for accessibility

00:21:59 ◼   ► reasons for the most part they were

00:22:01 ◼   ► people who just wanted a black theme and

00:22:03 ◼   ► because it looks so it looks so good on

00:22:05 ◼   ► their on their iPhone 10 and it's better

00:22:07 ◼   ► on the battery and stuff so even though

00:22:10 ◼   ► I thought you know I should probably

00:22:11 ◼   ► like you know true black I thought

00:22:13 ◼   ► looked kind of you know it's it's it's a

00:22:15 ◼   ► pretty bold look when you first see it

00:22:17 ◼   ► I now think it looks good but it took me

00:22:20 ◼   ► a few hours before I thought it looked

00:22:21 ◼   ► good after I started using it but the

00:22:25 ◼   ► reality was the market was making their

00:22:27 ◼   ► own black-themed even if I wasn't like

00:22:29 ◼   ► they would just turn on my white theme

00:22:30 ◼   ► and turn on smart invert colors and

00:22:32 ◼   ► accessibility and they were making a

00:22:34 ◼   ► black theme so the correct thing for me

00:22:37 ◼   ► to do which I did in the next update was

00:22:39 ◼   ► fix my spar time for colour support to

00:22:41 ◼   ► make the images look correct and then

00:22:43 ◼   ► also add a black theme because like

00:22:46 ◼   ► that's the reality no matter what you

00:22:48 ◼   ► think of black themes everyone now wants

00:22:52 ◼   ► them and if you don't give them one they

00:22:54 ◼   ► will work around that to get themselves

00:22:57 ◼   ► one with accessibility settings and that

00:23:00 ◼   ► makes a lot of the system look really

00:23:01 ◼   ► weird and it's over like if you don't

00:23:04 ◼   ► need invert colors as an accessibility

00:23:06 ◼   ► option you shouldn't be using it there's

00:23:07 ◼   ► lots of weird side effects to it but

00:23:09 ◼   ► people wanted the black themes so badly

00:23:11 ◼   ► that they did that so again it's one of

00:23:14 ◼   ► those situations where like designers

00:23:16 ◼   ► thought you shouldn't do pure black but

00:23:18 ◼   ► the market says differently and you just

00:23:19 ◼   ► have to adopt at that point yeah but it

00:23:22 ◼   ► is always tricky and I think that it

00:23:24 ◼   ► comes to mind is I've made the mistake

00:23:25 ◼   ► in the past of going too far down this

00:23:29 ◼   ► road of adding things like themes and

00:23:33 ◼   ► like these kind of more super

00:23:35 ◼   ► superficial options to an app and then I

00:23:37 ◼   ► end up regretting it in this way in the

00:23:39 ◼   ► sense that you now have three themes you

00:23:44 ◼   ► need to support in your app and you know

00:23:47 ◼   ► when you're anytime you add a view

00:23:49 ◼   ► anytime you make a change you're gonna

00:23:50 ◼   ► have to make sure that it works well now

00:23:52 ◼   ► in all three of those themes it reminds

00:23:56 ◼   ► me in a weird way of localization where

00:23:58 ◼   ► I like and in my early days with

00:24:02 ◼   ► localization I would get somebody who

00:24:04 ◼   ► would say hey you know I I would like to

00:24:07 ◼   ► localize your app into you know whatever

00:24:09 ◼   ► language they spoke natively and I use

00:24:11 ◼   ► it all the time I think it'd be great

00:24:12 ◼   ► and I'll you know be happy to do the

00:24:14 ◼   ► translations for free just send me the

00:24:16 ◼   ► strings file and it's like initially I

00:24:18 ◼   ► used to think that's a great idea like

00:24:20 ◼   ► sure let me let me know the more the

00:24:24 ◼   ► better and then you realize now I need

00:24:26 ◼   ► to support this going forward and every

00:24:28 ◼   ► time you add a string to your

00:24:29 ◼   ► application

00:24:30 ◼   ► you have to have it translated or now

00:24:32 ◼   ► you when you do screen shots you need a

00:24:34 ◼   ► different thing to manage there and so

00:24:37 ◼   ► while it is sometimes it's good to you

00:24:39 ◼   ► know it's good to be adaptive and good

00:24:41 ◼   ► to be you know responsive to users that

00:24:44 ◼   ► it is in the back of my mind I've been

00:24:45 ◼   ► burned enough by it now that I'm like do

00:24:47 ◼   ► I really want this theme do I really

00:24:49 ◼   ► want to have to you know that your your

00:24:50 ◼   ► great trick where when you're checking

00:24:52 ◼   ► you know dark mode switches have it

00:24:54 ◼   ► switched between the themes you know

00:24:56 ◼   ► every every set for a couple seconds so

00:24:58 ◼   ► that you can make sure that you want

00:24:59 ◼   ► your you wanna looks good so I really

00:25:01 ◼   ► want that to cycle through three well or

00:25:03 ◼   ► now maybe if someone suggests four or

00:25:05 ◼   ► five like you it gets to a point where

00:25:07 ◼   ► you can end up being too responsive to

00:25:11 ◼   ► your users and so I guess there's a

00:25:12 ◼   ► certain element where you have to draw

00:25:15 ◼   ► lines and you have to say like or at

00:25:17 ◼   ► least to think about do what how willing

00:25:18 ◼   ► am i to support this you know how

00:25:20 ◼   ► endured injustice create confusion to

00:25:23 ◼   ► people now when they look in your

00:25:25 ◼   ► Settings app and look at the themes area

00:25:27 ◼   ► and they're like do I want dark or do I

00:25:29 ◼   ► want black it's kind of like when you

00:25:32 ◼   ► know Apple is selling the matte black

00:25:34 ◼   ► and the jet black iPhones interesting to

00:25:37 ◼   ► be like it now I have this choice I

00:25:39 ◼   ► don't know like how do I know which

00:25:42 ◼   ► one's better and you know they're having

00:25:43 ◼   ► a bit of opinion there may actually be

00:25:45 ◼   ► helpful but in this case it's like

00:25:47 ◼   ► you're you're at a certain point you're

00:25:49 ◼   ► being forced to do it just by the volume

00:25:52 ◼   ► of requests but it's always yeah it

00:25:55 ◼   ► makes it makes my developer since this

00:25:57 ◼   ► tingle a little bit and be like do I

00:25:59 ◼   ► really want to do this do I understand

00:26:00 ◼   ► what I'm taking on by adding this it

00:26:02 ◼   ► seems like just a little option you know

00:26:04 ◼   ► it's just oh it's just black but you

00:26:07 ◼   ► know you this is just the way things go

00:26:08 ◼   ► I suppose I mean honestly I did it on a

00:26:11 ◼   ► plane flight like that's how fast it

00:26:13 ◼   ► wants to do because it's almost worse

00:26:15 ◼   ► yeah it's so easy it's like oh let me

00:26:18 ◼   ► just do this one thing and then it turns

00:26:20 ◼   ► out you have to support this one thing

00:26:22 ◼   ► forever well to be fair like you know

00:26:24 ◼   ► it's similar to localization going from

00:26:27 ◼   ► one to two themes is a lot more work

00:26:30 ◼   ► than going from two to three and you

00:26:33 ◼   ► know like going from two to three like

00:26:34 ◼   ► if you if you did you're one to two

00:26:36 ◼   ► migration like if you added theming to

00:26:38 ◼   ► your app in a reasonable way adding

00:26:41 ◼   ► additional themes is actually pretty

00:26:43 ◼   ► straightforward

00:26:44 ◼   ► and you are right that that you know

00:26:45 ◼   ► there is certainly a testing and

00:26:46 ◼   ► maintenance burden and maybe a screen

00:26:48 ◼   ► shot bird and if you if you go that way

00:26:50 ◼   ► for certain things but you know it

00:26:52 ◼   ► really is not as big of a deal but again

00:26:55 ◼   ► the reality is if the market is like if

00:26:58 ◼   ► people are willingly making their

00:27:00 ◼   ► experience with your app worse and

00:27:02 ◼   ► possibly their experience with their

00:27:04 ◼   ► entire phone worse in order to achieve a

00:27:07 ◼   ► thing that you don't think you have to

00:27:08 ◼   ► do

00:27:08 ◼   ► the reality is you probably have to do

00:27:10 ◼   ► it sure yeah I mean ultimately it's like

00:27:15 ◼   ► there's always a tension there but you

00:27:17 ◼   ► have to a certain point you just have to

00:27:18 ◼   ► relent and it's I suppose it's more just

00:27:20 ◼   ► a question of making sure we're being

00:27:21 ◼   ► thoughtful when we were lent that we

00:27:23 ◼   ► understand what we're taking on that

00:27:25 ◼   ► these seemingly small choices that we

00:27:28 ◼   ► you know can make we have to live with

00:27:30 ◼   ► for a long time and I guess that in some

00:27:32 ◼   ► ways that's the theme of all three of

00:27:33 ◼   ► these kind of challenges you've had to

00:27:35 ◼   ► navigate is that whenever we are making

00:27:37 ◼   ► choices as we're developing we have to

00:27:40 ◼   ► understand that you know we're going to

00:27:42 ◼   ► live with all of these choices going

00:27:43 ◼   ► forward and so at least trying to be

00:27:45 ◼   ► thoughtful about them or thinking or

00:27:46 ◼   ► take a moment to be like do I really

00:27:48 ◼   ► want to do this or how can this come

00:27:50 ◼   ► back to bite me in the future you know

00:27:51 ◼   ► how am I going to hate myself in the

00:27:53 ◼   ► future because of this choice that I'm

00:27:55 ◼   ► making and having a little bit of

00:27:57 ◼   ► thoughtfulness is probably wise just to

00:27:59 ◼   ► avoid you know getting into situations

00:28:02 ◼   ► that we may regret later thanks we'll

00:28:05 ◼   ► see everybody and we'll talk to you next

00:28:06 ◼   ► week bye