Under the Radar

Under the Radar 27: Fast App Review

 

  welcome to under the radar a show about

  independent iOS app development I'm

  Marco Arment and I'm David Smith under

  the radar was never longer than 30

  minutes so let's get started so an

  exciting thing happened over the last I

  guess it's about a week or so at least

  since I first noticed this and started

  to hear about it so as you would imagine

  as someone who has quite a few apps I

  submit to the App Store on a

  semi-regular basis you know usually I'll

  have at least a Napper to in review at

  any given time and you notice something

  a bit funny the my apps started to get

  through review really quickly and in

  fact this is now sort of gone from being

  this sort of a freak occurrence which

  I've had before you know in the past

  I've had like a freak app review that

  just went through in a day or two or I

  think I've even had a couple that went

  through in a few hours and you know who

  knows what that is but this has happened

  now three times in a row where my last

  app reviews have taken 27 25 and 17

  hours respectively and I've started to

  hear more generally from app developers

  that this is a thing now that whatever

  is causing this to happen is a

  consistent thing it isn't just my

  account and if you go to app review

  times which is this lovely little

  community source yeah I don't know site

  that Dave or who's the guy who also does

  the iOS dev weekly newsletter runs where

  you can you know you can submit your app

  review times it seems like this is a

  thing that the vast majority of apps

  that are you know people who have

  submitted their app review times

  recently are saying that it's taking you

  know a day or two you know somewhere

  between you know one or two days which

  is a massive departure from what we had

  before and you know certainly it's fair

  to have a side note obvious yes I know

  Android has no review time and it's

  amazing and it's wonderful but this is

  the world in which the iOS App Store

  lives and has lived for a very long time

  where there as long as I can remember

  the after view has taken about a week

  sometimes it takes a bit less than a

  week typically around big updates you

  know so like in September sometimes

  things will speed up a little bit

  because you know there's new update lots

  of updates coming out the Apple wants to

  get those read

  for the new phones great sometimes you

  can get it slower sometimes it can be

  you know 10 days 2 weeks but I would say

  about a week has been what it's been for

  seven or eight years now and how all of

  a sudden it seems like you know fingers

  crossed

  dabble has changed something in their

  process in their staffing in their

  automation in who knows what but they're

  doing something that's meaning that app

  review is taking a day and that seemed

  like a pretty big deal and something

  that we should talk about because I feel

  like a lot of the ways in which I

  approach development are in many ways

  sort of there were they're working

  backwards from it's whatever whenever I

  submit it's going to be a week until

  that shows up in the store and so if you

  change that calculus to essentially be a

  day then a lot of things about the way

  that you know updates that I think are

  big enough or good enough to go and

  actually submit or things might change

  and just in general like it's it's good

  for a lot of reasons there's also some

  tricky things that I think will also be

  worth updating so does that make sense

  yeah I mean I I'm as surprised as

  anybody because as you said like it has

  been about a week you know whether it

  was six days or eight days like it's

  been about a week for almost the entire

  history of the App Store

  and we should clear for the iOS App

  Store the Mac App Store has gone all

  over the map because it seems to have to

  have gotten a lower priority most of its

  life and and so the Mac App Store times

  have often been as high as like 30 days

  which was horrible and that's pretty

  crippling to to any actual efforts to

  get quality software in the market but

  on iOS it's been pretty much yeah pretty

  much a week forever and you know I don't

  think there was I don't think this is a

  coincidence I think Apple is pretty

  clearly setting internal metrics of

  whatever guideline they they consider to

  be good enough they hit that guideline

  and it's pretty obvious that for most of

  the history of the App Store that's been

  a week and there's lots of reasons why

  Apple might want to not get faster and

  one of the biggest ones obviously is the

  faster they review apps the more

  frequently people will submit updates so

  just

  it'll increase the total volume of

  updates they're going to have to deal

  with which will increase their cost

  increase the size of that team but that

  the team has to be et cetera and you

  know as you get more and more people in

  this in the app review team you're

  probably gonna have more mistakes made

  and that increases the risk of angering

  developers getting negative press you

  know possibly letting letting bad things

  through instead of you know being a too

  aggressive on the good things so it's

  certain there's certainly reasons for

  Apple to want to keep the review times

  high for all these kind of accessory

  benefits they get from it and and I

  really it really does seem like for the

  last eight years they have just thought

  that one week was good enough and that

  was a good balance of all these factors

  according to them and developers have

  kind of gotten used to it and have just

  I was resigned to accepting that it

  would just always be about a week

  because they obviously were targeting

  that and so that was just the way it was

  it was always going to be forever and

  the idea now that that that over the

  past few months like if you look at the

  graph on on the Trant development thing

  I look at the graph like it it hasn't

  been like a dramatic drop just yesterday

  it's been like a pretty straight

  downward slope for the last few months

  basically since Phil took over and it's

  it's probably related although Phil

  already ran that department before so

  like Phil already was the head of the

  department that contains a preview so he

  was already the top of that before so

  presumably he could have you know he

  probably could have just heard this

  before and just and didn't because I he

  probably thought he was he was however

  the one who made the decision that seven

  days was was a good target so there are

  changes afoot like this is it there's

  because it has not just been like all of

  a sudden in like a day it's been like

  this because it's been this this kind of

  steady progression downwards over the

  last few months I think it's more likely

  that this is intentional and that this

  is possibly the new goal possibly here

  to stay and that will just have

  tremendous effects on our development

  because like like I remember back when I

  purview you know back when the apps were

  started

  you settled pretty quickly into this

  one-week delay I I said a number of

  times back then like yeah cuz back that

  iOS developers had not quite resigned

  ourselves this is like the inevitable

  forever state and we were like oh I

  preview is terrible there should be no a

  preview we should have direct access and

  and be able to update as much as we want

  just like people could always do on the

  Mac etc and and at the time I remember

  saying like there's a huge difference

  between having a short a preview and

  having a long a preview and the longer

  it takes generally like the more of a

  burden it places on developers and the

  more of like a slowing effect it has on

  on technological progress on development

  and on your ability to ship quality

  software I remember even saying back

  then seven days is you know we can

  tolerate this but it would be it would

  be totally different if it was 24 hours

  and now it appears that after eight

  years they are actually changing it so

  that it is 24 hours and I never thought

  this would happen I am very happy about

  it it will change the way we do things

  and not all those things will be for the

  better but I think most of them will I

  don't know I mean what do you what do

  you think how will this change I mean

  your special case also because you do so

  many apps how do you think this will

  change the way you work so I was trying

  to think through how this changes things

  and the biggest like I think that it

  makes different has different impacts on

  different kinds of updates that I'm

  trying to do and it sounds probably

  worth one thing I looked up ahead of

  time was I was curious what the current

  adoption rate for updates is recently

  you know since iOS 7 there's been

  automatic app updates so you know

  customers don't have to go to the App

  Store and hit update anymore like and

  for most people that just happens

  although I believe that's off by default

  isn't it

  if it is then a lot of people are

  turning it on or going to the App Store

  a lot because what I saw is that for me

  I'm seeing about three days to get 80%

  of people updated to the latest version

  because that's also an important thing

  to keep in mind because like even if the

  app update cycle is a day

  it's still a while until that update

  will be running on everybody all of your

  customers apps but it's still a pretty

  substantial change if say we're going to

  a world where 80% of your customers get

  the update four days one day for app

  review three days for actually updating

  it on their phone you know versus eleven

  twelve thirteen days like that's a

  pretty substantial change but that's you

  know so good to keep in mind that so

  within about you know three or four days

  I can get people doing things for small

  changes like bug fixes you know I see

  pan update and I you know I get a crash

  report back something like that some

  situation where there's this little

  change that is obviously wrong with

  what's currently out there this changes

  the dynamic for me a lot for looking at

  it and saying well maybe I'll just do

  like a point oh one release 0.02 release

  point O three release like doing the

  more iterative model in terms of these

  little little fixes when I used to look

  at those kind of updates and I'd bundle

  them together because you know a lot of

  those fixes really are short and simple

  and kind of verifiably correct you know

  you look if you look at you'll find the

  line of code where oh then you know what

  am I thinking you know I have an off by

  one error I have an out of bounds there

  like something that is just obvious and

  fixable and I want to get that out to my

  customers as quick as I could but you

  know doing a going through at one-week

  process when I can't submit and any any

  other changes it usually didn't make

  sense and so I tend to bundle these

  things up and then you know do a big

  sort of a bigger submission and you know

  when that one came out maybe I'll do

  another one

  but you that the cadence of that was

  very much you know I'd be doing maybe an

  update or two a month but you could

  imagine a world where you would do

  updates much more regularly I mean in

  many ways it's much sort of like you

  imagine with web development where you

  can do releases on a fairly regular

  basis and your as a result you're sort

  of your average lifespan of a bug of a

  known bug will drop dramatically

  you know you discover a bug you fix a

  bug you ship that out to your customers

  so for small changes it seems like

  that's a

  new something that I'll have to think

  through on bigger releases it sort of

  changes things I think the biggest thing

  that it changes is if it can become

  consistent ly around a day or two then

  when I'm doing a big update something

  that I'm trying to mark it around

  something that I'm trying to get

  attention for it'll be great to have

  some a better sense of if I put it in

  review

  it'll be approved or rejected within 24

  hours say that was like what the this

  becomes and I really hope Apple doesn't

  change their mind on this and this is

  just some kind of cruel experiment but

  if this is actually the new reality then

  that changes that a lot because I can

  submit my update I can get it approved

  and I can coordinate my marketing versus

  the thing that I ended up having to kind

  of do now where I submit it and I say

  like tell you I'm tell people in the

  press and talk to people and say hey

  sometime in the next 7 to 10 days

  I hope it'll be approved and you know

  maybe if you can write an article about

  it that would be great but like the

  worst thing on those updates is if it

  gets rejected and if you get a rejection

  and then getting the rejection reira

  viewed takes a week like suddenly you

  can end up the you know of some that you

  hoped was going to be one week can

  become three weeks or more and so

  turning that around to you get rejected

  you resubmit you get approved and that's

  two days rather than two weeks is just

  massive for release planning for big you

  know marketing pushes and things like

  that and yeah I mean like the rejection

  thing I think is one of the biggest

  savings here because you know I said

  like if you're especially if you're

  submitting a brand new app Apple's gonna

  nitpick a lot of things you might have

  thought of or they're gonna disagree

  with you on something that's kind of

  near the edge of a rule or something

  rejection is a multiplier of this app

  review time where like you know if it is

  not uncommon for for up for the a 1.0 of

  an app to get rejected one to three

  times before one finally gets approved

  you know with little changes here and

  there the more you you go near third

  rails like in-app purchase or pay

  services like the more likely that is to

  happen but either way like you always

  had to plan like you know well this

  might get this app I'm planning to

  launch my

  get rejected one to three times before I

  can launch it so now your buffer which

  used to be like well this is probably

  gonna be in the store sometime in the

  next month can now shrink down to this

  can probably be in the store sometime in

  the next week and that that is

  substantially better it just it

  dramatically decreases the cost of

  rejection and and it also decreases how

  bad it is if Apple wrongly rejects you

  this is true

  yeah and I'm delighted to vent it's just

  it's I'm honestly still trying to wrap

  my head around some of the changes

  because I think about this and I

  combined it with thinking of recently

  about the business models and the things

  that are all on both sides in the App

  Store where you know things like paid

  updates which we don't have would have

  would meek would make bundling up the

  changes in two large things making make

  a lot of sense you know sort of the the

  the model of software where your version

  number you know it's like you have a 1.0

  and then a year later you have a 2.0 and

  that kind of a a model which in this I

  would say feels fairly outdated at this

  point the changes too dramatically if

  there's no incentive on the business

  side to bundle up your changes and if

  releasing updates it becomes really

  lightweight and straightforward it's

  kind of like this is a situation you end

  up with I believe I think this is what

  chrome does the browser we're like

  there's just a new version all the time

  or even a scenario where I think I don't

  actually use the Facebook app but I

  believe it's just sort of every week

  gets an update and the number goes

  bigger or something along those lines

  I've heard of a lot of software

  companies moving to that kind of a model

  where you're you just ship on a very

  regular basis and the app the product

  you're sort of always gets better rather

  than there being these discontinuities

  in updates you know that rather than

  having these big moments where the

  customers have to be like whoa what's

  going on and you know sometimes you'll

  still have those if you're doing a big

  UI refresh or something like that but in

  general if the app is just sort of

  always getting better if it's getting

  updated in the background behind the

  user

  you can kind of get into a very

  interesting pattern there that kind of

  completely gets away from the concept of

  like oh I'm working on my next big

  three-o

  it's like no I'm just the app is just

  always getting better and I don't need

  to break them up into small into these

  big updates because updates are

  lightweight they're like this sort of

  this very simple thing that you know

  it's even if there was no app review it

  taking you'd like the difference between

  a day and zero days is a fairly minor

  because the majority of the time is

  going to be spent in getting people to

  actually go and download it from the App

  Store's like to get the new software

  anyway so as long as that number is on

  that very low end it's warped by the

  update rate and so it's essentially for

  like free to do which hopefully Apple

  you know can adapt to because I would

  imagine the rate will necessarily go up

  but I mean the Android store has had

  essentially no app review well they have

  a purview but it doesn't take time like

  I'm not entirely sure in the details of

  that but I remember they famously

  announced that they had added app review

  to the Google Play Store but nobody

  noticed yeah I think it takes like a few

  hours to a day something like that but

  it's it's short and that works and

  that's great and if we give that's our

  world to now like that's kind of wild

  like I would love the amad you know the

  the who worst thing and I think you ran

  into this recently with an overcast

  update where is where you ship an update

  that you know like you're all your

  testing all these things you somehow you

  missed a pretty substantial bug you know

  a date of lost bug something that's

  crushing your server is something like

  some some kind of really heinous bug and

  previously for those things we would

  have to go and do the like the bet you

  know like again our knees and beg Apple

  hey please give me an expedited review

  an Apple was usually pretty good about

  that you know in my in my years I've

  requested probably a handful of those

  they've always granted them I mean they

  the email you get back was almost always

  a bit comical because it has this kind

  of like well we'll make a special

  one-time exception for you and really

  shouldn't rely on this you know it's

  like poor you know it's like sort of

  this kind of very condescending email

  you get back but whatever I don't mind

  being condescended to if that meant that

  my app got really you know good approved

  in a few hours and

  this kind of terrible bug you know I've

  had I've shipped updates where you know

  it's like if you launch the if you

  launch that particular version it would

  like start corrupting your database or

  things like these things happen and

  they're terrible but I almost wonder if

  Apple would still have the expedited

  system you know in a world of you know

  if it's a one-day review cycle like if

  you can be you know a request a one-hour

  review like I don't even know if that

  would even make sense but it's lovely to

  think that at worst it would be a day

  before I could get something out I mean

  it takes them about a day to respond to

  those requests so it's like I think that

  actually might be a sign like if they

  end up eliminating the extra had to

  review system I think that would be a

  sign that they intend to keep through a

  few times this low because with the

  review times being approximately a day

  that system doesn't make sense anymore

  it's it's there's no gain there because

  it takes about a day to get the FFI

  reviews so yeah it's I mean that would

  man that would be great it this really

  would like many of the problems of app

  review are proportional to the time it

  takes every few to happen and so you

  know you don't get rid of all the issues

  of that review by having short review

  times but you do make a lot of them a

  lot smaller of a problem and so I man

  this is great

  anyway we are sponsored this week by

  Braintree go to Braintree payments comm

  slash radar why make payment integration

  more difficult than it has to be

  Braintree powerful full stack payment

  platform allows you to accept nearly any

  type of payment from any device with

  just one integration it's flexible to

  your systems needs and supports most

  languages where they're using Java Ruby

  or Python you'll always have a range of

  server-side and client-side SDK is

  available this code supports Android iOS

  and JavaScript clients and this takes

  just ten lines of code to implement

  Braintree now by next year maybe even

  next week there could be a whole new way

  to pay maybe it'll be the next Bitcoin

  the next app will pay maybe both

  fortunately brain trees full stack

  payment platform is easily adaptable to

  whatever the future holds so you can

  adapt easily to accept everything from

  pounds to PayPal to the next big

  innovation from any device with just

  this one integration and when that next

  payment method comes out all you have to

  do is update a few lines of code no late

  nights

  no complicated recoding no stress about

  staying ahead of the curve Braintree is

  here to help

  Braintree makes payments and your job a

  whole lot easier learn more at Braintree

  payments comm slash radar thank you very

  much to Braintree for supporting under

  the radar and all of real AFN so I

  wanted to also mention the the impact

  this has both ways on quality so like

  for me I mentioned before like I stress

  out like crazy when doing it release and

  and part of the reason I stress out is

  what you mentioned earlier of like I

  know that if I screwed up something

  big-time it's gonna take me a week to

  get to get a fix out there and that's

  just horrible and like worst case

  scenario if the app it's something's

  really bad like a data loss bug I'm out

  to pull the app from the store until I

  can get a fix out there and lose like a

  week of sales I've never had to do that

  but it always been like kind of like the

  like the the failsafe in the back of my

  mind like well I might someday have to

  do that if it's really bad I've had to

  do that yeah it's it's bad pull the net

  from from the store for a week and it's

  just like well that's that yeah I like

  it that's a that's a huge dent in your

  income and in the app's growth and usage

  and sucks for the customers so yeah so

  like I've always had in the back of my

  mind like man if I screw this up this is

  gonna take me a week to fix and if it

  only takes me a day to fix that's a

  pretty big difference and so this the

  faster you can get updates out the more

  you know the lower stress it is but also

  I think the more likely I might be to

  let my standards slip of of like testing

  and quality assurance to be like because

  before like if you've ever rejected a

  binary more than 24 hours after you

  submitted it then if you think about in

  this new system that would have shipped

  to customers and I've done that a few

  times I did that look two weeks ago so

  you know I feel like this will this this

  is a double-edged sword it will increase

  the ability for us to to to get fixes

  out there and to improve quality and to

  be faster but I think it will also

  increase the likelihood that we ship bad

  bugs

  and it's gonna take a certain degree of

  self-control of quality assurance of

  good rigorous practices to prevent that

  one of the weird side effects of this is

  that right now getting your app approved

  through the App Store takes roughly the

  same amount of time or even less then

  getting test flight beta approval for

  the first version of the baby ship in

  test flight so hopefully that's a

  temporary hiccup and hopefully that will

  change maybe to the point of getting rid

  of test flight review entirely because

  it doesn't make a lot of sense and

  especially in this world because like

  we're basically right now I am more

  incentivized to just update the app for

  everybody than I am to run a beta

  because running a beta will take way

  longer now then then just getting the

  update out there which is very strange

  and I think it also I was thinking too

  like it and one of the things I remember

  I when I was talking about this online

  people were the comment I got back was

  this like well this is just gonna make

  people use the public as their QA

  Department which i think is sort of what

  you're saying it's like well I may as

  well just ship it out into the world and

  if people if there are bugs people will

  tell me and I'll submit an update and

  we'll go back round and round and I mean

  I guess that the reality is that may be

  true and for some people that may happen

  but I think overall like the average

  quality will still go up because it's

  you know it's like our ability to sort

  of ask them like in a lot of ways I

  think of software quality you know it's

  like you're trying to get to say like

  for some arbitrary measure say it's like

  crash free users which is I think the

  way fabric and crashlytics measure it

  it's like I want that line to approach

  100% and the faster I can iterate on it

  the faster I'm going to get it there and

  so while some like some reality because

  I think think of you viewing like public

  QA as a bad thing like it can be but the

  reality is a lot of bugs can only be

  found in the wild you know if beta

  testing and internal QA were perfect you

  know then we wouldn't need a crash

  reporter we I wouldn't mix in my apps I

  would have just caught all the bugs

  before I shipped it but like that's not

  the reality like I had respect and

  especially any app that involves user

  data in any way

  a guy Avena of a recipe book app and

  sometimes I find some very strange and

  interesting bugs that are caused by

  issues that I just wouldn't have I never

  even dreamed of or imagined would be

  issues but it's like that's just the way

  people are using you know using the data

  or it's their particular language or

  character set has a weird quirk that

  causes a strange issue see also our

  episode number 16 designing for misuse

  yes exactly like some of these things

  you're only gonna find and but for me

  like maybe there is definitely I have to

  keep I think it's in it's like it's a

  good cautionary word to keep in the back

  of our minds that just because our

  review is quick and fixing a potential

  bug is more lightweight we still have to

  have the discipline of keeping of

  keeping ourselves to high standards that

  we're not just gonna like oh we'll just

  ship it and fix it and you know sort of

  fix it in post kind of mindset but if we

  embrace the ability of what that means

  if we keep our standards high they can

  now the overall quality can get to that

  much higher and having a much more

  higher quality level so much faster you

  know sort of in way that I think of in

  some ways like iOS updates like iOS you

  know big major updates where Apple

  doesn't update once a year with you know

  a few point releases on the way that's a

  much slower process necessarily to

  getting towards really high quality

  overall because you just it just takes

  longer and so there's a much higher

  period where things aren't going to be

  as good yeah and I think you know and we

  can see like it's not like this is going

  to be the first platform ever that has

  fast app updates available like we can

  see from existing platforms the Mac the

  web maybe Andrew it I don't know but you

  know you can see from other platforms

  what happens when developers come update

  their software whenever they want and

  the answer is some some products become

  like crashing unstable and changing

  every day but most don't

  most just have releases that are

  carefully issued when they need to be

  and like most of my Mac apps don't

  update themselves every day

  it's fine like they they wait until they

  they have a solid release and then they

  update them web apps tend to be the same

  way we know we're like you're generally

  not noticing changes every single day to

  web apps that you use

  they're generally not like crashing

  constantly because everyone's running

  untested code like people just have

  disciplined and good practices and they

  figure out how to make it work and I

  think iOS is gonna be the same way you

  know as this time stays down if it stays

  down as we hope it does then we're just

  gonna see just in the long run better

  software and there might be some hiccups

  along the way as some developers get a

  little bit too careless with it but I

  think that will iron itself out fairly

  soon and it'll just be great and I'm

  just assuming this stays here I could

  not be happier that they're making

  changes like this because this means

  that they care and that they recognize

  that the previous system was not as good

  as it can be exactly and I like honest

  genuine thanks to if anybody who happens

  to listen to the show happens to work in

  app review or know someone who works in

  app review you know highest of fives

  like this is awesome this is exactly the

  kind of thing that is an encouragement

  to me as a developer especially as a

  smaller developer that you know there

  are changes afoot that things you know

  we've been talking about for a while

  that the App Store is a trickier and

  trickier place to to make a business and

  to make a run at and these changes in

  some ways are small but they are

  impactful in terms of they allow me to

  ship better product more easily and

  that's awesome and so thank you all so

  big big congrats the app review team for

  the amazing degree of like secrecy that

  they have like i i've through my web

  presence and everything else i have met

  a lot of apple contacts or heard from a

  lot of apple people from every

  department you can think of in the

  company except a preview I have never

  heard from or met or heard about anybody

  who either worked in a preview or who

  even knew anybody who worked in a

  preview that's pretty crazy that's

  saying a lot anyway so a good job on

  them they're more secret in the car yeah

  so congrats i purview thank you for

  doing this please keep it up we are

  looking forward to this being our new

  reality I believe it's all the time we

  have for this week

  so thank you very much for listening

  everybody thank you Phil Schiller you're

  amazing and

  please keep this and we'll talk to you

  guys next week bye