0:00:00Welcome to Under the Radar, a show about independent iOS app development.
0:00:04I'm Mark Orment.
0:00:05And I'm David Smith.
0:00:06Under the Radar is never longer than 30 minutes, so let's get started.
0:00:10So today we wanted to talk a little bit about testing, about that process we go through
0:00:16towards the end of developing something where we try and take it from mostly working to
0:00:22actually working or fully working.
0:00:25And this is particularly relevant and interesting for me at this point because I am in that
0:00:29very final of phases for my next app, the one we talked a little bit about last week,
0:00:36that I'm in this phase now where functionally and from a feature perspective, I've kind
0:00:41of drawn the line and said, this is the app.
0:00:43This is as far as I'm going to go with it.
0:00:45Any other features or any other things that come up are going to be in the next version,
0:00:49version 1.1, or I'll push it down the road.
0:00:52All I'm now going to be doing is fixing bugs and testing and finding all the little issues
0:00:57and problems.
0:00:59And it seemed like something that would be an interesting topic to kind of walk through
0:01:03because ultimately the goal is to make sure that we ship things that are quality, are
0:01:10not riddled with bugs, they're having all kinds of problems.
0:01:14And it's a tricky thing, I think, especially from an indie perspective.
0:01:18And I think it's really fair to say, I'm definitely talking about this from the more
0:01:23of the indie perspective rather than if you are in a large corporate environment where
0:01:28your app is being worked on by lots of different people.
0:01:31You may actually have an entire QA department whose job it is to test your application.
0:01:36I used to work at a job I used to work at.
0:01:40There was a room full of probably 20 people who all day just sat there using the product,
0:01:46testing it, trying all the crazy edges, and they had these big test plans that they had
0:01:50to go through before we did a build.
0:01:52That is a very different world than I think either of us work in.
0:01:55And so we're not talking about that kind of testing.
0:01:58That is a discipline unto itself.
0:02:01It is useful and important.
0:02:02But for an independent, I think that's just not the reality, both from a resources perspective.
0:02:09I don't have the time to realistically do that, to do 20 people times 40 hours a week
0:02:17of testing.
0:02:18That's just impossible.
0:02:19And also, I'm not sure if it's necessary.
0:02:22And so this is coming into a certain category of bug that I'm less worried about.
0:02:29This is sort of the integration bug rather than the functional bug, where sometimes you
0:02:35just have bugs that come up because you had five people working on a product, and one
0:02:40person changed something here, someone else changed it over there.
0:02:42And obviously, myself, in different time domains, is different.
0:02:46I can be, like two weeks ago, me and my current me can be conflicting, but that happens less
0:02:51often.
0:02:52But either way, being independent, it's still something that I have to do.
0:02:56And it's a process that I go through now.
0:02:59And it seems like kind of a good thing to walk through.
0:03:01But I don't do it in a formal way.
0:03:03Marco, do you do any kind of formal testing or processes that you have around putting
0:03:08something out, or do you just kind of try it out a bunch and see if it works?
0:03:12I mostly just try out a bunch.
0:03:15And so we should clarify also that test-driven development is not what we're really talking
0:03:20about.
0:03:21Neither of us really do that.
0:03:23We've talked about it briefly before.
0:03:25And I don't think either of us really has a strong opinion on it, except just to say
0:03:28that we don't do it mostly as a choice because of our resources and our style of coding,
0:03:35of just being single person shops.
0:03:38It makes it harder to justify the additional amount of time it takes to write all the tests
0:03:43for test-driven development.
0:03:45So that's not what we're talking about here today, at least.
0:03:48So when it comes to whatever testing people would call what we used to call testing.
0:03:55So yeah, just like you or other people actually just using your app and trying out all the
0:04:02different things, testing out edge cases, the things that a good tester or QA person
0:04:08would do.
0:04:09That's what we're talking about today.
0:04:12And so my answer to that is generally I mostly rely on my own usage and reports from beta
0:04:19testers in order to do this.
0:04:23And this has pluses and minuses.
0:04:25I mean obviously this is a less disciplined approach than having like a formal unit testing
0:04:32process, a formal UI testing process, having dedicated testing people or services.
0:04:39I should look more into those things.
0:04:42I haven't to date and it mostly has not caused problems.
0:04:47And part of this is because of the way I developed the app in the first place.
0:04:50I mean step number one is I only make apps that I use.
0:04:55If you are making an app that you don't use yourself on a regular basis, and there's lots
0:05:00of reasons why you might want to do this.
0:05:02If you're a consultant, obviously that's a big one.
0:05:04But lots of people have these needs.
0:05:06If that's the kind of need that you're working with where you don't use the app, I really
0:05:10don't know what to tell you.
0:05:11I think things like unit testing make more sense there or having like dedicated QA people
0:05:16or services that you can hire to do that.
0:05:18But because I use the app myself, and I also tend to only implement features that I will
0:05:26use.
0:05:27And this is not, I don't do that 100% of the time.
0:05:30There are certain features like my nitpicky details feature about showing the number of
0:05:35unplayed episodes as a red badge on the icon of the app.
0:05:40I think you can tell by the description text in that interface that I don't want to use
0:05:44this feature.
0:05:45But enough people requested it and it was a small enough feature.
0:05:48I mean that feature is something like four lines of code.
0:05:50It's a very, very, very tiny little feature.
0:05:54And it satisfied large user demand.
0:05:57A lot of people requested that feature always before I added it.
0:06:01And so it satisfied a large user demand.
0:06:03It was not that much work to do.
0:06:05And it's very easy for me to test every so often, even though I don't use it myself.
0:06:08So that's something I added anyway, despite not using it myself.
0:06:12But otherwise, I try not to add features that I won't use myself very often, if at all.
0:06:18In part because I don't care, and in part because it makes it harder for me to maintain
0:06:23the quality of the app because I won't see problems in day to day testing.
0:06:28One of the issues that I faced recently is I have a pretty severe carplay bug in the
0:06:32most recent build.
0:06:34And I'm working on fixing that.
0:06:35I should have it for the next one, but I don't have a carplay vehicle.
0:06:38We don't own a carplay vehicle in our household.
0:06:40And we actually are about to get one in about a month.
0:06:43And that'll probably help.
0:06:44Although it won't be my car, which is unfortunate.
0:06:46But at least my wife will have one, so I will have hardware to test on sometimes.
0:06:51But carplay is this whole area of the app that I never see because I don't drive a
0:06:56car that's carplay enabled.
0:06:57And so there are gonna be all these different areas where my own day to day usage is not
0:07:02going to cover it or is not gonna reveal subtle problems at least.
0:07:08And so for those kind of uses I really just rely on beta testers and reports from users.
0:07:13Hopefully it's before I release these versions that have problems, but sometimes they get
0:07:19out there.
0:07:21So in order to minimize the damage I do try to at least cover the biggest most common
0:07:29uses of the app in my own formal or informal testing.
0:07:33So for example, when I'm releasing a big version I will almost always, if I remember
0:07:38to which isn't always the case, which is a problem, but I should probably, again this
0:07:42is one of the reasons why it might be useful to have a checklist or something, or a more
0:07:47formal structure of doing these things or an automated way of doing these things.
0:07:50But one of the things that I do in my app is to basically start fresh, start from a
0:07:56brand new user every time I do a release.
0:08:00Take a device that doesn't have the app on it, install a fresh version of the app, and
0:08:04then try creating a new account and starting all over again.
0:08:08And this helps for a number of reasons.
0:08:10Obviously that's a very important thing for your app to make sure it works at all.
0:08:15And then secondly it shows me the onboarding experience, it shows me the first run experience
0:08:20a lot.
0:08:21Like I see that whenever I do a release.
0:08:23I will see the first run experience at least once during that process.
0:08:27So that kind of helps remind me of any shortcomings in it, or it helps show me if anything's
0:08:33broken about it, or if things aren't quite right.
0:08:36So that's nice in that way as well.
0:08:38But this is all a very long way of saying basically the way I do testing is I just use
0:08:44the app a lot myself before I let anybody else use it.
0:08:47And then when it gets close to release I do beta tests.
0:08:52And I can talk a little bit more about that maybe a little bit later, but the beta tests
0:08:56I use tend to find almost any other ruining problems.
0:09:00I think it's important to say like, I mean I think my testing strategy is very similar
0:09:04to yours.
0:09:05It's the, I use the app a lot and I try and sort of use it, like as I get closer I
0:09:13tend to kind of use it slightly in anger.
0:09:15Like I use it in a way that is kind of trying to break it.
0:09:18Like I'm bashing on all the buttons and like what if I change this setting really
0:09:21quickly back and forth, back and forth.
0:09:23Like is anything weird going to happen?
0:09:24Like you can use it in that way.
0:09:27And then yeah like you said it's that kind of testing where, and honestly what I usually
0:09:30do is for the like trying out a new user testing it's the, you know I currently use an iPhone
0:09:377.
0:09:38That's the phone that I use like during development like 98% of my development happens
0:09:42on that phone.
0:09:43And so before I launch I need to at the very least run it on like a 5s and on a 7 plus.
0:09:51Just in terms of to make sure that there's not any weird UI bugs or issues or sometimes
0:09:55even just weird, there's performance things obviously going if you go back to like a 5
0:09:59or a 5s they're much slower.
0:10:01And so that's I find a great place to do the kind of onboarded, onboard testing because
0:10:07you know I'm starting fresh on those devices half the time anyway.
0:10:10I will also say one thing that has bitten me many, many times in the past and so now
0:10:15I always try and make sure I do too is I do the fresh user onboarding experience process
0:10:22but I do that running the old version of the app like the existing, like I go to the app
0:10:28store and download whatever this is, whatever the current version of the app is from the
0:10:32app store, onboard a fresh user and then install the version that I'm trying to verify and
0:10:39test with.
0:10:40Nice.
0:10:41Because often what I've found is there's issues that you can run into that like the,
0:10:43you know, usually it's a data issue or a settings issue or something that isn't being
0:10:48translated over correctly and obviously the majority of your users are going to be, are
0:10:54coming from that previous version and then running your new version and so if there's
0:10:58any issues there you want to catch them.
0:11:00But that kind of approach of just taking a lot of devices like on my desk I have a bunch
0:11:05of different, all the different iPhones, like I never sell an iPhone whenever I get a new
0:11:10one and so I have all the different testing devices available to me that way.
0:11:15You know, similarly like right now I'm wearing two Apple watches.
0:11:18I have a 38 millimeter Apple watch on my right hand and a 42 millimeter Apple watch on my
0:11:23left hand and they're paired to separate phones, running seven sizes and I'm just running the
0:11:28app that I'm testing constantly now.
0:11:30Like I just keep this process of kind of banging up against it.
0:11:35And if in general I think like you said, it's amazing how good the coverage you can get
0:11:41from that kind of informal but like constant high volume testing.
0:11:48Like maybe it's not as rigorous and so it loses something there but it definitely benefits
0:11:53from just doing it all the time and if you build apps that you use, you know, you certainly
0:11:57have that benefit at the very least of exposure to it.
0:12:00- Yeah, I mean one other note on the devices before I forget, app review does not seem
0:12:07to test much if at all on a variety of devices.
0:12:11As far as I can tell they test on one device only and usually it's a recent one.
0:12:15So as far as I can tell, so like a lot of times people will have issues where like they'll
0:12:20get an update into the store approved that like crashes on launch on the iPhone 5S or
0:12:25something like that, right?
0:12:26Because it's like an older device that is still supported and then it still runs on
0:12:29that your customers might still use but it doesn't work for, you know, and app review
0:12:35didn't catch it, you know.
0:12:36And so there's, you can't depend on app review to catch like the big obvious crashers because
0:12:42if it happens on anything, any conditions that they're not trying including any other
0:12:46device besides whatever one they happen to be using, you're not gonna have a good time.
0:12:50So that's very, very important to note.
0:12:52- And it's also probably worth saying that app review is not QA.
0:12:56Like app review is this very like course sieve that every now and then will catch problems
0:13:03and I love it when they do this but like that is not at all what they're doing.
0:13:07They're not really looking to test your app to find problems.
0:13:11They're kind of running it through at a very different level.
0:13:14And so I would not, it certainly isn't one of those like, well, I'll just see if app
0:13:18review can find any bugs and if they don't, then I'm sure it's fine.
0:13:21Like that is a terrible, that would be a terrible approach to take to with your app.
0:13:25- Yeah, it's like the seatbelt on a plane.
0:13:27It's like that is not the safety mechanism you should be relying on.
0:13:31You should definitely like, basically if app review rejects your app for a crash, you have
0:13:37a serious problem in your QA process.
0:13:39Like whatever you submit to app review should not be in that kind of state where it would
0:13:42crash in like the first four minutes of somebody using it once, you know.
0:13:47But you know, overall like, and I should also suggest too that if you are going to rely
0:13:52so heavily as we do on self-testing basically.
0:13:57And again, I do, I apologize greatly to people who enjoy formal testing and automated testing
0:14:04and unit testing and 100% test coverage.
0:14:06I apologize because this entire discussion must be driving you totally crazy.
0:14:11Hi Casey.
0:14:12But ultimately it's hard for single person developers to really justify the additional
0:14:20time and or expense that more rigorous testing methods bring on for things like this.
0:14:28So now, that's a lot of qualifiers in that sentence.
0:14:31Obviously if you have a larger staff, if you have a lot of money, if your app is more important
0:14:37than like, you know, than like a sleep tracker for your wrist or for a podcast player.
0:14:45Like if really bad things happen, if you like lose people's important data or you know,
0:14:51somebody will be seriously put out or hurt if your app breaks, obviously that's a place
0:14:57where you need a more formal testing procedure and framework in place.
0:15:01But most people who make iOS apps aren't working in that kind of capacity.
0:15:06And not only don't have those resources, but it would actually be kind of a bad way
0:15:10to spend resources basically like tripling all of your work.
0:15:14And this is a huge debate in programmer circles of whether test driven development is really
0:15:18more work or whether it ends up saving you more work in the long run.
0:15:21I honestly don't care which one of those it is.
0:15:24I think it's probably unknowable because it's probably like the answer like everything
0:15:28else is probably it depends.
0:15:31But ultimately this isn't the way we do things and I think there's a lot of developers
0:15:35out there who do things more like what we do.
0:15:37Anyway, that being said, one more thing before I go into the sponsor for the week and that
0:15:41is if you're relying so much on self-testing as we do, if your app has different modes
0:15:47or settings or ways you can do things, change it up.
0:15:52In your day to day usage, change it up every so often and spend a lot of time in each kind
0:15:57of condition.
0:15:58So for instance, Overcast has dark mode or light mode and during development I usually
0:16:03switch about once a week between which one of those I'm using because I want to kind
0:16:07of get used to it and live in it and make sure that I'm designing and making the best
0:16:11app I can for both of those things.
0:16:12Same thing with streaming versus downloading.
0:16:14Like I wrote this whole streaming engine and I hardly ever use it and that's kind of
0:16:18a problem.
0:16:19So basically I force myself to change my own preference and spend a good amount of time
0:16:26like a week or a month in streaming mode in the middle of development cycles where I would
0:16:31otherwise be in downloading mode.
0:16:33Just to make sure I'm getting a really good idea of these vastly different modes that
0:16:39my app can be in so that my own testing is more effective.
0:16:42Anyway, we're sponsored this week by Pingdom.
0:16:46Pingdom can help remove the need for you to have somebody who automatically logs into
0:16:50your website every minute and refreshes it to make sure it's up because Pingdom does
0:16:53that automatically with their testing framework.
0:16:56See for yourself at Pingdom.com/radar.
0:17:00Pingdom is an awesome monitoring service for websites and web services and when you use
0:17:05offer code radar at checkout you get a holiday special of 50% off your first invoice.
0:17:12This is an awesome deal, usually 20% now it's 50% for the holidays.
0:17:15So check it out at Pingdom.com/radar.
0:17:19Pingdom offers powerful easy to use monitoring tools and services for your website and web
0:17:25servers.
0:17:26Websites are very sophisticated these days and you need to, you know, there's all sorts
0:17:30of things that can go wrong and they know this, they see it because they monitor it
0:17:33for lots of people.
0:17:34You need Pingdom to be monitoring your site.
0:17:37They can monitor as often as every minute by requesting pages, by running through checkouts,
0:17:41by logging in, by doing, you know, transactions.
0:17:44They can do all sorts of things, all simulated from their more than 70 global test servers
0:17:49and they can do it as often as once a minute.
0:17:51And if anything goes down they can alert you via so many different means, whatever you
0:17:55want.
0:17:56You can have email, text, push notification or any combination of these things and you
0:18:00can set the granularity.
0:18:01It's incredibly customizable.
0:18:03I've been using Pingdom now for, oh geez, I think eight years, nine years, something
0:18:07crazy, it's a very long time because I've used it way before they were a sponsor, way
0:18:11before I was even a podcaster.
0:18:13I was using Pingdom to monitor Tumblr back in the day and I just love them.
0:18:17I use them constantly.
0:18:19Highly recommended that you use Pingdom for all your monitoring needs.
0:18:23It is really the best and if you run any kind of website or web service, the last thing
0:18:28you want is for strangers on Twitter to be telling you when your site is down when you
0:18:32didn't already know that.
0:18:33You do not want to be learning that from Twitter.
0:18:36You want to know when your site is down.
0:18:38You want to be the first to know and then you want to be able to go in and fix it before
0:18:42all the people on Twitter see it and start bugging you about it.
0:18:45And with Pingdom you can because Pingdom will alert you the moment things go down with that
0:18:49up to one minute granularity.
0:18:51That's really incredible and they are so fast and so reliable.
0:18:54I love Pingdom.
0:18:55Check them out yourself.
0:18:57Go to Pingdom.com/Radar.
0:18:58You will get a 14-day free trial and when you enter offer code radar at checkout, you
0:19:03will get a holiday special of 50% off your first invoice.
0:19:07Check it out today and you'll be the first to know when your site is down.
0:19:10Thank you very much to Pingdom for sponsoring our show.
0:19:13So I think for the last part of the show it seemed like something that may actually make
0:19:16sense for us to focus in on is a bit of the actual like the details of what this kind
0:19:23of user testing approach that we've just been talking about looks like.
0:19:28And for me what that usually means is so I get into this phase now where I'm doing this
0:19:32final level of testing and I'm just constantly trying to think of things to try.
0:19:36I'm trying them out.
0:19:37I'm using the app in different circumstances in different places like I'll be using it
0:19:41on different networks like hey I'm out on a cell network.
0:19:43I'm at home.
0:19:45And so whenever I find an issue, you know inevitably you'll find a bug.
0:19:47You'll see something that you're like huh that's a little interesting.
0:19:51What I found is a very important thing is to have some kind of reliable way of collecting
0:19:58all these little issues and bugs and problems.
0:20:00For me usually that is I have a list in OmniFocus that I am just adding items to.
0:20:08And so anytime I see something usually I'll grab a screenshot of it and if it's obvious
0:20:13I'll just attach the screenshot to the task in OmniFocus or if not I'll take it and run
0:20:19it through what used to be an app that you made Marco which used to be called Bugshot
0:20:24what I think is now called Pinpoint which is a great little tool.
0:20:27We have a link in the show notes to it but it's just a little thing for just drawing
0:20:29arrows basically or circling things.
0:20:32It's great for screenshot sort of when you're like huh you know there's this weird graphics
0:20:36glitch or something and so you can mark it up add it to a ticket.
0:20:39And then you know I go through all of like when I'm in the phase now like this morning
0:20:45I sat down I had the head of list about five or six little bugs I've noticed yesterday.
0:20:50I went through and I'm just knocking them out and it's kind of like this I work all the
0:20:55way down till I have no bugs and then I make a build and then I go through the process
0:20:59again and you kind of just keep cycling through collecting as you go.
0:21:04And for me that kind of this that kind of process of just keeping this running log seems
0:21:10to work well.
0:21:11It I like doing it that way rather than kind of having like sitting down and just testing
0:21:17for hours never really works for me.
0:21:19I find that I get kind of bored or I start doing the same patterns over and over again
0:21:24and so it's better I think for me I try and do it spread it out throughout the day and
0:21:28obviously if it's an application that you would use normally and naturally throughout
0:21:32the day then this makes it easier.
0:21:35But otherwise you just kind of have to remind yourself to do it and this is also a little
0:21:41bit tricky if you have an app that requires a certain amount of time for testing.
0:21:46So like for example I have a sleep tracker like in order for it to really like I cannot
0:21:51compressed beyond just running it on multiple devices the testing of that because it takes
0:21:57about eight hours to have eight hours of data collection and like I have simulated stuff
0:22:02in the app for you know when I'm doing like debug testing that I can kind of like create
0:22:06a night's data and run my algorithms on it.
0:22:10But for actual testing there's nothing you can really do other than wearing multiple
0:22:13watches which is what I do like when I'm doing extensive sleep testing like I'll have all
0:22:18my watches on my wrist during the night and I'm running it three or four times.
0:22:22But you know sort of in the same way I imagine for you with overcast like there's a certain
0:22:26amount if you just have to play lots of audio because if you don't you're never going to
0:22:30find all the weird audio issues or glitches or problems.
0:22:33There's nothing that you can really do to speed that up or compress that testing.
0:22:39Yeah basically I mean I like I just find ways to shove podcasts into more of my life so
0:22:46like you know like I've reached and also to test out things like syncing you know like
0:22:50my syncing engine took forever to really figure out I mean it took me almost a year to really
0:22:54nail syncing and one of the ways I started finally getting better at that was I started
0:22:59needing it more because I started using an iPad to play podcasts in the kitchen and I
0:23:04use my iPhone to play them everywhere else.
0:23:06So I will I was very frequently switching between multiple devices that would use the
0:23:10sync backend and just by doing that myself by needing that myself for a few months I
0:23:16really fixed a ton of sync problems and got it to a pretty good place now.
0:23:20And so yeah I mean it really is just a matter of just you know again it's like if you
0:23:24only make apps that you will use a lot this becomes a lot easier.
0:23:28If you don't make apps that use a lot this you know you're gonna have to find other
0:23:33ways to do this effectively but it really works out great for me and I will again once
0:23:38again go in briefly about beta testing also and we pretty sure we did a whole episode
0:23:43on it if not we will and we should but I think we already did.
0:23:47We have so many episodes now I've kind of lost track but I do rely a lot on beta testers
0:23:53to really do like a final check on the app.
0:23:57They aren't my first testers.
0:23:59I am my first tester and I will usually use changes to the app for weeks or even months
0:24:06before I give it to anybody else to test just because I really want to make sure that I
0:24:09know I have a feel for it, make sure it's good, make sure it's the right choice and
0:24:12then I will finally share with testers like right before the public gets it.
0:24:16But basically beta testers can help a little bit not a ton but a little bit.
0:24:22I will also add that the what you mentioned earlier about your workflow about taking taking
0:24:28screenshots and then adding arrows and then adding them to OmniFocus and things like that.
0:24:33If you want to go off the deep end there are lots of crazy tools and frameworks that automate
0:24:37that process to varying degrees and have all sorts of cool integration.
0:24:40So there are some like I made bug shot kit which is basically shoved bug shot into an
0:24:45app as you could do things like it will detect screenshots being taken and then automatically
0:24:51bring up a modal controller with the screenshot saying here mark this up and send it in to
0:24:56report a bug.
0:24:57You can give that to all your testers and you can use that yourself during your test
0:25:00process and so you can basically have like easy or automated bug reporting right embedded
0:25:05into your app.
0:25:06And there's again there's lots of ways to do this.
0:25:08I'm not going to recommend any particular one because I haven't really looked but
0:25:10there are many of these that exist now and you can you can really go off the deep end
0:25:15with like the levels of integrations.
0:25:17You can have it like post to slack and then make an entry in your bug tracker.
0:25:21All sorts of crazy things you can do now if you really want to get into all that stuff.
0:25:25So the options are pretty good for now for like tools and frameworks and things like
0:25:30that to automate like the tedious stuff.
0:25:33But for the real just using the app really just again just using it a lot yourself is
0:25:39a very very effective way to do it.
0:25:41Obviously as I know automated things exist sorry Casey again but just using your app
0:25:45yourself will show you so much and will tell you so much and will show you things like
0:25:49you know this this workflow is kind of clunky or this animation doesn't really look right
0:25:54or this I really need this text to be lighter when it's really sunny outside or something
0:25:58like that like there's a large category of either problems or or good and bad design
0:26:04decisions for which just using it yourself is really the only way to know and really
0:26:09the only way to test and that isn't true for everything obviously sorry again Casey but
0:26:13that's true for quite a lot that we do in our apps.
0:26:15Yeah and I think with other building I would say with beta testing that's an important
0:26:19thing to it or another use for beta testing is the importance of not overly preparing
0:26:26your beta testers for what you know for the app so in terms of trying to give in giving
0:26:32them an experience that is similar to the experience that your app store users are going
0:26:36to have and so when they download the app and they're like what do I do or this is really
0:26:42confusing like some of the best feedback I get from beta testers is sort of accidental
0:26:47where they're not actually they don't know it but they're telling me something is fundamentally
0:26:53flawed or is really confusing where they're asking a question that like implies a fundamental
0:26:59misunderstanding about something and that kind of feedback is always really good and
0:27:03so I try whenever I'm beta testing to just almost like blindly send out the betas to
0:27:08people where I'm not giving them too much preparation or like giving them step by step
0:27:12walkthroughs or details it's very much like as though you just downloaded this from the
0:27:17app store you know maybe you've seen a screenshot or two you've had a high level discussion
0:27:21but you're working it out just as you would as a regular user and that's a great test
0:27:26because as the developer of the app obviously I know how everything works everything is
0:27:32intuitive to me because that's the way I made it but it is often not the case for your users
0:27:37and so one of the best uses for beta testing isn't you know it's great when they find you
0:27:42know actual bugs and issues and problems but it's also it's really useful for capturing
0:27:47those kind of like it's like if you're asking this question you clearly I clearly have not
0:27:51communicated the purpose of this feature correctly or I've hidden it in a weird way or you think
0:27:56it should do something and it does the opposite and those are the only thing that is almost
0:28:01exclusively possible to find in beta testing because no matter how hard you try to kind
0:28:08of like reset your brain to what if I was a fresh user opening the application it's
0:28:13kind of impossible where you know like the current app I'm working on I have been working
0:28:16on it essentially every day for months at this point so I'm so in it that I just don't
0:28:24see it's like I'm so far into the woods that I have no idea that there is a forest
0:28:27like all I have is trees like it is deep down into it I mean and maybe when I come back
0:28:34to it in a couple of you know like I'll put the app to the side for a little bit and
0:28:38I'll come back to it in maybe a month maybe I will have that kind of fresh experience
0:28:43but until that time the only way I can get that is with beta testing and so it is definitely
0:28:47a valuable and important aspect of that.
0:28:49All right and I guess we're going to see your new app pretty soon right?
0:28:53Yeah it should be hopefully launching next week so it should be kind of exciting and
0:29:00I expect to I'm sure we'll talk about it on the show how the launch went and kind of
0:29:03some of the details of it as the app goes but it is very exciting to get to this point
0:29:08and I will say as you know it's like starting all the other independent developers out there
0:29:12it's like I just put in the work because there's definitely been times in this project
0:29:17where I was like oh gosh what have I gotten myself into maybe I should just like can this
0:29:21project and move on but there is hope eventually we'll get there eventually we'll ship and
0:29:25it is delightful when you get there.
0:29:27Can't tell you a secret to the audience?
0:29:29You can.
0:29:30I know what this is and I've used it and it's really good so look forward to it anyway
0:29:34thanks a lot for listening everybody we're out of time this week and we'll talk to
0:29:37you next week.
0:29:38Bye.
0:29:38Bye!
0:29:39[ Silence ]