0:00:00[Music]
0:00:06From Real AFM, this is Connected, episode 149.
0:00:10Today's show is brought to you by Mack Weldon, Hover and Igloo.
0:00:14My name is Myke Hurley and we've got the band back together because Stephen Hackett is here.
0:00:19Hello, Stephen Hackett.
0:00:20Hello, Michael Hurley.
0:00:22How are you?
0:00:23I'm good. It's the 4th of July. It's raining outside.
0:00:28We're here together.
0:00:29Is this a haiku? It sounds like a haiku.
0:00:32Yeah, I don't know how to think out the syllables, but I'm a little disappointed that I haven't gotten any gifts for 4th of July yet from you.
0:00:38What about Federico, did you get me anything?
0:00:41No, and this reminds me of a joke on the Big Bang Theory when, like,
0:00:47someone convinced Rajesh that on the 4th of July people from another country are supposed to give Americans gifts.
0:00:55And so for like for 10 years he gave his friends gifts on the 4th of July to celebrate America.
0:01:00That's pretty good.
0:01:02Yeah.
0:01:03That's not a thing.
0:01:04I don't think you're supposed to give your American friends gifts on the 4th of July.
0:01:07So I don't have anything for you.
0:01:09I'm sorry.
0:01:10I'm assuming that the postal service is closed on the 4th of July, right?
0:01:14I would think so.
0:01:15That's why you haven't got your gift from me.
0:01:16It's in the mail but then I'm delivering it today.
0:01:19Interesting.
0:01:20Tomorrow is going to be a great day for you.
0:01:22That's a real promise that I feel like you're going to break.
0:01:25Who knows?
0:01:26Who knows?
0:01:27Amazon.com.
0:01:28Yeah, same day delivery.
0:01:32So we had some people write in and ask about the prompt curse.
0:01:37You all mentioned this, I think on last week's episode about the iPad and iOS 11 and exploding.
0:01:45We should explain what it is because we have a lot of listeners who weren't around during
0:01:48the prompt days.
0:01:50We had a show before this called The Prompt.
0:01:52The archives are on the Relay website.
0:01:53You can go dig through and find them.
0:01:55You find the one about the iPhone keynote.
0:01:57I think it's episode 30 is probably by far the best one.
0:02:02But the prompt curse happened because we
0:02:07used to cover photo services on the show a lot
0:02:11back when they were kind of a thing before I called Photo
0:02:13Library and Google Photos took everything over.
0:02:15So we'd cover companies like Everpix.
0:02:18And what were some of the others?
0:02:19I don't remember.
0:02:20Yeah, I don't know.
0:02:21they're all gone because we spoke about them.
0:02:25So we would talk about a photo service
0:02:28and the next week they would have a Medium post
0:02:30apologizing that they were going out of business
0:02:32and all your photos were gone.
0:02:34This led to a website that'll be in the show notes,
0:02:37prompt.photos, which is, I haven't seen this
0:02:41in like a couple of years and it is amazing.
0:02:44So the prompt curves, we talk about a service or an app
0:02:47and then the service or app goes out of business.
0:02:50It has happened to things other than photo services, but it is a thing that happened,
0:02:55it is a thing that continues to happen.
0:02:56We have this, I don't want to call it a gift, I don't know what to call it, we have this
0:03:00power.
0:03:01It's a curse that's in the name.
0:03:03We have a very specific set of skills.
0:03:05That's right.
0:03:06Which is to find services and to kill them.
0:03:10Yeah.
0:03:11This is what we do.
0:03:13So if you have an app or a service that you would like sunsetted, get in touch and we
0:03:17will talk about it on the air.
0:03:18We'll talk about it and kill it.
0:03:19I'm not even sure Medium used to be a thing when we were doing the prompt and these companies would shut down.
0:03:25Back in the day it was so difficult to announce that your company was laying off people and shutting down.
0:03:32It was a real struggle and then Medium came along and sort of provided this new niche.
0:03:37These companies that need to announce that they've been acquired and sort of Medium kind of revolutionized that space of shutdown announcements.
0:03:46It's really notable in hindsight.
0:03:50This leads to a paradox. Undoubtedly Medium will go out of business.
0:03:54Like probably sooner rather than later. They are heading that way.
0:03:58Yeah, they're doomed. Where does Medium announce that they are going out of business?
0:04:02My money is on the Medium programmers
0:04:06leaving some ASCII code in the source code
0:04:10of the webpage. Like you can inspect the webpage and you will see the announcement
0:04:14into the HTML, you know, that kind of thing.
0:04:18Sort of like a meta shutdown announcement.
0:04:21Yeah.
0:04:22- Or they could go old school and put it on Tumblr
0:04:24or LiveJournal, there's lots of options.
0:04:26- I would say a self-hosted Jekyll install
0:04:29with a Dropbox folder on GitHub.
0:04:33- Yeah, all of Medium just redirects to a gist
0:04:36and it's just like, oh, sorry, we ran out of money.
0:04:40VCs have no place to apologize now.
0:04:42It's gonna be sad.
0:04:43So Myke, have you installed the iOS 11 beta yet?
0:04:46Yep.
0:04:46I gave up.
0:04:49All right.
0:04:50That didn't take long.
0:04:51I was setting myself up for a summer full of jokes.
0:04:54It's over in one week.
0:04:56It was just-- I really wanted to do it.
0:04:59And then--
0:05:01Can't wait to do it.
0:05:01We had to record upgrade early.
0:05:04And there wasn't any news.
0:05:06So the only thing that we could think to talk about was iOS 11.
0:05:11And I didn't want another episode
0:05:12where I was just asking people questions, so I installed it.
0:05:16- So you're asking yourself questions?
0:05:19- Well, it's just me and Jason were just talking
0:05:21about some of our experiences.
0:05:23Mine were very early 'cause I only did it that morning,
0:05:25so I'd only used it for a couple of hours.
0:05:27But it was really, it was just the excuse that I needed,
0:05:31'cause I really wanted to do it.
0:05:32I kept backing up my iPad, I just kept doing it.
0:05:36I would back up my iPad and then I would chicken out.
0:05:39I did this for four days.
0:05:40I kept like, I back it up and there's no, no, I can't do it.
0:05:43I can't do it and I back it up and I can't do it.
0:05:44So I'm using it and I really like it.
0:05:47There are some things that are horrifically broken,
0:05:49but that was expected, right?
0:05:50Like TestFlight is just kaput for me,
0:05:53which I know is a, if things happen to some,
0:05:56not happen to others, some third party apps are working,
0:06:00but missing key features.
0:06:01Like I currently cannot export anything out of Dropbox.
0:06:05I have to use other apps to get things out of Dropbox.
0:06:08this
0:06:15absolutely in love with it. Like I am,
0:06:19I already feel so much more productive.
0:06:21Like it takes getting used to cause you have to adapt your workflows,
0:06:24but I feel like I have so much more control over the apps.
0:06:29Like I did this one thing today where I had two apps open side by side, right?
0:06:33And I was like,
0:06:35I'd be really much so much easier if I could just move this one to the left.
0:06:38And I was like, I wonder if I can do that. And yes you can.
0:06:41you can swap the apps around now.
0:06:42And I nearly squealed of excitement.
0:06:46And then I was like, oh, I need a calculator up,
0:06:48and I'd usually just open another app.
0:06:50But no, I can bring up Peacock
0:06:52in the little slide-over window.
0:06:54And there are some things that are different,
0:06:56like Command + Tab, whilst also being pretty broken
0:06:59right now, doesn't do what it used to do,
0:07:02where it would just switch out one app.
0:07:03But I'm getting really quick at just flicking up the dot,
0:07:06bringing out the app, and I also feel like I just have
0:07:08way more control about where apps are going to go.
0:07:11Like I can open them in the specific place that I want them to be rather than
0:07:15opening and then readjusting everything around it. Like,
0:07:17I know a lot of people are struggling right now and think it's complicated and
0:07:22coming from their previous workflow,
0:07:23but I think we're just holding onto what is fundamentally a broken way of doing
0:07:28things with the iOS nine version of multitasking.
0:07:31Like this is vastly superior and I think almost every single way,
0:07:36like I love it.
0:07:37Yeah, I have to agree with you, especially now that I'm using a bunch of third-party
0:07:43apps that have drag-and-drop support, putting together the various bits from my review.
0:07:50And just using this stuff, the slide-over with three apps at the same time, and drag-and-drop
0:07:56and some of these other changes are really just so impressive.
0:08:02coming from iOS 10, I'm saving so many steps that I would have otherwise used workarounds
0:08:12and workflows, and now what I'm noticing is I'm using way fewer workflows than I used
0:08:17to do.
0:08:18And also I'm copying stuff to the clipboard way less, because you don't need to use the
0:08:23clipboard anymore just to move data back and forth between different apps, you can just
0:08:27hold it and move.
0:08:28And I know that this is a discussion that we'll probably have later on.
0:08:33It takes, you know, you need to use multiple fingers, sometimes multiple hands,
0:08:37and it can feel like a whole circus going on, like you need to be a contortionist
0:08:41and perform these gestures, but they do work.
0:08:44And there's a kind of like geeky pleasure of being able to have these gestures
0:08:52and this multi-touch enabled drag and drop.
0:08:54It feels fun, and that's hard to explain.
0:08:57I feel like I'm really in control.
0:09:00Like I feel like I have so much more control
0:09:03of what's going on than I did before.
0:09:05'Cause I don't know, like I feel like I'm able to bend it
0:09:07all to my will a little bit, but like more than I used to,
0:09:10rather than me working like around all of those things.
0:09:13Like it's much nicer.
0:09:15I will say I'm not using drag and drop, right?
0:09:18Because most of the apps that I'm using don't support it.
0:09:20So I'm looking forward to that additional world
0:09:23of drag and drop as things move along throughout the year.
0:09:26I will say, just to follow up on last week,
0:09:28I don't want to get into this again because it makes me sad,
0:09:30but the notifications and widgets cover sheet thing
0:09:32is as much as a disaster as Federico outlined.
0:09:36- That's literally the next thing in follow up.
0:09:38But, yeah, so I'm running it too.
0:09:43I put it on my 10.5 inch iPad on Friday,
0:09:45and I very quickly realized that my entire
0:09:49iPad home screen was wrong,
0:09:50because apps in the dock are blessed in a way
0:09:53that apps not in the dock aren't.
0:09:55kind of like watchOS actually,
0:09:57that those things in the dock have,
0:09:59are easier to get to and can do more things.
0:10:01And so I basically collapsed my home screen into the dock
0:10:04and sort of promoted stuff that had been in folders
0:10:07on the second screen up to the home screen.
0:10:09So nothing is where it used to be.
0:10:11But like the two of you,
0:10:13I do feel like I'm in more control.
0:10:15And I'm still sort of learning how everything works.
0:10:20Sometimes I'll try something
0:10:22and it doesn't work the way I expect it to.
0:10:23And some of the gestures are a little less than ideal.
0:10:27Like if you have an app, for instance, that uses sliding panes like Slack, and you have
0:10:32it in the third popover window, and you want to push it back off the screen, and if you
0:10:38don't get it just right, you're just moving the interface around inside of Slack instead
0:10:42of moving Slack itself, because it's kind of a very thin strip you need to hit.
0:10:46I think a lot of developers still need to approach that.
0:10:51I feel like people haven't done that yet, right?
0:10:54Like, worked out, like, how do I make sure
0:10:56I give people enough space to be able to move
0:10:58the UI around, right?
0:11:01- Yeah, but all in all, it's impressive,
0:11:03and I think that iOS 11 is gonna be awesome.
0:11:08So, way to go, iOS 11 team.
0:11:12But you said you didn't want to get in the cover sheet,
0:11:14and that's too bad, because that is the next thing
0:11:15in follow up.
0:11:16We have a long tweet thread, we'll link to the first one
0:11:19the show notes about kind of following up on Myke on what you said about you
0:11:24know maybe is there something about this design that is giving us hints or make
0:11:30more sense with the next iPhone. So Apple's done this in the past right
0:11:34where they'll put things in iOS that don't quite make full sense until you
0:11:38see the new hardware and I think that's what this tweet threat is getting to. So
0:11:43Federico, do you want to take this?
0:11:46So the idea here is that if we consider a future iPhone without a home button and with
0:11:54a more prominent use of 3D touch, the argument from this person goes, the coversheet design
0:12:01makes sense because to open the phone, to unlock the phone, you tap on a notification,
0:12:10So all you need to do is you 3D touch on a notification, you unlock the phone and you
0:12:19see what that notification is all about.
0:12:23And it makes sense if you follow this idea to merge the notifications and the lock screen
0:12:31because you pick up the phone, you look at what's new, you tap on a notification and
0:12:36you authenticate somehow, whether that's Touch ID or maybe by looking at the phone, which
0:12:41we're going to talk about in a few minutes.
0:12:44So the idea is you remove confusion by merging these two areas of iOS and you have a list
0:12:53of notifications, you tap it and you go into the phone, which is unlocked and it lets you
0:12:56view the notification.
0:12:58I sort of understand the idea behind this, to merge everything, to make a single place
0:13:05where you can view all of your messages and alerts from apps and to also unlock
0:13:11the phone the moment that you open a notification. But the problem
0:13:16that I have with this idea is it kind of falls apart if you have no notifications.
0:13:22And at that point what is left to do on the lock screen? You authenticate and
0:13:28you're back into the phone so we're not really merging these two aspects because
0:13:34then again if you unlock the phone and then you swipe down and then you view no notifications,
0:13:39then you need to swipe up again. And as I said last week, my main problem is this sort of
0:13:45this seesaw approach of I need to swipe down and I need to swipe up even more carefully until I feel
0:13:53this taptic feedback from the iPhone that tells me you're scrolling up to view your older notifications.
0:14:01Whereas before, if I swipe down, I knew I could just keep scrolling without a release mechanism
0:14:07to load my previous notifications. And so, while I understand the current design of "we want to
0:14:14make it easier for people because they associate the lock screen with notifications, therefore,
0:14:19when you swipe down, you view notifications, so you view the lock screen", it's the interaction
0:14:24that bothers me. It's the fact that you need to release very intentionally to load the older
0:14:31notifications and also the complete absence of gestures to triage individual notifications,
0:14:39which is basically impossible now because you need to carefully tap on a notification,
0:14:43which a lot of people have problems. For example, Sylvia cannot long press on her iPhone. I don't
0:14:48I don't know if it's because she's a woman, she has obviously, you know, she has longer fingernails, you know.
0:14:54She's a different kind of user, so she cannot use 3D touch properly.
0:15:00She cannot even long press properly on the screen because her touches are often not recognized.
0:15:05And so you're making it more difficult to expand a single notification.
0:15:10Whereas it was much easier to just swipe across the screen and do one thing at a time with individual notifications.
0:15:18It's just, I understand the design, I understand the idea, it's the implementation that could be so much better.
0:15:25And again, the release mechanism of dividing the non-scene notifications versus the older ones,
0:15:34and you need to swipe up and you feel the tap and you need to load them,
0:15:39it's very intentional and it really slows me down.
0:15:42And I would like to see in Beta 3 or Beta 4, whatever,
0:15:46refinement of this idea because it can go somewhere but not with the current
0:15:52design in my opinion. I think that knowing everything that we know and
0:15:56we're gonna get into these iPhone rumors next because I think we just have to
0:15:59follow with this. I see that there is some thought with this that that makes
0:16:07sense to me like if looking at your phone now unlocks your phone then all
0:16:12you need to do is just interact with it in a way right so like just swiping up
0:16:16from the bottom or I expect there to be like some virtual button on the home screen that
0:16:21maybe you would just tap and it would take you there right like it just opens it up or
0:16:25something right because it's going to be all this new screen with a state or something
0:16:28but I could see a world in which this makes sense if touch ID is not needed anymore. I
0:16:34agree with you like some of the implementations wonky and that can get better but it really
0:16:38does feel to me like that there's something in this which is lining up with the rest of
0:16:43the rumors that we've heard about the iPhone considering how little sense this
0:16:47currently makes with what we're currently using that it feels to me that
0:16:50there just has to be something there. I don't know what hardware would dictate
0:16:55the the us losing the ability to dismiss a single notification. That's
0:17:00broken but I'm not talking about that like I'm talking about the idea of
0:17:03merging all of this stuff together as a concept. So the implementation is still
0:17:08leaves a lot to be desired like the fine details but the overall idea of why you would do this
0:17:14I think lends to that.
0:17:19I think that's it for follow up.
0:17:20Alright let's...
0:17:21You've done it.
0:17:22Yeah I want to just talk about this iPhone stuff now because we're moving into it already.
0:17:25Today's show is brought to you by Mack Weldon.
0:17:28The most comfortable underwear, socks, shirts, undershirts, hoodies and sweatpants that you
0:17:32will ever wear are made by this company because they believe that by pairing premium fabrics
0:17:40with meticulous attention to detail and giving you a simple shopping experience to get them
0:17:44home Macweldon can deliver to you a new level of daily comfort straight to your door. They
0:17:50believe and I agree that Macweldon is better than anything else that you have is better
0:17:55than whatever you're wearing right now. They make undershirts that stay tucked, socks that
0:17:59stay up on waist bands that do not roll. Everything is made with premium cotton blended with natural
0:18:04fibres and I love that their website is built to get you in and out as quickly as possible.
0:18:08They don't want to waste your time, they want to do their best to get you what you
0:18:11need and get you on your way. Macweldon are so confident in the quality of their products
0:18:16that they have a no questions asked return policy. They want to make sure that you're
0:18:20super comfortable in what you wear. If for any reason you don't like your first pair
0:18:23just keep them and they'll refund you, no questions asked. I love my MacWalden clothing
0:18:29and I desperately need more of it and I'm going to take care of that very soon as I'm
0:18:33making another trip to the US. So Stephen look out for a package, that won't be your
0:18:37gift but it might be some new underwear for me. Not only do MacWalden's underwear, socks
0:18:41and shirts look good, they perform well too. They're good for working out, going to work,
0:18:45travelling or for everyday life. Listeners of this show can get 20% off at MacWalden.com
0:18:50by using the co-connected. Thank you so much to Mac Weldon for their support of this show
0:18:55and Relay FM.
0:18:58I feel like we've gone around this tree quite a few times but we're back to talk about TouchID
0:19:03again on the potential next iPhone. Which I think for the case of this discussion we'll
0:19:10just refer to as the iPhone Pro. Are we all happy to refer to this phone as the iPhone
0:19:14Pro for this discussion?
0:19:15Alright, yeah, sure. Okay.
0:19:16Just so we can differentiate it in some way.
0:19:19So our good old friend at KGI Securities, Ming-Chi Kuo has issued a predictions report
0:19:25for the iPhone Pro and these are some of the things that Ming-Chi Kuo mentions.
0:19:30So I'll read these through and then we'll jump into the Touch ID one because it's probably
0:19:33the more interesting of all of this.
0:19:35So Ming-Chi Kuo says that the next iPhone will feature the biggest screen to body ratio
0:19:40of any phone on the market, therefore having the thinnest bezels.
0:19:44The screen will be OLED.
0:19:46The home button will be virtual.
0:19:49The front camera will include 3D depth sensing technology that will help with face scanning
0:19:55because Minchukko says that the next iPhone will have no Touch ID sensor of any kind.
0:20:06We predict the OLED model won't support fingerprint recognition.
0:20:09Reasons being 1.
0:20:10The full screen design doesn't work with existing capacitive fingerprint recognition
0:20:14and 2.
0:20:15Scan through ability of the under display fingerprint solution still has technical challenges.
0:20:20So Ming-Chi Kuo is saying that Apple has not been able to embed a Touch ID sensor into
0:20:29the screen of the phone.
0:20:31So instead of putting it anywhere else on the device, they are getting rid of Touch
0:20:36ID in favour of face scanning, 3D depth sensing face scanning which they have implemented
0:20:43for this phone and that it would be good enough. So there's a few questions about this and
0:20:49there's also a Bloomberg report which we'll get to in a moment which I think addresses
0:20:52some of these questions but I want to talk through them as a group.
0:20:57So first off, would Apple remove Touch ID? So removing Touch ID could potentially cut
0:21:04support for Apple Pay, right? If there's no authentication method. So let's just assume
0:21:08that there's no Touch ID. Apple's not going to just get rid of this method, right? There's
0:21:12going to be some kind of authentication because otherwise where does Touch ID go?
0:21:16Oh sorry, where does Apple Pay go, right? Like they're not gonna remove that.
0:21:21Just a blood sample. You just prick your finger on the phone every single time you buy
0:21:25something. It's a real Pavlovian response to buying new things.
0:21:31I don't see them removing it unless there's something equally secure
0:21:39and easy to use and we're gonna get into some things that could do that potentially.
0:21:44But they're not going, I don't see them getting rid of it without a really strong case to
0:21:50replace it with something else.
0:21:52And I can hear you now, Steven, the headphone jack!
0:21:55They removed the headphone jack and I don't think, I still don't think they've given enough
0:21:59of a reason why but that's a different type of feature.
0:22:04I will say on that note though, they did remove it but they gave me AirPods and AirPods are
0:22:08amazing and I know they could have given me AirPods anyway, I know they could have done it anyway, but
0:22:13you know there is a better thing. I think that sidesteps the actual argument of Touch ID like
0:22:19you said is key to not only Apple Pay but like device security and like a headphone jack like I
0:22:26was going to write off that complaint because headphone jack is is like it's kind of a pain
0:22:30to live without it but it's not it's not device security it's not payment security and can they go
0:22:38all their banking partners and deal with this in a way that keeps
0:22:44everybody happy or such ID like the linchpin and all of it. I just I don't I
0:22:49don't know if any of us on the outside know how important Touch ID is to some
0:22:52of those deals but Apple is not going to do something that is less secure
0:22:58right that goes against everything they've done like the last five years if
0:23:02not longer with this phone. So I just I don't buy this unless the replacement is
0:23:08equally good to use and equally secure. All right I have many many thoughts here.
0:23:14Here we go, settle in everyone. Let me go with an opening statement which is the
0:23:18idea of just getting rid of Touch ID because you cannot figure out a way to
0:23:23make you work with the iPhone Pro is the equivalent of getting a new car when you
0:23:27blow a tire. Like well we couldn't get that to work so I think it really matters why
0:23:31get rid of it. It is a real scorched earth situation, right? We couldn't work it out, so whatever.
0:23:37Yeah, exactly. So that said, we cannot agree on the fact that there's no way that Apple is going back to a world where it's just a passcode.
0:23:46There has to be a second, possibly based on biometric authentication method of providing an extra layer of encryption and security on the iPhone.
0:23:57So far, that has been Touch ID, and Touch ID powers a lot of services and features on the iPhone, from
0:24:03encrypting, backups, and the lock screen authentication with the App Store, Apple Pay,
0:24:09the API for developers. There's a whole API to use Touch ID in third-party apps.
0:24:16And Touch ID is really everywhere. And the thing that sells Touch ID, it's not that it's a fingerprint.
0:24:23Yes, it's a fingerprint reader, but it's the fact that it's so fast, accessible and easy to use.
0:24:30And also the fact that you don't have to look at your phone while you authenticate with HID.
0:24:36Anyone who's ever taken the tube in London, Myke, you know this.
0:24:39You're grocery shopping and you're in a hurry, you just place your finger on the home button
0:24:44and you feel the tap and you hear the sound effect and you're done.
0:24:47on. You know that you authenticate it, it doesn't get in the way, and it's an unassuming
0:24:53interaction. It's not flashy. We're at the point where most people are getting used to
0:24:59Apple Pay, most people are getting used to Touch ID. It's a natural way of authenticating.
0:25:04So if Apple wants to, in my mind, if Apple wants to provide an alternative to Touch ID,
0:25:09it has to be as quick and easy to use and accessible, because remember, one of the benefits
0:25:16Touch ID is that it's in a fixed position on the device, even if you have
0:25:21motor impairments or visual impairments, you can just feel Touch ID
0:25:26and you can use it. Anyone can use it, unless you don't have fingers or you've
0:25:31burned off your fingerprints, but that's another problem. So a faster solution to
0:25:36Touch ID, let's assume that it's face recognition. The problem with
0:25:42face recognition is not that most implementations so far have sucked,
0:25:46which is, you know, we cannot agree on that. They're not perfect. They're not great.
0:25:49And it wouldn't surprise me if Apple comes out with a much better, faster, more powerful solution, and we're all surprised.
0:25:56Well, Apple did it again. You know, they reinvented face recognition.
0:25:59I could see that actually. And I could see how in iOS 11, you know, with the machine learning stuff, if you look at the vision APIs,
0:26:06Apple has much improved the system that they provide to third-party developers. You know, the system can recognize faces.
0:26:13There are partially occluded people with sunglasses, with hats, profiles.
0:26:18The API itself is more powerful.
0:26:22So you can only imagine if the API is even more powerful, imagine the stuff that Apple keeps to themselves.
0:26:27So it's totally possible that Apple can pull this off, especially with the 3D mapping stuff with the depth perception.
0:26:34My problem is in the physical
0:26:37implementation of face recognition.
0:26:40So there's also a report from Germen on Bloomberg, and he says that Apple has figured out a way to make this work
0:26:47even when an iPhone is sitting flat on a table.
0:26:49So, you know, I have to assume even if you place your iPhone next to a scanner, next to a
0:26:57point-of-sale system, somehow the camera is gonna look right back at you and authenticate you. And I struggle to imagine how
0:27:04that could be possible.
0:27:07Unless there's a multi-camera array
0:27:10along the top edge and the bottom edge of the phone, I struggle to imagine a way to make this work without holding up an iPhone and basically taking a selfie.
0:27:22And if we reach the point where we imagine an iPhone Pro and a bunch of people in the London Tube or at my local supermarket, and everyone before authenticating has to basically take a selfie, even if it takes a second,
0:27:35It's not, you know, the operation itself can take even less than a second.
0:27:39It's the gesture of pulling out your phone, looking at your face, basically taking a selfie.
0:27:44Now everybody's looking at you and you end up with a whole system that takes at least three seconds
0:27:49and it's much slower than Touch ID. And this is where I come down on this rumor right now.
0:27:55It's hard for me to imagine a system that is faster than Touch ID. Not more secure,
0:28:01because I can't believe that Apple maybe has a solution that, you know, it can identify
0:28:062D pictures, so it won't authenticate you, it can identify, you know, even the finest
0:28:13details of your skin and your eyebrows and your retina, whatever.
0:28:17I totally believe that it's gonna be possible somehow.
0:28:21But it's the different physical behavior of using a face scanner and it's the accessibility
0:28:27of that.
0:28:28in low-light conditions, and there's a few people that say, "Well, Apple is going to use an IR scanner,
0:28:34so it can work even if it's dark." All right, sure. What happens if I'm a blind person and I
0:28:41cannot look at the screen? I don't know if I'm looking straight at my face or if I'm holding
0:28:47the iPhone at a weird angle and it doesn't authenticate me. Whereas with Touch ID before,
0:28:52or I could just feel the button and place my finger.
0:28:55It's such a different way of authenticating,
0:28:59and it's such a different way of holding the iPhone
0:29:01and saying it's me,
0:29:05that combining that with the even higher emphasis
0:29:10on Apple Pay in iOS 11,
0:29:12from Touch ID with Apple Pay in iMessage,
0:29:15the App Store dialog when you purchase apps in iOS 11,
0:29:19it looks like an Apple Pay sheet,
0:29:20there's a fingerprint icon right front and center on that dialog. It's so strange to
0:29:26me that Apple would get rid of Touch ID right now because they cannot figure out how to
0:29:31make it work with the iPhone Pro and that the solution to that is a face scanner. I
0:29:38just don't know what to think, honestly.
0:29:42Mark Gurman follows up with a report. Some quotes from Mark Gurman's report.
0:29:46Apple is testing an improved security system that allows users to log in, authenticate
0:29:50payments and launch secure apps by scanning their face, according to people familiar with
0:29:54the product. It can scan a user's face and unlock the iPhone within a few hundred milliseconds,
0:29:59the person said. It is designed to work even if the device is laying flat on a table, rather
0:30:05than just close up to the face. However, the intent is for it to replace the Touch ID fingerprint
0:30:10scanner. In testing, the face unlock feature takes in more data points than a fingerprint
0:30:15making it more secure than the Touch ID system,
0:30:18the person said.
0:30:19The feature is still being tested
0:30:20and may not appear with the new device.
0:30:22That last line, I put it in there.
0:30:23I think that this is one of those things
0:30:25that Bloomberg makes Garmin put in personally.
0:30:28I think that they're just like,
0:30:29you gotta cover yourself.
0:30:30- The safety net.
0:30:31- Yeah, and I think that they're making,
0:30:33putting weird stuff like that in there
0:30:34as well as like the, as the person said.
0:30:37So I'm just gonna, I'm just gonna do this, you know.
0:30:39Remember, year of optimism and all that,
0:30:41that I'm going for, it's my plan here.
0:30:44So I remember that fingerprint scanners were terrible and that the idea of
0:30:48Apple putting a fingerprint scanner into the iPhone seemed like a stupid one
0:30:52because fingerprint scanners were terrible, right?
0:30:55Like every fingerprint scanner I'd ever used on like a ThinkPad just,
0:30:58just didn't work, right? Like they just flat out didn't work. Touch ID works.
0:31:02It's flawless effectively, right? With how, how successful it is.
0:31:05So let's assume that what Gherman's saying is true, right?
0:31:11That all of this stuff is correct.
0:31:13This sounds fine to me. This sounds good.
0:31:16Like, speed, okay, so let's just take the argument about speed.
0:31:21I understand that as a thing about why speed in these situations is good,
0:31:26but let's say it takes a little bit longer than Touch ID.
0:31:32What are the other benefits, right?
0:31:34Like, speed isn't the only thing here, and there may be something that's nicer,
0:31:42there may be something that's just cooler about it, right?
0:31:44That all you do is just pick up your phone.
0:31:46The action that we all have for picking up the phone,
0:31:48the raise to wake.
0:31:49As soon as we do that now, our phones are locked.
0:31:51Like we don't need to worry about tapping anything
0:31:54or clicking anything.
0:31:56We don't need to have a home button anymore.
0:31:58So not having a home button means that our screens
0:32:01are even bigger than before, right?
0:32:03So that is a big benefit that we get.
0:32:05Maybe we don't need buttons at all on the next phone.
0:32:08It's just this beautiful thing
0:32:10because it's all about this using this face detection technology.
0:32:13Like in trying to have faith in this, I can maybe see that there could be some
0:32:18very interesting stuff in here and like stuff about like which I agree with.
0:32:22How do you do buying of stuff when you're already looking at the device? Like do you
0:32:28like there is an implementation detail in there that's not for us to decide, but if we assume
0:32:34that they work it out in as nice a way as they've worked out putting my thumb on the
0:32:41screen, then it'll be fine. Let's say for example it's kind of playful. Blink! Or like,
0:32:47you know, like there could be something in it that is interesting. Maybe you pick your
0:32:51own facial expression that helps you buy stuff.
0:32:55So the password. Show an emotion.
0:32:58So I think I agree that the pure idea of Touch ID going away is bonkers but if you think
0:33:05it just becomes Face ID and we use our faces instead of our fingerprint scanners and we
0:33:09get all of the functionality we had before and just leave it up to them to work out the
0:33:12details the benefits that we could potentially get just from a device perspective could be
0:33:19great and I agree with you Federico that on paper it maybe sounds harder for accessibility
0:33:24but I also have faith that Apple's got that part covered because that's something that
0:33:27they care about so much right like I will assume that whatever they do it
0:33:32will not become harder for people that have accessibility issues to use it may
0:33:37expose some new ones but everything does right like everything they add may solve
0:33:41something for some people make it harder for others but I'm sure that they will
0:33:46try their utmost to make sure that this feature includes as many people as
0:33:51possible and then make a epic amount of options to allow for people that now
0:33:57are struggling to be able to access their device in a different way, right?
0:34:01Like I'm confident that they will find ways to tackle that part because maybe
0:34:06more than any other technology company, and you mentioned technology company,
0:34:09Apple cares about this stuff more than anybody else, right? I'm really torn on
0:34:13this and I totally understand your point and I'm gonna play devil's advocate
0:34:17and let's think that it's happening and it's real. So I think you're totally
0:34:21right when you say even if it's slightly slower, not because the
0:34:26sensor takes milliseconds, but because there's humans holding phones and humans don't just take
0:34:32milliseconds to operate. But even if it's slower and it takes two or three seconds, if it's more secure
0:34:39then I totally buy the argument. Apple could say it's, I mean they won't say it's slower, but
0:34:44it's times more secure and unique than Touch ID, so we believe this is the right technology
0:34:51to use going forward. And that I could buy. And I could also say, you know, there could
0:34:57be an elegant way to make this work for the thousands of apps that already implement Touch
0:35:04ID and maybe the same API used for Touch ID authentication could automatically fall back
0:35:08onto Face ID when you're using an iPhone 8. So right at launch, one password without doing
0:35:14any work, for example, could support Face ID with the same Touch ID dialogue.
0:35:18It's just the same call, like it's just the same API called and Apple just throws up the
0:35:21new UI that they've built.
0:35:22I think so.
0:35:23Like that's how that will work.
0:35:25So that could work.
0:35:27You know, I think part of me, the problem that I have right now is that I'm feeling
0:35:33the same way that older people felt about the headphone jack going away, because Touch
0:35:41Touch ID is a feature of our generation.
0:35:46And to think about it going away already, it makes me feel uneasy right now.
0:35:52And it makes me come up with all kinds of questions and reasons why it would be a bad
0:35:58idea.
0:35:59But there's also the argument that Touch ID doesn't work for everyone.
0:36:05If you have Face ID, it doesn't matter if you're sweaty or you just took a shower,
0:36:11you know, because the camera doesn't touch your skin, you're just looking at your face
0:36:14so you can always authenticate, whereas Touch ID has that kind of problems.
0:36:18And Face ID would work even if you're wearing gloves, you know, or hats or glasses.
0:36:24Once again, because it doesn't matter whether your skin is in direct contact with the sensor.
0:36:28So I could see all the arguments for that.
0:36:31just that kind of feature that I was there when it launched and it's become
0:36:36so entrenched in the way that we use iOS that it makes me skeptical to think of a
0:36:42replacement not even four, five years after the first Touch ID sensor. Again I
0:36:50should say if, and that's a big if, it's really safer and more secure than Touch
0:36:58If this face ID can provide more unique data points about whether it's really me.
0:37:05And by the way, the argument that, you know, what happens if you change your hairstyle?
0:37:08That's really not relevant because right now I'm looking at my photos app
0:37:12and the photos app with no training can recognize my face when I was 16,
0:37:18when I was bald because I was doing chemo, and when I had a beard when I did not have a beard.
0:37:23So it doesn't really matter.
0:37:25I think the things that it's looking for, like it's not looking for is your
0:37:29hair spiky, right? Like it's looking for the distance between your eyes, you know?
0:37:32Like it's, you know.
0:37:33Yeah. I can tell you myself it doesn't really matter because I've had all kinds of hairstyles.
0:37:38So that's not a problem.
0:37:39My problem is what happens in busy situations where there's a lot of people behind you.
0:37:44So that's, you know, I could think about that and maybe say, well, that's where the 3D
0:37:48stuff comes in because you have a 3D map so you can block out people in the background, for
0:37:52example, people waiting in line at the cash register or people waiting in line in the
0:37:55tube and it could identify you or it could use some kind of proximity with the Apple
0:38:00Watch. I don't know, but I'm sure that Apple can find a way to say there's 50
0:38:05people here, but the person that is trying to authenticate with Face ID is this user,
0:38:10you know, X centimeters away from the camera.
0:38:12So that I could totally believe.
0:38:14My main question is, how does it work in the physical space?
0:38:19What am I supposed to do to make this work?
0:38:22because Touch ID is a known quantity. I just have to pick up my phone. I don't
0:38:27have to look at it. I place my finger and it works. Face ID will require some kind
0:38:31of selfie-like authentication process, which I'm not sure about, but I guess
0:38:34we'll see what happens. I'm trying to be optimistic, but it's my second nature to,
0:38:39you know, find all the possible problems and consequences and how things could go
0:38:43badly. Here's the thing, right, like I've been thinking about this in the idea of
0:38:47Fraser Speer's fantastic article that's been shared a few times this week where
0:38:51Like he argues like he makes like does a review of a MacBook Pro as an iPad user like flips it on its head
0:38:58Like can the MacBook Pro replace your iPad right? Like he did this it was like in 2015
0:39:04but it's still relevant now because for some reason this has become a problem again and
0:39:08Let's imagine that we already have face detection and we're gonna move to fingerprint detection
0:39:14Right, and it's like oh I have to put like specific fingers on my phone
0:39:18Like all I have to do right now is pick it up and it unlocks
0:39:21What if my girlfriend wants to use my phone?
0:39:24She now has to like
0:39:25She has to pass it to me and I have to unlock it when she could just point it at my face and unlock it
0:39:29Like that's so much easier, right?
0:39:31Like there are just these arguments that I can imagine if this thing is great about like why it could be better
0:39:37and then I think it's fun to turn it on its head like I'm optimistic about this because I
0:39:42think that in removing the home button completely and therefore touch ID going with it and
0:39:48there are a lot of advancements in the hardware that we could get and then it's
0:39:52up to Apple to make face ID better than any thing that's come before it and
0:39:58better than touch ID and I believe that they would not remove touch ID from the
0:40:03next phone the phone that's going to be the most expensive most premium iPhone
0:40:06that's ever made if it's worse I just can't see that being a thing because
0:40:10there's no way they got to a point where they were like oh this won't work oh no
0:40:15we have nothing we have no plan B like there was always a plan B then like
0:40:21that's silly to me and my hope is that face ID was plan A touch ID was plan B
0:40:27and they just couldn't get it to work so you know I'm calling it now there's
0:40:32going to be people if face ID happens there's going to be people who 3d print
0:40:36heads like like models to test the whether it works or not and I bet that
0:40:43whatever system Apple comes up with, it's gonna have some of, you know, some clever
0:40:48workarounds for that. Like it detects, like, those minimal movements of your eyes or, you
0:40:54know, of your skin, like, sub-skin elements even.
0:40:57What about that?
0:40:58You know, it can recognize a human versus, like, a mannequin holding an iPhone. So that's
0:41:05gonna be super fascinating if it happens.
0:41:07It does potentially open up an opportunity on the Mac as well.
0:41:13If Touch ID is going to be married to the Touch Bar on the Mac and the Touch Bar is dead in the water
0:41:19or difficult to implement an external Bluetooth keyboard,
0:41:22I would imagine that they could put whatever this face sensing deal is.
0:41:27And we haven't really gotten into that. Surely it's more than just the camera.
0:41:30But that could come to the Mac and be really interesting
0:41:35if Touch ID is indeed getting replaced with some sort of facial thing.
0:41:39You could add that to an iMac and it doesn't matter what keyboard you have.
0:41:42Because every Mac already has a camera on the front of it anyway, right?
0:41:46Right. And I truly believe that this would be something more than just the camera,
0:41:50that it's using IR or something else. But you already have
0:41:54some infrastructure there. You could put the Secure Enclave in there somewhere and you could,
0:41:58you know every Mac has a bezel around the screen and there's already a camera
0:42:02up there just put something else up there too. So that's if this is the case
0:42:07that Touch ID is going to be kind of pushed aside for some sort of facial
0:42:12thing I would be very interested to see where that expands in the future to
0:42:16other products besides just the iPhone. All right this episode is also brought
0:42:21to you by our friends over at Hover. When you have a great idea for your next
0:42:27to
0:42:41by the way, you want to have a great email address. There is a bunch of different reasons
0:42:45to get a great domain and finding that perfect domain is so easy with Hover. They have over
0:42:52400 domain extensions that you can end your domain with, all of the classics that you're
0:42:56used to and all the crazy ones as well. And once you get your domain, you can use it to
0:43:00get an on-brand or maybe just a more professional email address. It is nice to have an email
0:43:06address that has a personalised domain rather than like Hotmail or something. And all of
0:43:10this stuff can be super easy to set up with hover and any email address that you create
0:43:16will work with whatever email programs you're already using and they'll be able to help
0:43:20you with that because they have fantastic features like their support team right their
0:43:23support team is always there for you you can give them a call they have an actual person
0:43:28that's going to pick up the phone they'll answer whatever questions you might have there
0:43:31are no annoying phone trees or being transferred to another department for you to deal with
0:43:36they also have hover connect which lets you set up your domain in just a few clicks with
0:43:40any
0:43:59Thank you so much to Hover for their support. Hover, domain names for your ideas. Thank
0:44:03you so much to Hover for supporting this show and Relay FM.
0:44:07Steven, we were talking about the reviews of the Echo Show last week and since that
0:44:13episode, yours was received. You now have an Echo Show at home and I was saying on the
0:44:21episode that I'm reserving most of my judgement until I hear what you have to say. So what's
0:44:27going on?
0:44:28That's a real vote of confidence.
0:44:31- I believe in you.
0:44:33- So I would, I would,
0:44:35I'm going to categorize my comments
0:44:38in sort of three buckets,
0:44:41hardware, software, and potential.
0:44:44So no doubt people have seen photos of this thing.
0:44:48It is not the prettiest thing in my household.
0:44:51Unfortunately, the screen is bright.
0:44:54I actually had a video call on it with Jason Snell,
0:44:58and we both sort of commented like
0:45:00how deep the screen looks recessed,
0:45:02like it's not laminated to the cover,
0:45:03like you're used to on an iPad or an iPhone.
0:45:05But it's really bright, it's really clear.
0:45:07The speakers are incredible,
0:45:08way better than the regular Echo.
0:45:11Definitely better than the Dot.
0:45:13I mean, I've got a Dot here on my desk,
0:45:14and the speaker is just fine for the voice,
0:45:18but that's about it.
0:45:19Interestingly, I noticed during the boot-up sequence
0:45:23a little flash of the Intel inside logo which I did not anticipate. I figured these things
0:45:28would have some sort of ARM processor in them.
0:45:30How peculiar to see that!
0:45:32I don't know what's in there. I tried to do some digging and there's not a lot of information
0:45:36on the web about what Intel processors are in here. Maybe that's where all the Xeons
0:45:40went that Apple didn't use.
0:45:43Does it have a Pentium 3 sticker also in the back?
0:45:46That'd be awesome. That'd be incredible. So it's got Intel inside.
0:45:52The microphone array and everything seems to work just as well as the regular echo.
0:45:59It is more directional in a way, you know, the echo and the dot being cylinders.
0:46:05The sound comes out of them the same in every direction.
0:46:08So you could put it kind of anywhere in the room.
0:46:11The speakers on the show fire forward, which makes sense because you have a screen, right?
0:46:15You're not going to point the screen at the wall.
0:46:18But it's something to be aware of as you set this thing up that you may need to turn it
0:46:22or have it slightly different if you're just dropping in where a regular Echo was.
0:46:28And one thing that Amazon's done that I think is incredible, and I think like Microsoft
0:46:32did this with the Xbox and now the Xbox One X, the power plug is the same, and it's in
0:46:40the same place.
0:46:41And so you can unplug your Echo and plug your Echo Show in with the same power brick and
0:46:47everything.
0:46:48You don't have to change anything out.
0:46:49you know stuff on the kitchen counter or an entertainment center somewhere you're
0:46:52not digging around you can just unplug it and plug the same little barrel
0:46:56connector in which I love those those little details that hardware makers do
0:47:00and Amazon gets a lot of that stuff right honestly. So the hardware is not
0:47:06much to look at but what is there is nice it does look kind of like a little
0:47:09old TV set but got a little a little more bulk than I had anticipated I may
0:47:16will find it for the show notes. I put a picture on Instagram of it and the
0:47:19molar Mac which is like the super ugly Mac Apple made in the 90s and they kind
0:47:24of look the same from a certain angle and so that's that's not great but it is
0:47:29what it is I guess. The the touchscreen itself kind of moving from hardware to
0:47:36software enables a lot of nice stuff and where I was immediately impressed with
0:47:41it this oh this is so much better than the regular echo experience was
0:47:44connected to Wi-Fi. So Amazon does have this thing if you have an account set up
0:47:48with an echo that you can have Amazon hold on to your wireless password. I
0:47:53don't have that enabled on my account for hopefully obvious reasons so I had
0:47:58to type in my Wi-Fi password and instead of like getting the echo app out, I
0:48:03actually pulled my phone out while the thing was booting up thinking I'm gonna
0:48:06need the phone to configure this and you don't because it has a screen you can
0:48:10just you know type in your Wi-Fi password. It was kind of great if you're
0:48:15used to the normal echo interface. The first thing, one of the first things I
0:48:21tried of course was the video calling. There's no interface to this and so I
0:48:27just said friend in a tube video call Jason Snow and it did it. Like I just
0:48:35tried the first thing that came to mind and it worked and I was impressed that
0:48:38that it understood that I got the vocabulary right
0:48:40by basically guessing.
0:48:41The call was fine.
0:48:44It did freeze at first, but then it was okay.
0:48:47And I'm willing to like, this was very early,
0:48:49like we got ours on like the first or second day
0:48:50they were out.
0:48:51I haven't had that problem on subsequent calls,
0:48:54so maybe it was just a hiccup.
0:48:56But Jason Snell had lifted his arms in the air
0:48:58and then he was frozen that way for a couple minutes,
0:49:00which was pretty funny.
0:49:01- Wish you got a picture of that.
0:49:03- It was pretty, I should have, yeah, I should have.
0:49:05He was just like standing with his arms
0:49:06raised high in victory.
0:49:08But the call quality and stuff is fine,
0:49:11which you expect from FaceTime or Skype on a good day.
0:49:14The software benefits from all the work Amazon does
0:49:20in the background, so it knew who Jason Snell was
0:49:24because I had already given them all my contacts, sorry.
0:49:29- Oh man. - Part of the deal.
0:49:31Yeah, if Amazon didn't know your email address, real sorry,
0:49:35but it is what it is.
0:49:37but it knew who he was because he already had access to that.
0:49:41Likewise, I didn't have to go in and set up Spotify
0:49:44because my Echo system already knows
0:49:48about my Spotify account.
0:49:49So I could just tell it, friend in a tube,
0:49:51play polka music and it knew what to do
0:49:53because it was already hooked up.
0:49:54- Polka, polka, polka.
0:49:57- That's right.
0:49:57Amazon does a really good job with that stuff.
0:49:59Kind of how Apple does with iCloud,
0:50:01if you signed iCloud in your account,
0:50:03certain things take place.
0:50:04It's the same with all this stuff.
0:50:07But unfortunately it's not all puppies and rainbows
0:50:11when it comes to the software on this thing.
0:50:14And I will preface my complaints by saying two things.
0:50:19A, I know it's early days,
0:50:21and Amazon generally does a pretty decent job
0:50:25at updating things,
0:50:27and there's some things on the Echo originally
0:50:31that they have resolved,
0:50:32things on the Kindles have gotten better over time.
0:50:35So I understand that my complaints could be resolved
0:50:37with future software updates, and indeed I hope they are.
0:50:41Two, there is an inherent weirdness
0:50:43in reviewing something over such a short period of time.
0:50:48And Dan Morin actually was giving me a hard time about this,
0:50:51about trying to make a decision on this product so early
0:50:54because it is so different than the other Echo.
0:50:56I appreciate that thought, but I also disagree
0:51:00with it a little bit that there are some things
0:51:02that you're just gonna know if you like or not.
0:51:04and the Echo Show is one of those things for me.
0:51:08- Like the Logitech Slim Combo.
0:51:09- Huh?
0:51:11- Like the Logitech Slim Combo.
0:51:12- Yeah, exactly.
0:51:13Exactly.
0:51:14It doesn't take long to realize
0:51:15that thing is a piece of garbage.
0:51:17So a couple things in the software that I really don't like.
0:51:21First, every screen has a hint.
0:51:25So if you are currently playing your polka music,
0:51:30it's got the album artwork, it's like an iPod, right?
0:51:33got all your your music information on it. Then there's a little line and it
0:51:37says try, you know, wake word, show me my photo albums. It's like I don't need a
0:51:45hint when I'm listening to music to try something else. Also, lol, no one uses
0:51:51Amazon photo service. But so if I kind of like advertising in a way for service
0:51:57that no one's using but also like explain to you what else your device can
0:52:01do and almost every single screen on the Echo Show has this little bottom third
0:52:07of saying "hey try this other voice command" and I appreciate the the
0:52:13helpfulness that they're trying like they're trying to surface new commands
0:52:16because there's no interface to this thing right I just had to say video call
0:52:19Jason Snell for it to work because I couldn't there's no interface to like go
0:52:22and do that thing but the trade-off is I feel like I'm being like annoyed to
0:52:29by the tips app on iOS 11 and I could not find a way to turn this off in
0:52:33settings which is a bummer. I wonder if it's something that goes away after a
0:52:37certain period of time. It may and I have not like sat down and like mapped out
0:52:42the frequency of it so I don't I don't know to be honest with you I would hope
0:52:46that it would it would trail off of hey once you've said you know X number of
0:52:50commands to this thing then slowly fade these away but for now at least starting
0:52:57off you know the first week or so of use there they're not there or they're there
0:53:02all the time excuse me. The other big thing that that that bothers me about
0:53:07the Echo Show is it is very attention hungry and and what I mean by that the
0:53:15Echo itself the Echo Dot and Siri and you know Bixby I guess if you have a
0:53:22Samsung phone, the Google Assistant, all of these things are kind of they're kind
0:53:27of in waiting and what I mean by that is that they are they are there for you
0:53:32when you want them to do something or when you need something but they're not
0:53:37reminding you of their presence. So the echo if I'm not talking to it the LEDs
0:53:43are off it's just sitting on my kitchen counter and you may know what it is if
0:53:48if you come into my house,
0:53:49or you may think it's a weird salt and pepper grinder
0:53:52or something, it's just an object sitting on the counter.
0:53:55Not necessarily the prettiest thing,
0:53:57but it's very inoffensive.
0:54:00If you're not using Siri, it's just locked away
0:54:01behind the home button or the wake word,
0:54:03same with the Google Assistant.
0:54:05But the show is not that way.
0:54:07The show has a screen, and that screen is always on,
0:54:11and you can fall back to a screensaver of just a clock,
0:54:15and if you put the thing in do not disturb mode,
0:54:17it goes to the clock and stays there.
0:54:20But every time you walk by it,
0:54:23or if you're like me and it's in your office,
0:54:25that's only 200 square feet,
0:54:27it detects your motion and so the screen's always awake.
0:54:30And what it's doing is showing,
0:54:32Amazon, I believe they're called HomeCards,
0:54:34and these cards kinda cycle through on a carousel,
0:54:37and I care about none of them.
0:54:39So you can tell it,
0:54:40"Hey, put my next calendar appointment in there,"
0:54:43if I use Google Calendar,
0:54:45"put these things in here."
0:54:46But it's also a lot of news, a lot of entertainment stuff
0:54:51that honestly I just don't care about.
0:54:53I don't care about,
0:54:55like when it was like some new thing on Instagram
0:55:00that was trending or what Kanye was doing
0:55:02for the Fourth of July, I just don't care.
0:55:04And I don't want that stuff shown to me.
0:55:08I don't want to see it.
0:55:13And there's no way to like, again, to turn that off.
0:55:15There's no way to tell it, just go to the clock and stay there unless you put it in
0:55:19Do Not Disturb.
0:55:20So it is, again, it is trying to pull your attention into it when all these other voice
0:55:27assistants are just kind of laying and waiting.
0:55:29Does that make sense?
0:55:30Like, does that framing make sense to you, Myke?
0:55:32Yeah, and I don't like the sound of it.
0:55:35I don't know if I'd assumed that the screen was always on.
0:55:40I did but just thinking of that is not what I like about my Echo in that as you say I
0:55:46like the idea of in waiting right that's a really good way of putting it but these things
0:55:51are just like they're like Jeeves and Jeeves is never seen right like he's just you know
0:55:56when you need him he'll come and help you out but like if you don't need him he's not
0:55:59going to be there like holding up a big sign saying like hey would you like me to get the
0:56:04quaggy master I don't know why I'm doing this but that's kind of where my mind went to I
0:56:09I don't like the idea of it just being this persistent screen that has stuff showing on
0:56:14it when that stuff a lot of the time feels like it might be like advertisements to use
0:56:19it more, right?
0:56:20Like come and do this with me.
0:56:23Like I don't need you to do that.
0:56:25Like I appreciate the ability of trying to teach someone how to use the device a little
0:56:31bit better but do that in context.
0:56:35So if I've asked you a question about photos, give me some other options for it or let me
0:56:40tell you that I need help with something.
0:56:42I don't know, there's something about like, as you say, when you're doing one thing and
0:56:46it's trying to give you a tip about another piece of functionality that's completely unrelated,
0:56:49it feels a little bit redundant and needy in a way that I'm not keen on for a device
0:56:57like this.
0:56:58I don't need neediness from my voice assistant devices.
0:57:04Yeah, the, you know, someone in the chat were asking what I want to see on that home screen.
0:57:10The calendar stuff is nice.
0:57:13But you know, I like the clock.
0:57:15I like to have weather, which I think you can tell the weather to be there.
0:57:18But I would like that stuff to be more sort of ambient.
0:57:22Like these, these tiles don't need to rotate around on a carousel.
0:57:25Like every time the thing moves, it catches my eye.
0:57:28And I don't, my wife did not want that in the kitchen, which is why this thing is in
0:57:33in the office and I don't want it here in my office
0:57:37catching my eye every time.
0:57:37Like I have a digital clock in my office
0:57:40and it shows the weather and like some stats
0:57:44like Twitter followers and YouTube subscribers and stuff.
0:57:46Like it was Christmas gift, I really like it.
0:57:48It's on the other side of my office.
0:57:50I can't see it from my desk
0:57:52because every time the thing moves, it catches my eye
0:57:56and I, you know, maybe I'm just a squirrel
0:58:00attracted to shiny things but that is really distracting
0:58:02when I'm trying to write or trying to edit.
0:58:04And this thing is like, every time you walk by it,
0:58:06it's got something new to show you.
0:58:08And there may be a place for that in the kitchen, right?
0:58:12Like having a screen that surfaces things.
0:58:15But I, and I think my household,
0:58:16I'll speak for my wife as well,
0:58:18we have come to appreciate the Echo
0:58:19because it's not there unless you start the conversation.
0:58:24And that difference may seem subtle,
0:58:27but I think it's really profound
0:58:28that something is always responsive
0:58:30as to something that is trying to initiate
0:58:34some sort of interaction.
0:58:35- Seems like you're not too hot on this in the overall.
0:58:38Like I can't imagine you're gonna be keeping this thing.
0:58:40- I'm not.
0:58:41I'm not keeping it.
0:58:42I see, so the third part of the view is potential.
0:58:49If they solve the software issues, that's great.
0:58:52I can see a real potential of things like the,
0:58:55and it's been mocked a lot,
0:58:57And there's been a lot of Apple commentators
0:59:02really down on this feature of being able to,
0:59:04the drop-in, right?
0:59:05So Myke, if I gave you permission,
0:59:08underline all of those words,
0:59:10then you can start a video call
0:59:13and I don't have to approve it.
0:59:14You can just show up.
0:59:16You know what, if you don't want that,
0:59:17don't opt into it.
0:59:19It's not a hard thing.
0:59:19You don't have to write a blog post burning Amazon down.
0:59:21You don't opt into it.
0:59:22But there are times and relationships, I think,
0:59:25that would benefit from that.
0:59:26I think Amazon's done a pretty good job
0:59:27at outlining what those would be.
0:59:29But that stuff is promising.
0:59:33It sounds great.
0:59:34Having a screen is nice.
0:59:35Like having a little news or having weather,
0:59:37that sort of stuff is nice.
0:59:39But I think Amazon has a long way in going
0:59:43and understanding what customers like
0:59:45about the Echo products and how the Echo Show
0:59:49breaks a lot of those things, at least for users like me.
0:59:53And I think the Echo Show is salvageable.
0:59:56Like I don't think it's a dead end product,
0:59:58but I think Amazon sort of missed the mark
1:00:00in understanding what people like about the service.
1:00:03And it's too pushy and it's too in your face.
1:00:06And if they can dial some of that back
1:00:07and the screen can be there when you want it,
1:00:10but not all the time, I think I at least would find it
1:00:13a much more attractive offering.
1:00:14But for now I'd prefer my $200 back or whatever it was.
1:00:18So I'm returning it.
1:00:19- It really sounds to me like what I expected
1:00:22that it adds a bunch of functionality, which is interesting. It's just not the stuff I
1:00:26want to do. And everything that I want to do with that device is already in the one
1:00:31I own. So I'm all good. Yes. Yeah, you know, Jason talked about he has some security cameras
1:00:40that the echo show can show him a live feed from the camera like that sort of stuff is
1:00:44cool, like do things that you can't do on a regular echo. But the other stuff I think
1:00:49I guess the water's just too muddy now for me.
1:00:52- All right, today's show is also brought to you by Igloo,
1:00:56a digital workplace platform that enhances
1:00:59your corporate culture, improving how you work
1:01:02with your teams and the people inside of your organization.
1:01:05Igloo will help connect your team to three important things,
1:01:09the people inside of your company,
1:01:10the information that you need to share,
1:01:12and the processes that you have to go to.
1:01:14With Igloo, your people have access to what they need
1:01:16using the tools that they already know
1:01:18a host of app integrations right inside of their igloo.
1:01:21So you get to be able to share everything you do in your company.
1:01:25One of my favourite thoughts about this is when I used to work in a big company everybody
1:01:28needed to sign a piece of paper to say that they understood some kind of safety procedure
1:01:32and somebody had to physically walk it around the whole department and get people to sign
1:01:35it.
1:01:36With igloo you don't need that sort of stuff because people can just acknowledge that they've
1:01:38seen something.
1:01:39You can send out everything, you can make sure everybody's seen it and then you're all
1:01:42set and ready to go.
1:01:44But you can also make sure that people are comfortable working within igloo because they're
1:01:48using the devices that they love to use, they're not being forced to use a certain browser
1:01:52or a certain machine and they're also able to integrate with the applications and services
1:01:57that they're using to get their other pieces of their work done.
1:02:00This is powerful stuff.
1:02:02Igloo can also be customized to represent your brand and culture.
1:02:05It helps define how your company operates.
1:02:09To enhance your processes and your culture inside of your organization there are four
1:02:12things you need to think about. Communication, collaboration, knowledge management and the
1:02:16workflow of your team. This is what you can get started on with igloo. That is what it's
1:02:21focused on enhancing, building and maybe even fixing. Igloo is a modern intranet designed
1:02:27to keep everyone on the same page. Try igloo for free with no obligation to continue after
1:02:31your trial. Go to igloosoftware.com/connected now to find out more and sign up. We thank
1:02:36igloo for their support of this show.
1:02:40So as we are still in the phase, I believe that Federico is still in the phase of his
1:02:44life in which he is devouring videos of sessions from WWDC?
1:02:50No, not anymore, surprisingly.
1:02:52Oh, congratulations, you're out of it?
1:02:55I have started writing the review.
1:02:59So you're now at that stage which means that you've watched every single second of every
1:03:03single session.
1:03:05I know that's what you've done.
1:03:06Every session has been consumed.
1:03:07Well, 50.
1:03:09No, I think it was all of them.
1:03:10I think you're being modest.
1:03:11Every single session.
1:03:12That's why I heard you watched them twice and has done that since WWDC.
1:03:17So Tichi teaches today, which is the segment that I have given this a name for and potentially
1:03:22maybe the last one, but at least I have a name for it now.
1:03:25Tichi teaches us today CoreML.
1:03:29What is CoreML Federico?
1:03:31So CoreML, it sounds like caramel, but it's not that. It's not caramel for AI. It's a
1:03:39different thing.
1:03:40I didn't realize that. That's so amazing.
1:03:45CoreML is, I'm afraid to disappoint you, Myke.
1:03:49I can't hear it any other way now.
1:03:51Yeah, I know, caramel. So CoreML, it's not the deep and open framework that you might
1:03:57expect.
1:03:58Ugh.
1:03:59It's really more of a...
1:04:00If it's not deep and open, who cares?
1:04:02Yeah, well, what do we need it for?
1:04:04Well, I'm sorry, you can end the show.
1:04:06It's more of a set of
1:04:08developer tools for
1:04:10machine learning. It's not a framework
1:04:12to write, to develop
1:04:14actual machine learning
1:04:16stuff. So, here's
1:04:18how it works. Machine learning is
1:04:20at least the way that Apple is doing it. It's based on
1:04:22models. And models are
1:04:24think of them as files
1:04:26of code. It's actual code
1:04:28that you download from public sources, in this case for iOS, and there are a
1:04:34variety of models created with other tools such as Keras or Caffe or there's
1:04:41one called Scikit, I think. So you download these models and they are code
1:04:46made by others to describe a series of objects or scenes or items, stuff that
1:04:56you want machine learning to recognize and patterns that you want your
1:05:01software to understand. So the way that Apple has done it, they have this format.
1:05:04So Core ML is largely a format and a set of developer APIs that allows you to
1:05:10take in these already trained models, so this code that has already been taught
1:05:15how to recognize specific elements of everyday life. I'm simplifying here but
1:05:20to understand. You take these models, Apple has a web page where they have
1:05:23some sample models that already work with Core ML and you make them work with
1:05:29Apple's format. So the Core ML format is an actual file format. You convert these
1:05:35models into the iOS Core ML format and you drop them into Xcode and you drop
1:05:40them into your app and you suddenly gain machine learning features from these
1:05:45already trained models. The way that Apple is doing it seems to me, I mean I
1:05:51don't understand this stuff. I don't know. I mean, I've taken a look at machine
1:05:54learning code and it's a bunch of mathematics that I really don't
1:05:57understand. But the way that it works is you set a goal. You're like, I want my
1:06:03photo app to recognize flowers and to recognize hotels and airports.
1:06:10I have these three different categories that I want my app to understand. So you
1:06:14find a model that is able to recognize those features in a photo. You take the
1:06:19model and you run it through the Python converter that Apple has put together
1:06:24and that generates a compatible CoreML file. You drop it into Xcode and your app
1:06:31can now identify flowers, airports and hotels because of the CoreML
1:06:37tools. At a higher level there's a bunch of machine learning APIs that Apple is
1:06:45offering in iOS 11. And Core ML itself is used by Apple in Siri for translation,
1:06:50it's using the camera for, I think, for scene and object recognition, and it's
1:06:57also used, I believe, in the QuickType keyboard. So, you know, you can, you know,
1:07:01the next word predictions and all the suggestions that you get in the
1:07:05keyboard are powered by Core ML. Below Core ML on iOS 11 you find the more
1:07:10specific frameworks, such as the Vision API, so computer vision, it can recognize
1:07:14faces, scenes, objects, rectangles, whatever, and you have the NLP API which
1:07:21is the natural language processing which can do a bunch of things like it can
1:07:25recognize sentences, it can recognize individual entities like verbs and
1:07:30pronouns and nouns, but these more specific APIs they are an implementation
1:07:37of machine learning and they depend on core ML for them to work. So really what
1:07:44What Apple has done here is, I saw someone say Apple has made the PDF of machine learning
1:07:51with Core ML and I sort of understand the idea.
1:07:55I mean it's not as easy as opening a PDF of course.
1:07:58You still need to write a bunch of code and you still need to understand what you're doing.
1:08:02It's not like you're taking a Core ML file and you copy and paste into Xcode.
1:08:06Boom, machine learning, it's happening.
1:08:07It's not that easy.
1:08:08Are you sure though?
1:08:09It sounds like it is that easy.
1:08:11That's what I'm getting from what you're explaining.
1:08:13It is much easier for developers to find a model trained by someone else, publicly available,
1:08:22open source, and use it in an iOS app.
1:08:25The fact that Apple is providing some models for free, some of them coming from Google,
1:08:29I believe, is a testament to the fact that they want to make it easy for you to get started.
1:08:36I'm interested to see how developers are going to take advantage of this, in the sense of
1:08:44what are the possible implementations that we're going to see as users, what does this
1:08:48mean for us.
1:08:50And I think stuff like photos and translations, the basics of recognizing things in a picture
1:08:58or understanding what a string of text means, that stuff I think we're going to see plenty
1:09:04of it.
1:09:05we're gonna get a bunch of enhancements for the apps that we already use starting this fall.
1:09:10We're gonna get, for example, your calendar now can make better suggestions for the locations of events or the times of
1:09:17specific events or we're gonna see apps that can
1:09:20automatically sort our photo libraries, you know, in a way that maybe not even the Apple Photos app does. We're gonna get, you know, all these
1:09:28image recognition,
1:09:31summarizing text,
1:09:32doing translation, maybe even if you could... the thing is you can even combine
1:09:37these APIs together. So for example, you can recognize some text using NLP, using natural language processing.
1:09:45Then you take that
1:09:47recognized text, you pass it to Core ML and you say things like, "Well, here's some text.
1:09:53I know that it's in English and I know that it's got two verbs and
1:09:56and the name of a person named Myke,
1:10:00and Myke says he's going to a concert,
1:10:02can you analyze the sentiment of this text?
1:10:05And Core ML will come back at you and say,
1:10:08well, it appears that Myke is happy,
1:10:09so maybe you wanna show an happy face emoji.
1:10:11That kind of stuff.
1:10:12You can mix and match these APIs,
1:10:15and you can do most of this work using train models
1:10:18that you can already find on the open web.
1:10:22You can take them, you can convert to the Core ML format,
1:10:25use them in Xcode and let your app take advantage of those predictions, that analysis, all those features
1:10:31that Xcode and the iOS SDK wouldn't normally give you for free.
1:10:36And then you can combine it with this other stuff like NLP and Vision.
1:10:40So I think there's a lot of potential here for even the smaller features,
1:10:47like the smaller additions, like now Fantastical is smarter when it gives you recommendations for locations.
1:10:53or now you can have a summary of your agenda based on the things that you want to know.
1:10:59And I think we're going to get this smaller, but yet so useful, enhancement.
1:11:04And I'm really, really curious to see how this goes.
1:11:07Did that make any sense?
1:11:09It does.
1:11:10It does.
1:11:12I am struggling to get my head around this one in ways that I haven't in the previous
1:11:17things that you've been talking about.
1:11:19And I've been that way since I first heard about CoreML because I don't fully understand
1:11:26how this framework is able to do anything.
1:11:30If you give it like this just list of things, how does it know what?
1:11:34It seems confusing to me.
1:11:36So the basic idea is that it's all in the original code, the original model.
1:11:41So imagine there's a long file with millions of lines of code that has been trained with
1:11:48the type of neural network to understand what an apple looks like or what a
1:11:55bedroom looks like. And that is actual code. There's a programmer who wrote that
1:12:00and taught the network how to recognize those items in a picture. And you take
1:12:06that knowledge, you take that code and you put it into an iOS compatible format,
1:12:11which is CoreML, and you start using it into your app. So CoreML, you know,
1:12:16it's a two-way communication. Your app as a developer, Myke, you're putting together
1:12:19one, two, three AI so you can pass text or you can pass an image or you
1:12:26can pass data, you can pass a dictionary. So you take some input from
1:12:30your app and you say "CoreML, help me figure out what is going on here" and the
1:12:35framework is gonna do its processing, it's based on
1:12:40metal, it's based on accelerate, so these high-performance frameworks that take
1:12:44you know, a fraction of a second to understand scene recognition and that kind of stuff.
1:12:48And it comes back with a message that says, "I believe there's a 90% accuracy that this
1:12:52picture is showing an apple or a horse or a mountain." And you take that and you say,
1:12:57"Well, I believe, as a user, here's your message. We believe this is a horse." But the essence
1:13:05of the actual machine learning is happening in the model. And that's the most important
1:13:12part. But what Apple is doing here is it's making it easy for developers to take these models that
1:13:19have been going around for a couple of years now, maybe even more, and to use them on iOS.
1:13:24It was also tricky for me to understand because I was like, "So where does the actual machine
1:13:31learning occur?" And it's not in the model. What CoreML does is it's a Rosetta Stone for machine
1:13:39learning and apps and it's helping them communicate and translate those inputs
1:13:45and commands from the machine learning code into data that can be used by an
1:13:52app on iOS. So it's this framework right in the middle between the algorithm and
1:13:58your app. That's what it is. Let me see if I can say that back to you in a way that
1:14:04that I think I've understood it with an example.
1:14:07So like Apple's photo recognition stuff, right?
1:14:11They had a bunch of data, which was saying,
1:14:16this is what this picture is. This is what this picture is. You know,
1:14:19this is what a horse is. This is what an Apple is,
1:14:21but they need something in the middle to translate that
1:14:26information back to the photos.app and CoreML is the thing
1:14:31that can like, it takes all of the information
1:14:35and can process it.
1:14:36So when the app asks a question, Core ML gives the answer
1:14:41because it has all of the information.
1:14:44- Yes. - Okay.
1:14:45So everything that goes through Core ML
1:14:48has to have some kind of huge data set
1:14:51that Core ML can read from.
1:14:53And it can then interpret or extrapolate from it
1:14:59to give answers to questions that the application may have?
1:15:03- Basically, and we're oversimplifying here,
1:15:06but yes, that's what it does.
1:15:08- Yeah, we have to oversimplify
1:15:09'cause this is way over my pay grade.
1:15:11- Yes, we uses the model, which is the actual code,
1:15:16the programming that went into machine learning,
1:15:20and it communicates with iOS back and forth between the two,
1:15:23and it spits out a reasonable answer for your app to use.
1:15:28What sorts of things do you see developers being able to do now that they have this as
1:15:34opposed to in the past when this was more difficult or out of the reach of more people?
1:15:40Well, translation services that used in the past like a web service or a web API, now
1:15:46they can do, if they have the right models and if they combine the NLP API and Core ML
1:15:53together, they can now do all of this stuff offline with local processing and it takes
1:15:58couple of seconds, maybe even less. So translation, doing things like summarizing text or of course
1:16:06categorizing pictures based on scene or objects recognizing a picture and then you can do things
1:16:14like, and this is all done locally, there's no web API, you don't have to be connected to the
1:16:18internet for this to work, you can do things like show me photos where I'm happy or show me photos
1:16:24where another person was crying. So you can do sentiment analysis.
1:16:29God! Don't make that such!
1:16:33I mean, maybe to an extent you could do things like, I want to help me find all of my diary
1:16:42entries where I was sad and I was typing in Italian. So if I have a diary, sometimes I
1:16:50I write in Italian, other times I write in English, and I can find the said entries of
1:16:54my diary in a specific language. Or maybe you could even do things like audio tagging,
1:16:59so you can process audio and recognize things in audio, or you could do things like, I mean,
1:17:07understanding a specific speaker, for example, in an audio stream, like show me the bits
1:17:13where Myke is talking, you know, all that kind of stuff.
1:17:17needs that. All apps should use that feature, the "when Myke is talking" feature.
1:17:21Yeah, it's a very... it's a wild west right now, really. It's all up to
1:17:29the developers imagination, the models they can find. And there's the
1:17:33criticism that is going on is that Apple is not making a real machine learning
1:17:37development tool. It's more of an API that takes in models from other
1:17:43other proper tools and makes them compatible with iOS,
1:17:47you know, it actually includes the models
1:17:49into the binary of an app.
1:17:51So, you know, like at runtime, the model is already there.
1:17:55And it's not like a real program
1:17:58that you open on your computer and you're like,
1:17:59"Okay, I want to write machine learning code."
1:18:02That's not what it does.
1:18:03It's a way to take the model, take the code,
1:18:06and make it work with iOS.
1:18:08- Yeah, I understand that criticism, right?
1:18:10That kind of cheat in a way,
1:18:12but they're also making something that sounds pretty impressive.
1:18:15But yeah, they are getting around doing that stuff themselves.
1:18:19Like, it's not TensorFlow, you know.
1:18:21It's not what Google is doing.
1:18:23It's a different approach.
1:18:24Yeah.
1:18:25Thank you for teaching, Tichi.
1:18:28Sure. I feel educated.
1:18:30Thank you.
1:18:31Is there anything else?
1:18:32Is there any other things that you want to do?
1:18:34Like, are there other things that you can talk about?
1:18:38There are other things.
1:18:39I need to decide, so I don't want to pre-announce it this time, because it creates an expectation.
1:18:47But there are other things that I would like to discuss with you and sort of wrap my head
1:18:50around because it can be difficult to understand these ideas just by myself.
1:18:55So there will be more.
1:18:56I think if you run out of stuff we can pivot and I can teach you guys about Apple Talk
1:19:02Nodes.
1:19:03Sure.
1:19:04If you want to find our show notes for this week just go to relay.fm/connected/149.
1:19:07I'd like to thank our sponsors again for helping support this episode.
1:19:11So a huge thanks to Igloo, Hover and Mac Weldon.
1:19:15If you want to find us online, there's a few places you can do that.
1:19:18You can go to 512pixels.net and @ismh on Twitter for all of Steven's stuff and @fotichiviticci
1:19:25and maxlories.net for all of Federico's.
1:19:28I am i-mike, i-m-y-k-e on Twitter.
1:19:31Thank you so much for listening to this week's show and we'll be back next time.
1:19:36Until then, say goodbye guys.
1:19:37Arrivederci.
1:19:38Adios.