Podcast Search

The Talk Show

158: Live From WWDC 2016 With Guests Phil Schiller and Craig Federighi

Select a time to play from there. The line being spoken is highlighted as the audio plays.

0:00:00Ladies and gentlemen, welcome to Mezzanine.

0:00:07Won't you please take your seats and silence your cell phones.

0:00:14Daring Fireball Productions, in association with the Daring Fireball Company LLC,

0:00:23is delighted to welcome you to a daring fireball presentation of the talk show

0:00:31live from WWDC 2016 and now won't you please welcome your host John Gruber

0:00:44thank you Paul I actually needed that reminder my cell phone was not on on

0:00:51silent thank you Mike I will start by thanking our sponsors this is a fifth

0:01:03year actually we've been doing I've been doing this show at least fourth time

0:01:06here at mezzanine and I think that MailChimp has been sponsoring our bar all

0:01:13along and if not it's at least as far back as I remember so it just in case it

0:01:19isn't clear the bar is on the house it is on MailChimp. MailChimp if you guys

0:01:23don't know they are an email newsletters like my friend Ben Thompson Strategery

0:01:31goes after MailChimp. They also have a bunch of new features stuff that

0:01:36integrates with online stores and integration just about any online store

0:01:42platform that you might be familiar with and then you can get your customers to

0:01:46get email when products if they're interested are available or

0:01:50Whatever great company if you need to send email

0:01:53Go to MailChimp.com and please let's hear it for them for the open bar

0:02:00Also back with us for the fourth consecutive year as a sponsor of the show is Microsoft and

0:02:15At four years it's not even like a whoa that's weird Microsoft sponsoring. No, it's like awesome and and it makes total sense

0:02:22They have this website. It's gonna give you so much more information than I have time to give you now. Any dev any app

0:02:29dot-com. That's the message they're trying to give that any developer if you're working a mobile or the web

0:02:35For any type of app if you need cloud services

0:02:38It's now called the Azure App Service

0:02:41If you need that sort of stuff go check it out. Their website has so much information. Here's the funny thing

0:02:45They had the same website last year, but instead of any dev any app.com. I said any app any dev.com

0:02:52And we are in fact streaming this live

0:02:56And you know the show went on and in meantime I gave out the wrong URL for a pretty pricey sponsorship

0:03:09And what happened is there was some kid in Australia who was watching the live stream who quick like jumped on and registered the domain

0:03:17This true story, this is an absolute true story. You guys see Matt Hensing. He's here representing Microsoft. He's about this tall

0:03:27him and Hockenberry are gonna have a fight after the

0:03:30show is over

0:03:32You could ask him he'll vouch for this so they got in contact with him. We're like, oh man, that's with Microsoft

0:03:37We'd better get this domain and it was already gone and they contacted the kid and they were like, oh man

0:03:41This kid is gonna you know, he's really gonna let us have it and he was like well one of those Xboxes would be nice

0:03:46So they they sent the kid like a box like with an Xbox and like all the cool stuff that you could possibly imagine

0:03:56It goes with an Xbox and they got the domain

0:03:58So I think it's safe that you you can just go check out the information from Microsoft go to any app or any Dev.com

0:04:05Now any dev, any app.com. Microsoft, great sponsor. And then last but not least,

0:04:13we have one more sponsor. This one's new and surprisingly this is the thing

0:04:18because if you think Microsoft, well how you gonna go bigger than that? But this

0:04:20is actually one of the few, I mean I'm guessing maybe three or four

0:04:25corporations in the world with a larger market cap than Microsoft. It's Meh.com.

0:04:35[Laughter]

0:04:37Matt.com is the store that I would run if I were gonna run like an online store.

0:04:43And I mean, let me be clear. I have absolutely zero interest in running a store. It seems like a terrible job.

0:04:48[Laughter]

0:04:50And a lot of hard work and I don't like either of those things. So...

0:04:53[Laughter]

0:04:55Yeah, I'm not gonna run a store, but if I did it would be like Matt. Here's the way Matt works.

0:05:01They have one product a day. That's it. You don't even know what it is.

0:05:03You have to like go there at midnight. You find out what they're selling today. It's one thing daily deal usually at like an

0:05:08Unbelievable price. I'm I said this before I'm half worried that they're like stealing these things and

0:05:14That I don't know if me endorsing it like this makes me complicit in a crime because when you're selling like

0:05:19$120 stereo for $14

0:05:21Usually that was you know, it's like that scene in Goodfellas where they're selling cigarettes out of the back of the truck

0:05:27But what they really do the other thing they do is they just concentrate on making everything real funny

0:05:32the descriptions of the products are real funny. They have funny videos every

0:05:36day and I really do get the feeling that they'd be happy if you just go there and

0:05:40check them out every day and you never buy anything. They even, that's like the

0:05:42gimmick of the thing is like here's the product, buy or meh and you can just type

0:05:47men and they're like well that guy didn't like that. So my thanks to them.

0:05:53So last year was a little different than the previous years because we had

0:06:02actual special guest. What happened was the backstory on it is it was a week

0:06:06before WWDC and I still hadn't asked anybody to be on the show and I was

0:06:10putting it off because I kind of had it in my head that I kind of wanted to see

0:06:14if I could get Phil and I put it off because I didn't want to hear no and it

0:06:20was like a week before I was like well this is ridiculous I'll just ask and so

0:06:23I sent an email to Steve Dowling I said look this is probably ridiculous and so

0:06:26just feel free to say no. But I do this show every year and I think it'd be

0:06:32really cool. I think it would work really well if Phil Schiller came on and the

0:06:36day after the keynote we could talk about it, nerd out and go into detail

0:06:40that you can't get into in a keynote. And he wrote back and all he said was, "Not

0:06:43ridiculous. Let's talk tomorrow." And next thing you know, a week later Phil Schiller

0:06:49was screwing around, not coming out behind the curtain and making me wonder

0:06:54whether maybe he went to the bathroom, maybe we miscommunicated on what the

0:07:00cues were gonna be and it was great. I mean I don't know how many people here

0:07:04were here last year. It really was great. It was the best time I've had on stage

0:07:11in my life and then I watched the video and I didn't even really die watching

0:07:15myself. I was like this is actually pretty good and it ended and it was a

0:07:20big surprise. We kept it under wraps. Everybody seemed pleasantly surprised

0:07:24and it just made it all the more fun.

0:07:25And then the show's over, and I go backstage,

0:07:29and people are like, "Wow, that was great.

0:07:30I can't believe it. That was amazing.

0:07:32That was amazing." And I start meeting people.

0:07:34And it was about three minutes, three or four minutes

0:07:35after the end of the show when the first person said,

0:07:38"Boy, you're really gonna have a hard time

0:07:39topping that next year."

0:07:40(audience laughing)

0:07:44And I thought, "Wow, that did not occur to me

0:07:47because this week has been a blur.

0:07:51Like, I really just asked a week ago,

0:07:52and then we set this up and I've been thinking on questions.

0:07:56And you're right.

0:07:57And there's only so far up I can go.

0:08:01There's only so many different ways that we could go up.

0:08:03So one of these years, it is absolutely gonna be the case

0:08:07that it is not as good a guest as the year before.

0:08:09I mean, one of these times,

0:08:11it really is gonna be John Moltz coming out.

0:08:13And that'll be great.

0:08:15And we'll have a good show.

0:08:17I mean, there might be more people

0:08:20leaving to go to the open bar in mid show,

0:08:22which you can do by the way, please. Really run up a good tab. We're good here.

0:08:26But this is not that year. This year I think I think it's a little better.

0:08:34So this year how do you top Phil Schiller? Here's how. Ladies and

0:08:44gentlemen, please give a warm welcome to Phil Schiller and Craig Federighi.

0:08:54[ Applause ]

0:09:18True story.

0:09:19Got a text message

0:09:22about 45 minutes ago, hour ago.

0:09:25Do you guys have any food here?

0:09:27No, we don't.

0:09:29We have lots of booze.

0:09:30So when they get here, here's what Craig,

0:09:38Craig has a boxed lunch from WWDC.

0:09:41And that's what he's eating.

0:09:46And a friend of the show and announcer Paul Kefausis

0:09:49asked him, is that a WWDC box lunch?

0:09:52And the answer is--

0:09:54It aged well.

0:09:56It was an old WWDC box lunch.

0:09:59So let it be said, Apple eats their own dog food.

0:10:02[LAUGHTER]

0:10:04[APPLAUSE]

0:10:07True that.

0:10:09I can absolutely validate that for over 20 years of doing

0:10:15surveys from WWDC every single year,

0:10:18the number one complaint is the food.

0:10:20And so we resign ourselves to the fact

0:10:22that if that's the worst thing that comes out of WWDC,

0:10:26all is good.

0:10:27It's tradition, so.

0:10:31(audience laughing)

0:10:33- So I always start the show,

0:10:34I started it the same way last year,

0:10:36I always ask the guests,

0:10:36how do you think the keynote went yesterday?

0:10:39- Good audience.

0:10:40(audience laughing)

0:10:41Great crowd.

0:10:42- Great presenters.

0:10:43(audience laughing)

0:10:44missing one. Once again Phil Schiller was not on stage at the keynote. This is

0:10:50becoming a new tradition. I was teasing with Tim that between Craig and

0:10:57Jeff Williams and now Bose I don't meet the minimum height requirement to

0:11:01present. But Craig I honestly thought I talked to you yesterday briefly and I

0:11:09told you I thought you did an amazing job. I mean how many people thought Craig

0:11:14(audience cheering)

0:11:17It's not just that you're up there and you're good

0:11:20and you're covering stuff, but you covered

0:11:22like three hours of stuff in 90 minutes

0:11:26or however long you were on stage.

0:11:28- Yeah.

0:11:29(audience laughing)

0:11:31It was a lot.

0:11:32Yeah, I mean, the team did a tremendous amount of work

0:11:35and we tried to--

0:11:36- Well, the article I saw on the ringer today,

0:11:39I don't know if you saw this.

0:11:40(audience laughing)

0:11:41I'm not gonna go into details of the article,

0:11:42But the headline was, "Apple's Craig Federighi is Perfect."

0:11:45[LAUGHTER]

0:11:47[APPLAUSE]

0:11:49I read that article, and I can only

0:11:54confirm that it's half true.

0:11:55[LAUGHTER]

0:12:03So I didn't think about it yesterday,

0:12:05but today it occurred to me that there sort of

0:12:08was a recurring, overriding theme in the announcements

0:12:11yesterday, which in broad strokes is that you guys have opened up a lot of stuff

0:12:17to third-party developers that was previously reserved for Apple's first

0:12:23party code. Quickly, I might even miss some, CallKit so that VoIP apps can get

0:12:30the same lock screen privileges for incoming calls as the phone app and

0:12:35FaceTime which is yours. Messages so you WhatsApp can you can specify a contact

0:12:42when I text Craig default by going to WhatsApp instead of iMessage.

0:12:47Siri API, iMessage apps, maps, extensions and even non Mac App Store apps can now

0:12:59use Cloud Kid and a bunch of other iCloud services.

0:13:03(audience applauding)

0:13:06Is that a coincidence or is that like a strategic

0:13:11part of the plan for this year?

0:13:13- Well with iOS 8, we started that with extensions.

0:13:18Opening up like the share sheet for instance.

0:13:20For a while there it was, if we didn't build it,

0:13:21it couldn't be in the share sheet.

0:13:23And so we had to build a Twitter interface ourselves

0:13:26and a Facebook interface.

0:13:27And as of iOS 8, we started having extensions

0:13:31for extending the system with sharing, widgets.

0:13:34And so we built a lot of the technology with XPC services.

0:13:39You folks know what those are, and out of process UI,

0:13:43and all the building blocks to make this possible.

0:13:46And this year we really felt like giving the developers

0:13:50more and more opportunities to let users do

0:13:52what they want to do across all these experiences

0:13:55was a way that we could really make the platform better

0:13:58for all of our users.

0:13:59So yeah, it all came together nicely with Siri as well.

0:14:05And a big part of it, it seems to me,

0:14:07as the platforms, plural, evolve,

0:14:09because it's definitely-- especially iOS and Mac--

0:14:14what it means to have an app is more than, like on the Mac,

0:14:19OK, you launch an app and a window opens up

0:14:21and you are in this window.

0:14:23And it's yours as the developer.

0:14:25Or on iOS, it's a little simpler.

0:14:27You get the screen.

0:14:28But now, to be an app that's really taking advantage

0:14:32of the best, the newest stuff that the platform has to offer,

0:14:36you need to be inside other apps.

0:14:40You know, widgets inside iMessage.

0:14:45I think that just makes sense for mobile.

0:14:47I mean, if you have an app and the right place to interact

0:14:52is on the notification on the lock screen,

0:14:54and you don't want the user to have to unlock the phone

0:14:56and launch your app in order to get something done,

0:14:58or invoking your app with Siri is

0:15:01going to be the quickest path to getting something done.

0:15:03We want to make that possible.

0:15:05And so I think that's what you're seeing here,

0:15:07as well as, as you say, inside of Maps.

0:15:09If you want to book a ride, or you

0:15:11want to get a restaurant, or any of those things,

0:15:14it's going to just be a quicker and smoother flow

0:15:16if you're integrated into the place where the user started

0:15:19instead of requiring switching around.

0:15:20And so this is opening all that up.

0:15:22And I think developers are going to do

0:15:24a tremendous number of things with it

0:15:25that we didn't even envision, so it should be an exciting year.

0:15:28It's also just an evolution of the success of the app model, right?

0:15:33Apps took off, been wildly successful, this amazing software process, and then you want

0:15:40to have apps in your maps, you want to have apps in your Siri situations, and you want

0:15:45to have apps in your messaging, and so we like apps, we like them everywhere, we want

0:15:49to use them in many places, so to me it's an evolution of what's going on with apps

0:15:53in general. And you mentioned XPC and I know, I mean this is a fairly, fairly nerdy

0:16:00crowd, but I do think it's a years-long shift where, and my layman's

0:16:08XPC is inter-process communication and it's a way for different processes that

0:16:13can be sandboxed and all of the privacy and hey you, this process can't

0:16:19diddle with the data of this process without having it in a shared location,

0:16:24that they can still communicate with each other in a rich way. Compared to the

0:16:29old days, the Mac OS has always been extensible and whether you want to go

0:16:34back to the classic Mac OS with like a NITS or the next step days with things

0:16:42like bundles and input managers and remember in the early days of Mac OS X

0:16:45when we had the hacksies and the input managers.

0:16:48And that was, and in layman's terms,

0:16:51the fundamental differences, those were ways

0:16:53to extend apps officially or unofficially,

0:16:55where the extension code was running within the process.

0:17:00- Yeah, and from a stability point of view

0:17:02and a privacy point of view, really bad news.

0:17:05So we started years and years ago with mock messaging

0:17:11and on that we built XPC as a form of remote procedure call and asynchronous messaging,

0:17:18structured messaging thing, but we then created what we internally called XPC containers,

0:17:24which are really what you now think of as XPC services, which are the ability to package

0:17:28a whole bunch of code and let the system manage launching that code, tearing that code down

0:17:33when it needed to, but exposing services in that way.

0:17:36And that turned out to be really important, even internally within the OS.

0:17:40We were using it for quite a while within the OS before it was exposed as a mechanism

0:17:43for third parties because it allowed us to set different security boundaries around different

0:17:48– this is really getting nerdy – but –

0:17:50No, this is good!

0:17:51[Laughter]

0:17:52[Applause]

0:17:53Because if you're going to go load some image format even or run a Spotlight converter or

0:18:03something that's going to run over all your documents, you want to make sure that if that

0:18:07Nothing crashes, it doesn't crash the overall spotlight indexer or the app.

0:18:14You don't want it to have any more access to anything but the one thing it's supposed

0:18:17to have to do the job.

0:18:18So this was all part of our security and sandboxing architecture.

0:18:22But then with iOS 8, we saw the opportunity to combine that with essentially remote views,

0:18:28the ability to say that the UI you see on screen that looks like it's all from one app

0:18:32is actually composed from the main app, but also one or more XPC services serving UI into

0:18:39that and we manage all that.

0:18:40And that gives you this single experience, but where all the security boundaries and

0:18:44the stability boundaries are in place.

0:18:47And that's enabled us to take this extensibility model from something that was really Haxi-prone

0:18:53in the next step and, well, init in CDAB days.

0:19:00And make it much more stable.

0:19:01And so that's been now a building block for all these things that we're doing.

0:19:06And iOS 10 was just really stepping on the gas on the places where we could do that that

0:19:10made the biggest difference for user experience.

0:19:13One of the most surprising changes, and again, I think that this is in the spirit of openness

0:19:18or flexibility on Apple's part and relinquishing control that previously wasn't relinquished.

0:19:25And it surprised me is that you can now remove a whole bunch of the default apps on iOS from

0:19:32your home screen.

0:19:33Though you would not want to.

0:19:35If you have the freedom, just knowing you have the power that you'll never use.

0:19:41It's one of my favorite pages on the What's New site is I love the page because it even

0:19:46goes out of the way to say, you know, because of all the compression that we use and the

0:19:50the techniques that we use and the shared frameworks,

0:19:52they only take up 150 megabytes.

0:19:54(audience laughing)

0:19:56- Yeah, well, okay, so, this is true, this is true.

0:20:01We should be really clear on actually what this feature is

0:20:03and what it's not,

0:20:04'cause it's not everything you might think it is.

0:20:07So, what it is is you are removing,

0:20:10when you remove an app,

0:20:12you're removing it from the home screen,

0:20:14you're removing all the user data associated from it,

0:20:16you're removing all of the hooks that it has

0:20:19into other system services, like Siri no longer will try to use that when you talk to it and

0:20:24so forth.

0:20:25We're not actually deleting the application binary, and the reason is really twofold.

0:20:30One, they're small, but more significantly, the whole iOS security architecture around

0:20:36the system update is this one signed binary where we can verify the integrity of that

0:20:44with every update, that there's no mixing and matching going on between all of these

0:20:47different pieces.

0:20:49And so if you go and say, well, I don't like, what's an app that someone would remove?

0:20:56I'm going to get myself a couple here.

0:20:57I can't think of one.

0:20:59Stocks.

0:21:00Stocks.

0:21:01Some people don't follow the stock market.

0:21:05Fair enough.

0:21:06Some people do not follow the stock market or there's not one in their country.

0:21:08Which is good for them.

0:21:09Yes.

0:21:10Or something.

0:21:11Yes.

0:21:12And so they might remove that app.

0:21:13And when you do, it's hidden, and any user data and preferences and so forth associated

0:21:18with it's gone.

0:21:19If you want to get it back, we were thinking, well, how do we let you restore this?

0:21:23And we thought, well, people are naturally, if they want to go get it back, they're going

0:21:26to go to the App Store and search for it.

0:21:28And so you go to the App Store and search for it, and it'll show up, and you'll say

0:21:32"get," and it will reappear.

0:21:35Because that's how they know to install it.

0:21:37The download will be remarkably fast.

0:21:39Exactly.

0:21:40Compression technology.

0:21:41- Impression technology.

0:21:44Good stuff.

0:21:44- And it has led some to mistakenly report

0:21:47that we're moving these apps out of the system bundle

0:21:50and into the store for downloading,

0:21:52and that's not really the case.

0:21:54We're just making that the easy mechanism for restoring,

0:21:56is seeing it from the store side.

0:21:58But it's really still part of the system bundle.

0:22:00- Okay. - Let me set the record

0:22:01straight here. - That's interesting,

0:22:02'cause that means there won't be an update to mail

0:22:04that comes through the App Store.

0:22:05It's just like it used to be.

0:22:07It'll be part of the system update.

0:22:08- That is correct.

0:22:09Well, speaking of the App Store, this last week,

0:22:13(audience laughing)

0:22:16a week ago--

0:22:19- There's a reason I sat on this side.

0:22:21I just thought these two were gonna get totally nerd out

0:22:23and I'm just gonna let 'em have fun

0:22:25and I have no problem with that.

0:22:27- A week ago, there were a bunch of changes,

0:22:32improvements to the App Store.

0:22:34In a certain sense, you could, you know,

0:22:37or one of them, and it did not get mentioned in the keynote,

0:22:39but review times for apps submitted to the App Store

0:22:42are way faster than they used to be.

0:22:44(audience cheering)

0:22:47- We thought this was one of those cases

0:22:55where we could address a problem

0:22:56before it starts to boil over.

0:22:58(audience laughing)

0:23:00Just an anticipation of potential future.

0:23:03- For the audience at the keynote though,

0:23:08That, to not even mention that and just take that applause

0:23:11is amazing because you know that it's coming

0:23:14and that developers are pretty happy about that.

0:23:17- It would have been an easy way to get applause

0:23:20but we didn't stoop to that trick.

0:23:22So, yeah, you know, it was exactly,

0:23:25people have all these awesome conspiracy theories

0:23:27and they're fun to read but it was exactly what we said

0:23:31which was the, we were working on the keynote.

0:23:34We actually thought about having a whole developer section

0:23:37to talk about the App Store and the keynote,

0:23:39and looking at keeping it really,

0:23:41we really wanted to get done

0:23:42in just under two hours if we could.

0:23:44And you couldn't really talk about that

0:23:48in the subscription stuff and the ad search stuff

0:23:52and all that in three minutes.

0:23:54We really needed probably about 15 minutes to explain,

0:23:57and it just wasn't worth losing 15 minutes of product time

0:24:01to talk about that if we could instead just talk

0:24:04to people ahead of time.

0:24:05so we decided to do something we've never done before,

0:24:07which is before the keynote,

0:24:09so, explain some of this.

0:24:11However, it was kind of tough to do

0:24:12because here we're talking to you and a few others

0:24:15and saying, here are things we're doing for the App Store,

0:24:18knowing that we still have to come a few days later,

0:24:22you know, apps working with Siri

0:24:23and apps working with Messages,

0:24:25and these are huge impacts on developers

0:24:27and a new store for message apps were gonna come out,

0:24:31and so we couldn't really tell the whole picture

0:24:33of all the things we're doing,

0:24:34So we told sort of half of it and waited for the rest.

0:24:38Well, part of it that goes together-- so one of the

0:24:40improvements last week was search ads.

0:24:43And I noticed--

0:24:46I don't know, Jerry, if you noticed, before we came out,

0:24:48there was an ad that showed up first, as John, you did your

0:24:52ads before we started this session.

0:24:55It was really nice.

0:24:57Thank you.

0:25:02And I found two of the three were relevant

0:25:05to what we were discussing.

0:25:06(audience laughing)

0:25:09I won't, for the benefit of your advertisers,

0:25:14I won't mention which one I didn't find

0:25:15quite relevant to my interests, but.

0:25:17(audience laughing)

0:25:20- I was gonna be nice, I was gonna say.

0:25:27(audience laughing)

0:25:29I was gonna say how there's a tie-in

0:25:31you couldn't mention a week ago where the idea of the search edge is it improves discoverability

0:25:38and there's a discoverability aspect with the iMessage apps where if I send you a widget

0:25:45through an iMessage app and you don't have it yet, there's a very subtle, you know, this

0:25:50was, I forget what it exactly says, but it gives you a...

0:25:52Yeah, a couple of really interesting things that the team did in working on these message

0:25:58JibJabs is number one, that if I send you something,

0:26:02if I send you a sticker, if I send you a JibJab,

0:26:06you get to receive it and experience it

0:26:09without having to download the app.

0:26:10And so you can do that on a lot of these things,

0:26:13where some other services you're always being hit with,

0:26:16hey, download this in order to see what someone's sending you.

0:26:19The team really wanted to have a great experience

0:26:20for the receiver.

0:26:21You don't have to do that.

0:26:23However, there is attribution there,

0:26:25and you can choose to get it if you're like,

0:26:26wow, those JibJabs are really cool.

0:26:28I want to download them too and share them with friends.

0:26:31And hopefully that'll become a nice viral marketing

0:26:33in addition to other ways for users

0:26:35to discover apps and messages.

0:26:37- On search ads,

0:26:40make the case, and when we talked last week you did,

0:26:44and on the phone call I thought, yeah that makes sense.

0:26:48And then when I went and looked at my notes

0:26:50and I was like, I'm not sure I get it.

0:26:51Make the case, on this particular part,

0:26:54that the system that you guys have designed

0:26:57can and should be to the benefit of smaller indie developers

0:27:02and it's not going to be dominated by the biggest companies

0:27:07with the budgets that are more than everybody here combined?

0:27:11- So the two sort of priorities we set on the team

0:27:17as they were working on it was if we're gonna do this,

0:27:21we have to do it in a way that number one,

0:27:23protects user privacy.

0:27:25There are many ways that companies do it

0:27:27where they're not protecting privacy,

0:27:28and we need to understand that.

0:27:30And secondly, how do you do it in a way

0:27:32that gives advantages to small and indie developers,

0:27:35because it's easy to imagine a system that didn't do that.

0:27:38And so we set out to think of all the things

0:27:40we could do to make that possible.

0:27:42And there's a long list of things,

0:27:44and I won't go through all of them to bore you all,

0:27:46but there are many things.

0:27:49Things like, first of all, there's no minimum bid,

0:27:52So you don't set up a bar if you have a very small amount of money.

0:27:55You can just do what you want with the small money.

0:27:57The fact that we're going to work really hard to try to make relevance the top priority

0:28:02over bid for why something gets shown, that the users are the ultimate deciders of what

0:28:09gets shown based on their clicks or big input to what is relevant to the search result.

0:28:16The fact that we're going to work hard to try to police and improve the whole metadata

0:28:21system if we find, as it easily could, be abused to hurt developers.

0:28:27The fact that—and this has been a hotly debated thing—the fact that you can do conquesting,

0:28:33you can use someone else's brand in your ad words that you want to use, as we thought

0:28:39about it, that is more likely to benefit the small developer than the big developer, because

0:28:43the big developer isn't going to pick on a lot of small developer terms, but a small

0:28:46developer can try to latch onto a big developer's name.

0:28:49If you want to search for Angry Birds in your game, you can.

0:28:54We think that that can help them.

0:28:55The fact that there is no exclusivity.

0:28:58So a large developer cannot say, "I want to be the top bid, and I'm going to spend everything

0:29:03I can to buy up this term."

0:29:04There will be no exclusivity.

0:29:05There's going to be a rotation there.

0:29:07As that rotation appears, the relevance will help drive it further.

0:29:10We're trying everything we can.

0:29:13I think one of the best things is right now, once we're in beta throughout the summer,

0:29:18The downloads the users get from the ads are real downloads to benefit the developer, but

0:29:22we're not charging during the beta time.

0:29:24So there's a chance for everybody to get in and try it out, help us learn from it, and

0:29:29drive real downloads and real business without any marketing spend.

0:29:33So we're trying to think of things we can do, and we'll think of more.

0:29:35We'll take feedback and see what's happening and where it works and doesn't work, and who

0:29:39feels like they're getting stomped on, and we'll try to do all we can to make it better.

0:29:46And the last bit of news with the App Store changes, or the third of it, was an expansion

0:29:57of the categories that are allowed for subscriptions.

0:29:59I don't know if you noticed, but there was a little bit of confusion last week about

0:30:03the difference between apps from all categories versus all apps.

0:30:08Let me explain that.

0:30:11So our intention is exactly as we talked about, which is we're opening up a subscription model

0:30:19to all categories, so what kind of an app you make doesn't directly have an impact on

0:30:26whether you can have a subscription model or not.

0:30:30We want to open up subscriptions to all developers of all apps.

0:30:34That is the hope.

0:30:36However, there are a couple little gotchas where we have to be careful, and so that's

0:30:40That's why there's some caution here.

0:30:43Number one, if you want to create a professional app and you're going to maintain it and do

0:30:47updates and you want to have an ongoing revenue stream, that's of course an intention of this.

0:30:53But do users really want, and I'm sorry to pick on this category if somebody makes this

0:31:04app because I'm sure there's examples where you would want it, but do you want a flashlight

0:31:08app to now be an app you have to pay for forever with a subscription model.

0:31:13Users probably don't want that.

0:31:15So we have to be sensitive, first of all, to is there some minimum functionality where

0:31:21users now get pissed off and say everything's turned to subscription, I don't want to buy

0:31:25stuff anymore, this is not okay, and now that's a drag on business on the App Store and therefore

0:31:31we all lose.

0:31:33We feel a responsibility.

0:31:35I read your thing that says, "Hey, why not just let the market choose?"

0:31:38What if the market screws itself up and it does badly, and then we all lose?

0:31:42We have to be a little bit sensitive to not do something we think that could backfire

0:31:47and hurt all of us.

0:31:48We want to be careful about minimum functionality, so there will be some guideline around that,

0:31:52which we already have a guideline on minimum functionality for anything.

0:31:56You can't just wrap a website and call it an app, but there will be a little bit more

0:31:59minimum functionality for subscription.

0:32:01I think the guidelines include, a long-standing guideline is that the App Store has plenty

0:32:06of FART apps already.

0:32:08That is absolutely one of the rules.

0:32:12There is a secondary issue, and we're working through this.

0:32:16There are certain states and governments where there are laws about creating a subscription

0:32:24revenue stream without a clear promise to the user of what they're paying for down the

0:32:28road.

0:32:29Our legal team has been working with us on this, on trying to make sure we put in place

0:32:33in the store the right way for developers to make clear their intention to deliver value

0:32:39for that customer, or else they'll be breaking the law by asking for a subscription with

0:32:42no intention to delivering value down the road.

0:32:45We want to be careful of those things.

0:32:48Those are the kinds of reasons we have caveats on it, but the intention, I think, is what

0:32:53we all want.

0:32:54[applause]

0:32:55(audience applauding)

0:32:58The Mac App Store,

0:33:03I'm not gonna say,

0:33:06I'm not gonna say it's been treated as the ugly stepchild,

0:33:10but maybe the slightly less attractive stepchild.

0:33:13And a couple of examples,

0:33:16TestFlight beta testing was in the iOS App Store.

0:33:19Craig, is it in the Mac App Store yet?

0:33:23I don't think so, all right.

0:33:24So no test flight.

0:33:25- How the hell are you doing?

0:33:26- Video.

0:33:27(audience laughing)

0:33:28Video reviews.

0:33:29I know that's, and it seems like that really works.

0:33:32Like there's, you know, instead of just static screenshots

0:33:34to show your app on iOS, you can have a video

0:33:37that shows it in animation.

0:33:39And a lot of times, for developers who are doing

0:33:41the cinematic experience of really making the app

0:33:45feel great, the video can do so much more

0:33:48than a static screenshot.

0:33:52And all of the news last week applies

0:33:55to all of the app stores.

0:33:57So that in and of itself is a change,

0:33:59a change in the way the app store

0:34:03distributing new features, yeah?

0:34:05- So we love all of our kids,

0:34:09and I'm sure all of you do as well equally.

0:34:11(audience laughing)

0:34:12And so we love the Mac App Store,

0:34:16we want it to do well,

0:34:18we want to support the developers in it,

0:34:20we care a lot about it.

0:34:21We use it ourselves.

0:34:22It's a very important store for ourselves.

0:34:23We've moved all of our software distribution into it

0:34:26and are very happy with that.

0:34:27So we're one happy software developer that's using it.

0:34:30And we still think, in the long view of all of this,

0:34:35it matters a great deal.

0:34:38We think it matters for privacy.

0:34:40We think it matters for security.

0:34:42We think it matters for quality on the store.

0:34:45We've all seen examples of apps that have been hijacked

0:34:48on servers where people download stuff

0:34:50that have viruses injected in them,

0:34:52and we don't want any part of any of that, all of us.

0:34:54So we think it's still an important solution,

0:34:56and we're dedicated to it.

0:34:58There are things through the years in the Mac App Store

0:35:01that haven't been fully implemented

0:35:03because they didn't make as much sense in the Mac

0:35:05as they did in iOS, or the engineering effort

0:35:08was really high for a benefit that wasn't seen as as big.

0:35:13Whatever, example, so test flight.

0:35:15For the engineering involved there,

0:35:19people have felt that there are a lot of opportunities

0:35:22in the Mac from website to download apps for test

0:35:25and distribute beta software.

0:35:28So the need wasn't as great, right?

0:35:30It was a clear need on iOS, not as clear on Mac.

0:35:32So that's why some decisions were made and trade-offs there.

0:35:36But as you say, as I've been working more with the App Store

0:35:39team since December, I've really pushed the team to please

0:35:45make sure everything makes sense across all the stores

0:35:48as much as possible and maybe there'll be some exception

0:35:51to that that we have to make but we don't want to.

0:35:53We want to try to do everything the same

0:35:55on all the stores as much as possible,

0:35:58including the Mac App Store.

0:35:59(audience applauding)

0:36:02- So one thing the Mac App Store has been good for

0:36:07and the Mac software ecosystem in general is good for

0:36:10is that it seems to support higher prices of apps

0:36:14for truly professional apps, deeper apps.

0:36:18And there's a consensus, or maybe not consensus is the wrong word, maybe you'll disagree,

0:36:24but there's a lot of people who think that one of the things that's holding back the

0:36:27iPad, especially now that it's the iPad Pro, from replacing a MacBook for someone who might

0:36:35want to, is that it lacks the same depth of deep apps for work that the Mac has.

0:36:43The reason is that the pricing pressure is more like iPhone-style couple of bucks as

0:36:49opposed to Mac-style, where $50, $80, $100 software has long been the norm.

0:36:57I think you see two things happening at the same time.

0:37:01Number one, the iPad's capabilities are growing as a PC replacement product for some people.

0:37:09I know some people have made some statements about that.

0:37:11I don't know who.

0:37:15And so we're trying to make it more and more powerful, making it larger screens, keyboards,

0:37:20the more powerful processors, and all that's happening to drive it into a more capable

0:37:25product.

0:37:26At the same time, you start to see more professional applications begin to make their way onto

0:37:30it, and so I think we're seeing changes there.

0:37:34We're seeing certainly apps that have a similar version on your iPhone that you want on your

0:37:39iPad will have similar pricing, but other apps that are maybe coming over from the Mac

0:37:43or the PC are bringing on pricing models that are more like that.

0:37:47So you're going to see this duality with iPad, that there's a little of both happening.

0:37:53We see an increase of the more professional apps happening.

0:37:56When you see stuff in flight with developers we're working on that's really impressive

0:38:00desktop quality software, more and more are coming to iPad.

0:38:03It's definitely not the hardware, because the iPad Pro stands toe-to-toe with the MacBooks

0:38:08on any technical measure you can give it.

0:38:11I mean, it's beautiful displays, powerful CPUs,

0:38:14and stuff like that, so it's not holding it back.

0:38:16- And I do think if you really look at some

0:38:18of the professional apps that are on the iPad,

0:38:20it's, I mean, some of them are really first class,

0:38:23and I think the iPad Pro's going to accelerate that,

0:38:27and we absolutely wanna find any way possible

0:38:31to make deep investment by developers

0:38:33in the platform possible,

0:38:36because I think we'll all win when that happens.

0:38:38- All right, new topic, privacy and security.

0:38:42I remember a couple of years ago, maybe more,

0:38:45I don't know how many years, but I was at WWDC

0:38:47and I somehow wound up in a session on security.

0:38:50I don't even know why I was there,

0:38:52but I was interested, I think I was talking to somebody,

0:38:55he's like, I gotta go into this thing on security.

0:38:57I was like, well, I'll go with you,

0:38:58and I went in and listened.

0:38:59And at the end, it was when they were still doing Q&As.

0:39:01And I remember this very vividly,

0:39:04And somebody asked the question of somebody

0:39:07who was on the engineering team in charge of security,

0:39:10gave a rant about how passwords are terrible.

0:39:13And people pick bad passwords because they're

0:39:14easy to remember.

0:39:15And passwords that are hard to remember, or hard to crack,

0:39:19or hard to guess are unusable, or less usable.

0:39:23Have you guys given any thought to what's

0:39:25next beyond passwords?

0:39:27And there is this pause.

0:39:29And the speaker-- yes.

0:39:33[laughter]

0:39:34It was like, "Well, that's a very interesting and truthful answer."

0:39:40We've seen, I think, in the intervening years some of the things that might have been circulating.

0:39:44Touch ID.

0:39:45Yeah.

0:39:46Now, one of my favorite features you guys announced yesterday, can't wait to use it,

0:39:50is Auto Unlock.

0:39:51Auto Unlock.

0:39:52Yeah.

0:39:53[applause]

0:39:54Can you talk about how that came to be?

0:39:59Which part of it?

0:40:00I mean, are caring about security?

0:40:02- Well, no, no, with Auto Unlock in particular,

0:40:04the details of what are you guys doing

0:40:07to make Auto Unlock truly secure,

0:40:10that it's not, that I'm not over here

0:40:12opening Phil's MacBook because he's in the room?

0:40:16- Yeah, yeah, well, I mean, of course,

0:40:20it's a continuation of the work we did with Continuity

0:40:25to develop really low-power,

0:40:28BTLE-based discovery protocols

0:40:30So your devices could discover each other continuously with acceptable overhead from

0:40:35a battery point of view and also all the authentication mechanisms we put in place as far as having

0:40:41your devices know that they're your devices.

0:40:43So that's kind of a foundation.

0:40:45The unique challenge with auto unlock is you don't want a kind of relay attack where Phil

0:40:53is actually, you know, well far away from his office and someone basically has a Bluetooth

0:40:59listener that's going to forward a signal to you because you're now by his Mac and this

0:41:07Mac is having a conversation with Phil's watch over a very long distance.

0:41:13And so we're actually able to do time of flight calculation using peer-to-peer Wi-Fi where

0:41:20we literally can measure how long at the speed of light it's taking for the signal to travel

0:41:26from your watch to your Mac and back.

0:41:29[applause]

0:41:30That's a very fast stopwatch.

0:41:36Because of that, if you interposed any kind of relay, it would introduce a delay that

0:41:40immediately would tell us there's high jinks afoot.

0:41:43Yeah, make sure they type in their password.

0:41:45Yeah, absolutely.

0:41:48That piece is critical.

0:41:49But I think on the bigger picture, I mean, Touch ID is one way that we've helped with

0:41:54passwords, but actually on iOS the secure enclave and that whole architecture, the fact

0:42:02that your device is not encrypted just with your password, with your passcode, which honestly

0:42:08whether it's four digits or six digits is short enough that if a brute force attack

0:42:13were possible, it would be, you know, you readily could break into something, but instead

0:42:19it's entangled with a hardware key that only the secure enclave runs and the secure

0:42:23Enclave will only do its unwrapping when running Apple signed software and will only let you

0:42:30try 10 times.

0:42:32And so fundamentally, yeah, that was first, yeah, very important step to saying you could

0:42:39have a practical length passcode with really industrial strength security.

0:42:43And so we keep pushing on this.

0:42:46Continuing on the sort of privacy vein, I think it's a good segue into Siri, quote-unquote

0:42:55deep learning, AI, these sort of features that you guys was a big part of the presentation

0:43:00yesterday, because a big part of your on-stage message about it was the emphasis on the way

0:43:09that the systems are designed to protect users privacy and the technical implications of

0:43:16that.

0:43:17Yeah.

0:43:18So, one of my questions, when does deep learning happen?

0:43:22So like I'm on the phone and I'm taking a couple of pictures of the event and stuff

0:43:28like that.

0:43:29When does the...

0:43:30That analysis occur?

0:43:31Yeah.

0:43:32So, if you upgrade your device to iOS 10 and you have your photo library there with your

0:43:3710,000 photos or 100,000 photos on it, the analysis of that kind of backlog will occur

0:43:43when you're plugged in on AC overnight, because this is a considerable amount of computation

0:43:48that's going to occur that we wouldn't have happen in your pocket.

0:43:52But when you're out taking a fresh picture, at that point we will instantaneously perform

0:43:58the analysis on that hot photo as it's going into your photo library.

0:44:03We can do it that fast.

0:44:05It is like scene classification I mentioned yesterday, is it yesterday?

0:44:11That it is like about 11 billion calculations that have to occur to do that thing like that's

0:44:19a horse, that's a mountain.

0:44:24But with the GPUs on iOS devices these days really cook, so we can get through that essentially

0:44:31instantaneously with the photos?

0:44:36And on the privacy part, my understanding, and correct me if I'm wrong, but my understanding

0:44:40from what I've learned is if you've got iCloud photo library, and I take a couple of pictures

0:44:46with my iPhone, the photos will sync to the cloud, and then they will go to my iPad and

0:44:52my Mac, but the deep learning analysis doesn't go with them.

0:44:57machine performs its own processing on its own time when it, you know, plugged in inappropriate.

0:45:05Is that true?

0:45:06That is true right now.

0:45:07So each device does its own processing.

0:45:12In the future, we could share the results of like the first one who does the work, just

0:45:18make that work go along for the ride.

0:45:21But today, it's going to be each device doing it independently.

0:45:25When you think about what's going to happen, if we release iOS and OS X on separate days,

0:45:34everyone's iPhones will race to do all this work on their library first, and then the

0:45:41Mac will be fine at that point.

0:45:44We wouldn't have saved the iPhones from doing the work if we'd had them share the work of

0:45:49the Mac.

0:45:50Just to add on that view of someday they may not all have to do it, it's a view where

0:45:55Apple will never actually know that analysis ourselves.

0:45:59We won't see that data.

0:46:01It's a way to do it.

0:46:02We're out of the loop.

0:46:04- Yeah, I mean, to be clear, the photos themselves

0:46:06the architecture sets that they're encrypted in the cloud

0:46:11and the metadata, any metadata about the photos

0:46:14that you create or that we create

0:46:16through deep learning classification

0:46:19is encrypted in a way that Apple's not reading it.

0:46:24(audience applauding)

0:46:27I wanna get nerdy on this differential privacy thing.

0:46:33'Cause it's a phrase, it's like an official thing.

0:46:36I've learned a little more.

0:46:37It's not just a phrase you guys made up.

0:46:39It's like a--

0:46:40- It wouldn't have been the phrase we would have made up.

0:46:41- Right.

0:46:42(laughing)

0:46:44- Yeah, we would have done a better name

0:46:46if that's what we did.

0:46:47(audience laughing)

0:46:50- But like in the State of the Union yesterday,

0:46:52I mean, there's real math behind it.

0:46:54This is not just a name that is applied to policies.

0:46:57This is a branch of statistical analysis that--

0:47:03talk to me about it.

0:47:04I know you touched about it in the keynote,

0:47:06but give us a little slightly juicier layman's overview

0:47:09of differential privacy.

0:47:11Sure.

0:47:12Yeah, I mean, of course the idea is

0:47:14that if we wanted to know what word--

0:47:21a new word that everyone was,

0:47:23that lots of people were typing that we didn't know

0:47:25so that we would stop marking it as a spelling error.

0:47:27Or maybe we'd even suggest it on the keyboard.

0:47:29- Like me or something.

0:47:30- Yeah, like now it's just, it's trending, it's hot.

0:47:33We want all our customers to be able to know that word,

0:47:36but we don't want to know that you and Phil

0:47:38are in particular typing it.

0:47:39We want to have no way to have any knowledge of that.

0:47:42You can imagine if what we're essentially assembling

0:47:46is a picture of little pieces of data,

0:47:50of the forest, but all we're getting is a little piece.

0:47:54And when we get that little piece, even each device will statistically much of the time

0:47:59even lie about its little piece.

0:48:02But those lies will all cancel out with enough data and the picture will suddenly resolve,

0:48:09with enough data points, will resolve itself.

0:48:12And so, and yet, and literally if we were trying to learn a word, we would send one

0:48:18One bit, we'd send a position and a single, we'd hash the word, we'd send a single bit

0:48:25from the hash.

0:48:26We'd say at position 23, Phil saw a one.

0:48:29But Phil's phone would flip a coin and actually say, "Actually, I'm going to lie about it.

0:48:33I'm going to say zero even though I saw a one."

0:48:35And that's the data that goes to Apple.

0:48:37Then Apple with enough of that data can build a composite picture and say, "Holy smokes,

0:48:42we have a word here."

0:48:43And this many people roughly are seeing it.

0:48:46And that's typically what you want to know.

0:48:47We want to know what's happening at large,

0:48:50but we have no desire to know what specifically

0:48:52who is doing what.

0:48:53- Which is typically what you would want to know.

0:48:55It's not typically what other companies in the industry

0:48:57would want to know.

0:48:58(audience laughing)

0:48:59- And part of the reason this is so important to get into

0:49:01is because, you know, there's the theory that,

0:49:04well, we can just anonymize the data and send it up

0:49:07and then all's good, and it's a bunch of crap

0:49:09because I can send all this data and say,

0:49:12"Hello, I don't know who you are,

0:49:14"but I happen to know that same location

0:49:16go to every night.

0:49:17I happen to know the same place you go to work every day.

0:49:20I've got all this data.

0:49:20I just don't know your name or your ID.

0:49:23Boy, it's really hard to reverse engineer

0:49:25that anonymous data, right?

0:49:26So what you need to do is create a system

0:49:28that goes beyond anonymizing to really make it impossible

0:49:31to reconfigure who that user is.

0:49:33So who--

0:49:34(audience applauding)

0:49:37- So the way I have it written down here is that

0:49:44if it works as you're describing it,

0:49:46It means it's not just that Apple doesn't use that information to reverse the anonymity,

0:49:52it's that mathematically you can't.

0:49:54We can't.

0:49:55Right.

0:49:56The design of the system is such that it's not even possible if new executives come in

0:50:01in a few years and maybe they would like to poke around.

0:50:05But companies change.

0:50:07No, no, that's absolutely true.

0:50:09And the point of view, I mean honestly, the point of view that someone says, "Hey, I

0:50:15know we know a ton about you, but don't worry.

0:50:19We're nice guys and it's all good.

0:50:23Well, okay, maybe you're nice guys 10 years from now who's running this thing."

0:50:28Or what if someone breaks into your computers?

0:50:30Are they nice guys?

0:50:32So you just don't want to have any central source that has that kind of knowledge because

0:50:36Because in the fullness of time, anything is possible.

0:50:40And so differential privacy is, I mean, there are mathematical proofs that will show that

0:50:48you cannot, with any confidence, determine anything about any of the people contributing

0:50:52to the data set.

0:50:53And we think that's important.

0:50:54All right.

0:50:55Speaking of companies that do collect some information about people, Google and Facebook

0:51:01are two competitors that, and I know a lot of times when you guys talk about these companies,

0:51:05You might talk about search engines and you might talk about social networks because you're

0:51:10gentlemen.

0:51:11But I will name names and I'm going to just point out that Google and Facebook are both

0:51:17and actively pursuing a lot of the same goals.

0:51:21I mean just the image analysis, that's a mountain, that's a horse, those companies are showing

0:51:27similar things.

0:51:28You guys are showing this.

0:51:31But it really is, I don't want to abuse the metaphor, but it's a 180 degree different

0:51:36tactic, where they're doing it with cloud servers and doing the computing in the cloud

0:51:41on data that they've aggregated there, and your method is to do it distributed on the

0:51:49actual devices.

0:51:52Critics are saying, and it's not me, I'm not saying this, I'm like, let's see.

0:51:56No, I mean, I'm totally like, okay, I saw your keynote.

0:52:00I'm looking forward to trying it and let's see if it works for me.

0:52:04I don't know.

0:52:05I think it might.

0:52:08But critics are saying, critics are already saying, and they've obviously since the keynote

0:52:12was just yesterday, I've seen it in a couple of articles, that your strategy is doomed

0:52:18to keep Apple behind them because the Google and Facebook way is the only way that works.

0:52:24And I'm not quite sure where that comes from because-

0:52:27Their PR department.

0:52:28I mean, of a prominent search engine or social network provider that we don't know about.

0:52:40I think part of it, in my mind, is maybe there's like an assumption on the part of some people

0:52:46in the press that a server farm has this massive amount of computational power and that a puny

0:52:54little phone can't compete.

0:52:57But it's not like there's one person's iPhone who's trying to do the image analysis

0:53:01for all the photos on iCloud.

0:53:03Like there are a billion phones to throw at this problem.

0:53:05Right.

0:53:06A billion active devices.

0:53:07So like the billion active Apple devices that are out there in the aggregate have an enormous

0:53:11amount of CPU power.

0:53:13That's right.

0:53:14That's right.

0:53:15The other thing is there's this idea that well if you don't have the data, how would

0:53:18you ever learn?

0:53:19Well it turns out if you want to get pictures of mountains, you don't need to get it

0:53:23out of people's personal photo libraries.

0:53:25Like, we found out we could find some pictures of some mountains.

0:53:30We did some tough detective work and we found them.

0:53:40So that's pretty good.

0:53:44So moving on, Siri.

0:53:50Siri now has an API and it's six categories.

0:53:55I don't know if I wrote, I didn't write them down,

0:53:58but it's like ride sharing.

0:53:59- Messaging, photo search.

0:54:03- Voice calls.

0:54:04- Payment.

0:54:06- Payment.

0:54:07- We're missing one more.

0:54:08(audience laughing)

0:54:09- Why?

0:54:10- We're sending money, yeah.

0:54:11No, we did payment, all right.

0:54:13I can say it a few different ways.

0:54:15We can get past this.

0:54:15- Well, there's six distinct categories.

0:54:17- Workouts.

0:54:18- Workouts, there we go, thank you.

0:54:20- Yeah.

0:54:21- This is why I should have a live audience for all my--

0:54:23- There's crowd sourcing right here,

0:54:24but it's totally anonymous.

0:54:24- For all my shows.

0:54:25And we don't know who said it, 'cause it's all--

0:54:27(audience laughing)

0:54:30So why restrict Siri to those six specific categories?

0:54:38- Yeah, it comes down to modeling the domains well.

0:54:42In order to understand what someone is saying,

0:54:46people are gonna say, are gonna speak to Siri

0:54:48in a whole bunch of different ways, and even in a whole bunch of different languages.

0:54:52And when they say, when they want to say, send a message to Phil saying that I'm going

0:55:00to be late for the interview, then we, I could have said that in dozens of other ways.

0:55:07I could have said, we chat Phil that I'm going to be late for the interview.

0:55:10We chat Phil using WeChat, telling him that, etc.

0:55:15And I even could say something like, "WeChat Phil," and then I'd need to know, "Okay,

0:55:20what do you want to say to him?"

0:55:23And Siri knows all of this because Siri understands the domain of messaging well.

0:55:27It understands all the vocabulary, it understands what the verbs are, what the objects are,

0:55:34and can collect them and can do so in a dialogue.

0:55:37And so we want to make sure that when you're talking to your assistant, that your assistant

0:55:43consistently intelligent about understanding you and how flexible you can be in talking

0:55:48to it.

0:55:49And so to do that, we had to develop those domains.

0:55:51So these are the domains that we've developed in a way that developers can plug in.

0:55:55We'll do more and more of that over time.

0:55:58And of course we'll search for more and more flexible ways to enable developers to do that

0:56:03at a time.

0:56:04But we want to make sure that what we do is preserve the intelligence of your assistant.

0:56:08It would have been super easy for us to say, "Hey, just tell us a trigger word or the name

0:56:14of your app and we'll hand you a string.

0:56:17And good luck."

0:56:18And so you'd say something to Siri and most of the time you get back the app doing something

0:56:23crazy and the user would say, "What in the heck, Siri doesn't understand me.

0:56:26I don't understand this."

0:56:28And in this case, we're able to be consistent about Siri's ability to understand you.

0:56:32And so we'll make models more and more powerful and we'll do more of them for more domains.

0:56:37but we start with the baseline of have a quality experience

0:56:39around what we cover.

0:56:40- And I think this is an insight into how we,

0:56:43it's not right or wrong, how we approach things

0:56:45differently than some other companies do.

0:56:47We've all been seeing stories for a while saying,

0:56:50"Hey, Apple, some other companies are doing some assistance

0:56:53"and they're allowing these other apps to be bots

0:56:57"and to hand off and do things for them.

0:56:59"You're not, you're behind."

0:57:01Where when we have thought about doing it for a while,

0:57:04we've thought about it since the very beginning of Siri,

0:57:07which is we needed a solution to how do you keep Siri

0:57:10from being smart at one thing and then stupid at another?

0:57:13That will be an inconsistent experience in all that we want.

0:57:15We need Siri to be equally smart at all the things we do,

0:57:18and as it gets extended, that intelligence needs to extend,

0:57:22and so the team has been working hard at that

0:57:24where others shoved in quickly to do things

0:57:26that don't translate that intelligence to third-party apps.

0:57:29And so to do that means you have to, with intention,

0:57:32add categories and domains.

0:57:34The hope is to add more and more

0:57:36so that users can ask anything they want over time

0:57:38and use any of their apps that they love,

0:57:39and it all works, it just takes time building domains.

0:57:42So we'd rather take the time to do it right

0:57:44than rush out just 'cause it gets a good story

0:57:46to say you have something.

0:57:47(audience applauding)

0:57:52- One of the things I've, like in the last year or so,

0:57:57maybe half year, but I've noticed it,

0:57:59and I'm sorry, I bang this drum a couple times a month

0:58:02on during Fireball is why the industry as a whole

0:58:06doesn't seem to count iMessage as a messaging platform.

0:58:11And the number that always gets thrown out

0:58:13is monthly active users,

0:58:15and WhatsApp has so many monthly active users,

0:58:17and so they're worth so many billions of dollars.

0:58:20iMessage has to be right up there

0:58:26in terms of monthly active users, daily active users,

0:58:29hourly active users, users sending iMessages

0:58:33during the talk show.

0:58:34Is that frustrating?

0:58:37- I mean, it's okay.

0:58:41- I mean, customers don't read those things.

0:58:45It's all inside the beltway kind of like

0:58:47who feels proud that they made a list.

0:58:49It doesn't matter to users.

0:58:51- Yeah, I mean, messages is the most used app on iOS period.

0:58:56So it's used a lot.

0:58:57And certainly we saw that every time we'd add a couple new emoji,

0:59:04it would be like the biggest thing.

0:59:06We'd work all year on like a new file system or something.

0:59:09And it turned out the rest of the world outside this room was more excited about the two new emoji.

0:59:21And so we figured, you know, if there's one place we could make a tremendous difference

0:59:27and how people experience iOS fundamentally.

0:59:31They're spending a lot of time in messages.

0:59:33And so we put a ton of creative energy into it

0:59:37and ultimately through opening up to developers,

0:59:39I think the collective energy

0:59:41that'll go into making messages great

0:59:42is gonna be phenomenal.

0:59:43- In the keynote, I was sitting like in the middle

0:59:46of the floor, halfway back, halfway in the center,

0:59:48just right in the middle.

0:59:49Really, it was a great place to hear the reactions.

0:59:52The biggest reaction I thought of the entire keynote

0:59:55was when you announced that emoji were going to 3x.

1:00:00I'm not exaggerating.

1:00:02I'm not exaggerating.

1:00:03And it was like a real visceral buzz.

1:00:06And it's like-- and here's a crowd of people,

1:00:08you know, developers who are more technically minded,

1:00:11and they are here to hear about technical details.

1:00:13And this thing that is really just, you know,

1:00:15it's just more fun, got this really powerful reaction.

1:00:21Well, next year we're going to 4x.

1:00:22(audience laughing)

1:00:25See, this is why we don't let you out.

1:00:27(audience laughing)

1:00:30Now we're gonna be held to that,

1:00:35and next year when we don't,

1:00:36we'll be like, "You said 4X!"

1:00:38(audience laughing)

1:00:40Apple, you're late, you're late!

1:00:42And then eventually it'll be finally 4X!

1:00:45(audience laughing)

1:00:47(audience cheering)

1:00:50(audience applauding)

1:00:53Coming down to home strategy.

1:00:58- There's a few competitors right now,

1:00:59back programming, they're like, "4X, let's beat him to 4X."

1:01:02(audience laughing)

1:01:03- Well, it's like the onion story about the Shik's CEO says,

1:01:07"Screw this, we're going to Five Blades."

1:01:09(audience laughing)

1:01:11And like three years later,

1:01:12Shik came out with a five blade razor.

1:01:14You can't, you can't underestimate what people will stoop to.

1:01:19(audience laughing)

1:01:22Any of the other iMessage stuff that,

1:01:24I mean, 'cause clearly there's a lot of work,

1:01:26and a lot of it's very fun.

1:01:27A lot of it is the developer integration,

1:01:30it really turns it into a platform.

1:01:34It's not just a thing that people can text with now,

1:01:35it's a thing that people in this crowd

1:01:38can write software for.

1:01:39Anything that stands out that maybe

1:01:41didn't get enough love in the keynote?

1:01:42- Well, we talked about the way in which

1:01:46I think these apps can spread kind of

1:01:48in a really great way, virally.

1:01:50We didn't talk about that at all.

1:01:51And I think that's going to be really powerful for developers

1:01:54and is going to make it worth developers' while to put

1:01:56some energy into them.

1:01:58We made them really easy to create.

1:02:00So if artists--

1:02:02we think there'll be a community of artists that will

1:02:03build sticker packs that are just really fun and they don't

1:02:05need to write any code to do it.

1:02:06So I think that's going to be really important.

1:02:08[CHEERING]

1:02:13Also, I think the way that they are distributed,

1:02:18if you have, it's not just about the extension,

1:02:21the extension can be part of your app.

1:02:24And so there's some cases where you want a model

1:02:27where the extension is sort of in cooperation

1:02:30with your larger app experience.

1:02:31I mean, one simple example would be like,

1:02:33if you have your sports app,

1:02:36your sports app knows what your favorite teams are,

1:02:39well, your extension in messages

1:02:41that let you share those clips

1:02:42or whatever is gonna know that as well.

1:02:46So there's a connection there.

1:02:48We have one where something that people like to do a lot

1:02:52is share music, especially you hear something,

1:02:54you're like, wow, this is great,

1:02:55I wanna tell my friend about this awesome song.

1:02:57Well, if you go to the Apple Music extension,

1:03:01it knows what's now playing in your music

1:03:05and it knows what you listened to with the last few songs.

1:03:08And so that's just one tap to share.

1:03:09And so I think there'll be interesting integrations

1:03:12where the message extension is sort of the tip of the iceberg

1:03:15of an experience that you have inside your app as well.

1:03:19All right, moving on to watchOS.

1:03:20What you guys do year after year is make iterative

1:03:26improvements to your software.

1:03:28And you add features, you take what was slow and you make it

1:03:31fast, you take what was ugly and you make it look better.

1:03:35But the performance improvement on app launch speed on

1:03:39- And watchOS 3 does not look like one year over a year.

1:03:44It's crazy and I really did in the keynote

1:03:47had a I gotta see this and then when I got hands on

1:03:51with a watch running watchOS 3.

1:03:54- It's for real. - It's for real.

1:03:55It really is.

1:03:56Anybody in the audience, have you guys upgraded?

1:03:58Anybody? (audience applauds)

1:04:00- It's for real.

1:04:02- How is that possible?

1:04:03(audience laughs)

1:04:06- I mean a couple of things.

1:04:07we certainly actually had some RAM to spare.

1:04:11- Really? - Yeah.

1:04:12Yeah, it turns out that if people are running,

1:04:16they have their favorite 10 apps,

1:04:18we can keep them mostly running.

1:04:21We can keep them mostly resident.

1:04:22We'll halt them so they're not burning CPU,

1:04:24but we can keep them mostly resident,

1:04:26which means you're not doing all the work

1:04:28that goes into launching an app when you take them live.

1:04:30But the other thing is, it turns out when we first,

1:04:35we're coming out with WatchOS,

1:04:36we were being really conservative about understanding how people were going to use the watch and

1:04:41trying to make sure we could hit our goal of very solid all-day battery life so you

1:04:45could use it all day and you could charge it at night.

1:04:47And we found we'd actually really overshot the goal, which was an area of just massive

1:04:53focus and paranoia throughout the release.

1:04:56We needed to make sure that we squeezed every little bit of juice out of the thing.

1:05:00And so realizing we had this budget, we said, "Hey, look, we actually have enough to do

1:05:05background updates."

1:05:06You know, we had overshot enough that we can keep these apps

1:05:08both in memory, but also keep them up to date

1:05:12throughout the day, so when you look at them,

1:05:14they're already there.

1:05:15It's not like launch and then wait,

1:05:17have them get the information.

1:05:18It's they already have the information.

1:05:19And so those were two really vital techniques.

1:05:23I think the other thing is, as you talk about,

1:05:25you build something as new and different as the watch,

1:05:28and until you finish and you live on it

1:05:31and you figure out like what's really the essence

1:05:33of this thing and appreciate which problems

1:05:35are the most important to solve,

1:05:37we realized that the watch is all about glanceability.

1:05:42It's useful to the extent that like,

1:05:45okay, I can solve my task, I'm done.

1:05:47If I'm up here and I'm waiting and I'm fiddling around,

1:05:50my arm's getting tired, this is no fun anymore,

1:05:52I'm gonna do it a different way.

1:05:54And with that as our obsession for the last year,

1:05:57we've taken all of those tasks and we said,

1:05:59you have to be able to finish the task,

1:06:01end to end, two seconds, right?

1:06:03And that meant the launch better be the instant part

1:06:06because now we need to let the user think

1:06:08and do something in two seconds and get it done.

1:06:10And with that focus, you find a way

1:06:12and chipped away at it.

1:06:14- What really strikes me once I got to hands-on with it

1:06:16and I could really see it is just how much

1:06:19the design changes to the navigation of the user experience

1:06:23are exactly coinciding with the engineering improvements

1:06:28to make it faster.

1:06:29So the fact that glances are no longer a separate thing

1:06:33is because the apps themselves in the dock

1:06:36can serve as glances

1:06:37because they're getting the background updates,

1:06:39because you made the changes

1:06:40that make them stay resident in memory.

1:06:43- Yeah, it's nice when it all comes together.

1:06:45- Really?

1:06:46(audience laughs)

1:06:47- Yeah, absolutely.

1:06:48- The other thing I'll add

1:06:49that once you start to use the new watchOS,

1:06:53in addition to having the apps come across faster

1:06:56and you can get access to them quickly,

1:06:57Your watch faces, in a sense, become sort of apps in themself

1:07:01in the sense that you change the ones you use

1:07:04and you rearrange them and change the complications.

1:07:06For example, I would normally keep the activity rings

1:07:10on my watch face, but now I can choose to make that the next one

1:07:13and I just swipe over to them and swipe back

1:07:15because I use the activity watch face versus needing the rings.

1:07:19And then I can have a different watch face

1:07:21for some other time of the day when I need some other actions

1:07:23and access to apps.

1:07:25So that becomes a much quicker and more useful way

1:07:28to expand the things you do with it.

1:07:29It's really profound changes through the interaction model.

1:07:33- Yeah, and really different complications too.

1:07:34So if you're gonna be in more of your workout mode,

1:07:39you would swipe over and the complications

1:07:41that are relevant to that, and therefore the launchers,

1:07:43essentially for that, are right there.

1:07:45So you kinda go, here's what, you know, I'm at work,

1:07:47I'm gonna be this way, I'm out with the family,

1:07:50I'm gonna go this way, and you have all the different

1:07:52activities that are relevant to that.

1:07:53It's like you've got almost a custom dock

1:07:55or custom launcher based on your watch space.

1:07:57So that's another element where I feel like

1:07:59it's really come together in a nice way.

1:08:01Yeah, the team has done a really great job.

1:08:03(audience applauding)

1:08:05- So, just wrapping up, coming down to home touch, Swift.

1:08:09Now you were on my show a few months ago

1:08:10when Swift went open source.

1:08:12It was very nice, we had a good time.

1:08:13- Lot of dynamism.

1:08:16(audience laughing)

1:08:18- And we talked about Swift use within Apple

1:08:21and why you guys can't yet write the OS in your apps

1:08:26in Swift, but that you're using it,

1:08:27engineers are using it to write unit tests

1:08:29and stuff like that and it's getting used.

1:08:30But I saw in the announcement that this new

1:08:32Swift Playground app is itself written in Swift.

1:08:34- Yeah, well actually in OS X, like most of the dock

1:08:39and most of mission control, yeah.

1:08:42Well, oh God.

1:08:44(audience laughing and applauding)

1:08:47- Another doll.

1:08:47(audience laughing)

1:08:50- In our Sunday rehearsals--

1:08:54- When was that name first hinted at?

1:08:56(laughing)

1:08:59- I don't know.

1:09:00- Last year. - Right here.

1:09:01- I thought I was being so transparent too.

1:09:05(laughing)

1:09:06- I saw right through it.

1:09:07- I know.

1:09:09You were very polite.

1:09:10- In our run through for the show,

1:09:13when I say, oh, we're changing the name to Mac OS,

1:09:17and this is like on Sunday,

1:09:19and my next slide is to say something about how

1:09:22we have these great new features in macOS.

1:09:24And I literally go, and so our new release is macOS.

1:09:26And so we have some great features in OS X.

1:09:28(audience laughing)

1:09:30It's tough.

1:09:31Oh, we spent a lot, 15 years.

1:09:33It's a long time, but I think we all feel great

1:09:35about the new name.

1:09:36And anyway, in macOS, the dock is substantially converted.

1:09:41So we're, in mission control,

1:09:44all those areas are using Swift a lot.

1:09:46So it's starting to spread a lot internally. There are some barriers,

1:09:50but I think this year the most important thing,

1:09:52and I think Chris Latner really covered it in the state of the union is getting

1:09:56to source stability.

1:09:58And so we decided this year,

1:10:01like we're going to put that over all the priorities,

1:10:05take what we've learned. You know, when we first shipped Swift,

1:10:08a couple of years ago, the objective was,

1:10:10let's make sure that it's familiar from an API point of view, you know,

1:10:14minimize the kind of transition of,

1:10:16I gotta learn all new method names

1:10:18for all the classes I already know.

1:10:20And so we really bias toward that.

1:10:21Now people are so familiar with Swift,

1:10:24the priority is let's make sure those APIs

1:10:26are all very native to Swift in their field.

1:10:29And so we've done all the hard work

1:10:32to update all the APIs, all the naming conventions,

1:10:35and take some major APIs like Core Graphics and LibDispatch

1:10:38and make them just awesome for Swift.

1:10:40And so that was, yeah, this is important stuff.

1:10:42(audience applauding)

1:10:44But what that means is we've achieved that level of source stability.

1:10:49So next year it won't be like, Oh boy, you know, as a developer.

1:10:54So that's, that's the important thing. ABI stability,

1:10:57which means literally that the Swift binary you built could be linked against

1:11:01the future libraries. That's one that we made a lot of progress,

1:11:04haven't gotten all the way there, but that's far less,

1:11:06that's far more of an issue for us internally than it is for developers.

1:11:10It's important for developers, but much,

1:11:13but I think the source stability one was the right priority

1:11:15and I feel really good about the progress

1:11:16that the team made on that.

1:11:18- Last question.

1:11:20How about one thing that you guys announced yesterday,

1:11:23whether it was in the keynote or not,

1:11:24maybe something that missed the keynote,

1:11:25but one thing that you think

1:11:28deserves a little extra attention.

1:11:29And I'll let you guys think about it.

1:11:30I will go first, and your correct answer

1:11:33is probably new file system.

1:11:34- Is that right?

1:11:35(audience cheering)

1:11:37- I'm gonna say universal clipboard

1:11:40because I've always wanted this,

1:11:42And for me, it's links.

1:11:44It's like I'm on my phone, and it's like, oh,

1:11:46I want to post this during Fireball.

1:11:47But I'm in my office, so why would I do it on a phone?

1:11:50I'll go sit down at my iMac and do it with the real keyboard.

1:11:54But how do I get this from here to there?

1:11:56And man, what I want to do is just copy it

1:11:59and go over there and hit Command-V.

1:12:00And again, the thinking through that you guys

1:12:05did of how to do this in a way that

1:12:08isn't going to surprise people in a bad way--

1:12:11There's like a two minute time out.

1:12:12- Yep.

1:12:13- So like if I copy something on my phone right now

1:12:17and tomorrow I paste it my Mac, I'm not getting that.

1:12:21- That's right.

1:12:21- Because it's really like what you're,

1:12:23ways of detecting what's in it.

1:12:25- Well even the communication is,

1:12:27like the other continuity features, is peer to peer.

1:12:30So it's not like you're sending everything you copy

1:12:32up to the cloud all of a sudden

1:12:33just so that it can get down to the other device.

1:12:35It really is about, you know, kind of two,

1:12:37did it here, copy, paste,

1:12:39which I think is absolutely what people want,

1:12:41and it has the right privacy and performance characteristics.

1:12:46And as you say, it gets rid of the surprises.

1:12:50And it just turns out to be the most, once you have it,

1:12:51it's the most natural way in the world

1:12:53to do these kinds of things.

1:12:54So I think the team did awesome work there.

1:12:57I think that's great.

1:12:58Do I have to say new file system again?

1:13:00(audience laughing)

1:13:01No, I mean, I think the new file system is great.

1:13:04And by the way, I mean, the prospect of,

1:13:08This is when you have to get right, let's say.

1:13:11(audience laughing)

1:13:13And so we have an awesome file system team

1:13:18who really knew which problems we needed to solve

1:13:22for a world of flash storage

1:13:25and has done a super solid job.

1:13:29And we're being conservative about how we're rolling it out

1:13:32as a developer preview so people can kick the tires

1:13:35on it this year.

1:13:36But we look forward to making it part

1:13:38the products going forward and I mean I think it's gonna be be great and

1:13:42obviously we didn't we didn't talk about it because we don't talk about peer

1:13:44developer preview material there but I think in terms of something that's

1:13:47important for the platform going forward it's it's big. What do you so let's just

1:13:50say three years from now we're all using iPhones that are using APFS instead of

1:13:57HFS plus what would be like a noticeable improvement to the to the experience? So

1:14:03so it'll help with performance it'll help with things like how we do software

1:14:07updates and other things because we can snapshot volumes, we can roll things back.

1:14:11I mean there are a lot of important attributes there.

1:14:14It's important when you think about multi-user, like how files are protected between multiple

1:14:20users on a Mac because we actually have file system level encryption now standard across

1:14:26both platforms.

1:14:28And so I think from a security point of view it's big.

1:14:34And I think performance, I mean now you do a copy or even like the safe save operation.

1:14:40When you save documents in a lot of apps, it's like move that one aside, create another

1:14:44whole copy of all of that, now overwrite some of it, now delete the old directory.

1:14:51Now that's atomic and the clone file makes all of that super fast.

1:14:54So I think it's just going to be great across the board.

1:14:58I want to answer in a very different direction.

1:15:02Of the keynote, the thing that we haven't talked about that to me

1:15:06was really amazing was we had a bunch of demo-ers who had never

1:15:11been in a keynote before.

1:15:13It was their first time.

1:15:14[APPLAUSE]

1:15:15And they were fantastic.

1:15:19Stacy did a great job.

1:15:22Bethany and Emron did a great job.

1:15:24Boaz did an incredible job.

1:15:27And Cheryl, I wasn't gonna forget,

1:15:30and Cheryl did an incredible job.

1:15:32And all of them work on the things they demo,

1:15:35they're just, and they were fantastic.

1:15:38So that's my sort of unsung thing of the keynote,

1:15:42was those presenters.

1:15:43- I said mid keynote, I was sitting with Ben Thompson,

1:15:45and I said, "I can't believe that none of these people

1:15:47"have ever done this before, because they're amazing."

1:15:51And they really did kick ass up there.

1:15:53That was great. - They sure did.

1:15:55That's it unless you guys have anything else for me.

1:15:57(laughing)

1:15:59- Just thank you for having us.

1:16:00- I wanna give some thanks here.

1:16:01I wanna thank our sponsors, MailChimp, Microsoft,

1:16:06and mad.com, M-E-H.com.

1:16:09Go there and buy some junk.

1:16:11(laughing)

1:16:12I wanna thank Jake Schumacher and Jed Hurt.

1:16:17They're doing the video here,

1:16:18so if you're watching at home, you can thank them.

1:16:21(applauding)

1:16:23They are the co-makers of the upcoming documentary

1:16:27App The Human Story, which has been in the works for a while.

1:16:31I've seen a rough cut.

1:16:32It is amazing.

1:16:33It is really coming along.

1:16:35Appdocumentary.com if you want to see more.

1:16:38Drew Bischoff from Hybrid Events is here

1:16:41running whatever apparatus is involved

1:16:43in doing the live streaming, which I've heard is very hard.

1:16:46[LAUGHTER]

1:16:49I want to thank Mezzanine and the entire staff here,

1:16:52who have been, they're led by Megan Rogerson.

1:16:57She's great, she's been here all four years

1:16:59that I've been here.

1:17:00The staff is great, the bartenders are great,

1:17:03security guy, I mean, this is a really great place

1:17:05and I really appreciate it.

1:17:06I want to thank Paul Kefasis and my wife, Amy Gruber,

1:17:12of Just The Tip fame.

1:17:17They're a podcast that is on, I don't know,

1:17:19some kind of hiatus.

1:17:22But they're the ones who made this event run so that I can just sit back here and be nervous

1:17:28and make these cards with questions and not pay attention to any of the details.

1:17:32I don't know anything that's gone on out here.

1:17:35The fact that you guys even have seats is thanks to them.

1:17:40And I want to thank Phil and Craig for being here.

1:17:44Thank you.

1:17:45Unbelievable.

1:17:46Thank you.

1:17:47(audience cheers)

1:17:49Last but not least, thank you for being here.

1:17:53(audience applauds)

1:17:56(audience applauding)

1:17:59(audience applauding)

1:18:03[ Applause ]