00:00:11 ◼ ► not in the beautiful California theater or elsewhere in California in front of a live audience,
00:00:54 ◼ ► They estimate that one out of every five Macs on earth are running some sort of software from MacPaw.
00:01:18 ◼ ► all sorts of stuff that might be moving with your Macs from one installation to another over the years.
00:01:23 ◼ ► CleanMyMac is a great utility that can help you clean it up in a way that gives you total control over what gets cleaned up.
00:01:49 ◼ ► MacPaw has, under their own belt, 10 other software products that are used by more than 30 million users worldwide,
00:02:03 ◼ ► Setapp is a great product. It's a $9.99 per month subscription that gets you access to 200 paid Mac apps from developers all around the world.
00:02:14 ◼ ► It's a subscription service run by MacPaw, $9.99 per month, that's it, 200 paid apps you get access to as a subscriber to Setapp.
00:02:23 ◼ ► It has been going on for a few years now. They're adding new apps all the time, and it is just simply a great deal.
00:02:45 ◼ ► Chances are you had to format the data before the CSV or the Excel file or whatever format you got it in was ready.
00:02:51 ◼ ► If you're an engineer, you've probably built your own CSV importer in the past, and it probably wasn't fun.
00:03:25 ◼ ► FlatFile's platform enables companies to elegantly import and migrate data from their customers
00:03:51 ◼ ► Go to linode.com/thetalkshow and see why Linode has been voted the top infrastructure as a service provider
00:04:09 ◼ ► You don't have to be like some kind of super user with a big, high monthly account to get 24/7, 365 support.
00:04:23 ◼ ► I host Daring Fireball at Linode not because they're a sponsor, but simply because I really do think that they're the best.
00:04:40 ◼ ► or build it from scratch and manage everything yourself with supported centralized tools like Terraform.
00:04:45 ◼ ► Linode offers the best price to performance value for all compute instances, including GPUs,
00:04:51 ◼ ► as well as block storage, Kubernetes, and their upcoming, this is a new thing that's coming soon,
00:04:58 ◼ ► Linode makes cloud computing fast, simple, and affordable, allowing you to focus on your projects, not your infrastructure.
00:05:07 ◼ ► And again, they're just added GPU hosting plans in addition to cloud hosting for machine learning and neural net use built using RTX 6000 GPUs.
00:05:19 ◼ ► Once again, linode.com/thetalkshow is the URL that will get you there, and they'll know you came from here.
00:05:26 ◼ ► You can create a free account with your Google or GitHub account credentials, or just your email address.
00:05:42 ◼ ► And now, as tradition dictates, good friend of the show, Paul Kefasis will introduce our guests, and we'll be talking to them soon.
00:05:51 ◼ ► Daring Fireball is pleased to present another very special, very remote episode of the talk show.
00:05:59 ◼ ► This episode is coming to you from Philadelphia, Pennsylvania, and WWDC 2021 in California.
00:06:08 ◼ ► Now, we're delighted to welcome back to the show, Mrs. Craig Federighi and Greg Joswiak.
00:06:19 ◼ ► I wish we were in person. We obviously can't be again this year, but this is still hopefully going to be a lot of fun.
00:06:35 ◼ ► Well, wouldn't it be great to have you know what I miss? I have to say I'm getting a little needy for it.
00:06:46 ◼ ► One day where I get to come out and people seem extremely happy to see me and there's applause.
00:06:53 ◼ ► And it's, for me, one day a year is exactly the right amount, but I miss it. But it's good to see you.
00:07:01 ◼ ► How do you think Craig feels? This is the one day a year that his kids get to know he's really an important guy.
00:07:05 ◼ ► You know, it's like he gets all that applause at the show. The rest of the year, not so much.
00:07:11 ◼ ► You are a big show. People always ask me, and it's weird. It's a lot more fun to have the surprise guest who is going to be on the show with the live, because nobody knows.
00:07:20 ◼ ► And so far, knock on wood, we have never once had it spoiled. Every time it has been, you guys, whoever it is, pops out from the curtain and nobody knows.
00:07:29 ◼ ► But I will say this, and no offense, Jaws, I swear, but when people ask me, they always ask, "Are you going to have Craig? Are you going to have Craig? Are you going to have Craig?"
00:07:51 ◼ ► Well, it's fun to have you. Let me just say this, because I get this. A couple of ground rules or background rules, people ask me, they always ask me, "Do you have to send all of your questions to these guys? Do they tell you what to talk about?"
00:08:19 ◼ ► That's been the case since Phil was the first person from Apple who was on the show quite a few years ago at this point.
00:08:28 ◼ ► But I remember asking him backstage, he was like, "No, you can ask me whatever you want. You just may not like my answers."
00:08:34 ◼ ► But we do have a loose, basic agreement that the scope of the show—this is John Gruber's annual interview with interesting people from Apple.
00:08:47 ◼ ► In general, it is a WWDC show, and that's what it's been since the get-go. That's what it is now.
00:08:54 ◼ ► And it's just a loose agreement, and there's plenty to fill an hour or an hour plus just talking about WWDC.
00:09:00 ◼ ► There are probably well more than an hour worth of questions unrelated to WWDC that I would love to talk to you about.
00:09:06 ◼ ► But that's really the only agreement is they want to talk about WWDC. I'm going to go back to Dub-Dub.
00:09:24 ◼ ► I got tons of questions about last year. How did you guys shoot this? What type of fancy camera did you use? What did you send?
00:09:41 ◼ ► And again, this was not any kind of promotional, "Hey, they'll do an interview remotely with you, but we want to be able to say everything was shot on iPhones."
00:09:51 ◼ ► This was, "Hey, we're shooting." Your people said, your tech people, "They're going to be shooting on iPhones."
00:09:56 ◼ ► And for me, convenience-wise, I'm in a room here with nobody else. I have no tech crew.
00:10:01 ◼ ► Having an iPhone where sunlight from over here might go down and it auto-adjusts, it's a great camera for this.
00:10:18 ◼ ► And it brings to mind, and I know we're going to talk WWDC, but just while we're talking about this, a few years ago when Phil was on the show, and I asked him a question about Apple and cameras, and I said, "Do you consider Apple to be a leading camera company?"
00:10:37 ◼ ► And it was the most, Phil, he ever got in those interviews where he suddenly got very serious and he said, "The leading camera company."
00:10:53 ◼ ► Well, but when I told people last year's show, which came out great, was shot, all sides shot on iPhone cameras, people were like, "No."
00:11:03 ◼ ► And I'm like, "Yeah, really." And it's really just a great solution for it, and it's really worked out well.
00:11:15 ◼ ► Let me say this, too. Before we get into WWDC-specific topics, this is the first time I've spoken to you in the Apple Silicon Mac era.
00:11:29 ◼ ► Because last year it was announced, but they weren't out yet, and now they are, and they seem to be at least somewhat well-received, to say the least.
00:11:42 ◼ ► But one of the things that struck me about the M1, and in very recent news, I mean, just last month, the new iPad Pros came out using the same M1 chip, is a very similar strategy hardware and software-wise.
00:12:02 ◼ ► Where, like, let's just say, to pull one example, that the M1 is sort of to Apple hardware, and the A14 is sort of a sibling chip in terms of the cores and the performance.
00:12:36 ◼ ► In the same way that SwiftUI is letting developers write UI, it's not like, "Oh, we're going to take these iPhone UIs and force them onto the Mac."
00:12:46 ◼ ► Do you guys feel that there's a sort of similar mindset with hardware and software now that every product you guys make is using your own chip designs?
00:12:58 ◼ ► Well, as you know, Jon, this was a big deal for us because the Mac was really the last major platform that wasn't running our silicon.
00:13:18 ◼ ► And if we were just using the same thing as everybody else's, how could we be any better?
00:13:22 ◼ ► And we made quite an investment to obviously develop a world-class silicon team that has just knocked the ball out of the park.
00:13:32 ◼ ► Now that the Mac has that, wow. I mean, the performance, the power efficiency, and the technologies that we have with the M1 and the Mac is just allowing the Mac to really go to a new level.
00:13:46 ◼ ► I mean, here's a product that has been around for a while, and it's never been more exciting, more vibrant than it is right now.
00:13:54 ◼ ► And certainly having one line that carries all the way, I mean, ultimately a common line with even a common GPU architecture, things like the A&E that can scale all the way from the watch up through the phone and into the Macs is just incredible.
00:14:10 ◼ ► So as we work on all of these different technologies that can now take full advantage of these things, you know, we talked about like on-device Siri and how that uses A&E.
00:14:20 ◼ ► I mean, having that kind of technology foundation that can scale is great for us as a developer and we think great for all developers on our platforms ultimately.
00:14:33 ◼ ► And this was the hardest thing for me to, as I outlined, what do I want to talk to you guys about?
00:14:55 ◼ ► And it clarified to me, and I thought watching the keynote and the State of the Union and squeezing in a couple of developer sessions, it's clear to me that privacy deserves its own topic.
00:15:09 ◼ ► And it's not just something that you guys care about on your end and spend a lot of time on, you clearly think this is something customers care about too because you're talking about it, you're making ads about it, you're putting ads on NBA games that are specific to privacy.
00:15:33 ◼ ► I think there was a sense in the industry that Apple was oddly focused on design, right?
00:15:41 ◼ ► Look at all the PCs that are sold from other companies that they, everybody will admit these are not as nicely designed and they outsell the Mac by, you know, whatever fold.
00:15:54 ◼ ► Well, if you even think about the iPod, right, it's like, oh, the little brown thing's going to come out and it's going to be cheaper and it's going to, then nobody's going to buy iPod.
00:16:05 ◼ ► But when the iPhone came out, it really, to me, changed the industry and everybody had to catch up.
00:16:12 ◼ ► Where you guys are the ones banging the privacy drum now and you're putting your muscle behind it with real engineering work.
00:16:25 ◼ ► I mean, we have been saying for quite some time that we look at privacy as a fundamental human right.
00:16:41 ◼ ► And this wasn't something, as you know, that we just, you know, scotch tape on the side.
00:16:56 ◼ ► Customers should care about their privacy, but this is something for us that's much bigger than trying to be popular.
00:17:06 ◼ ► Just think back to the floppy era where you held your own data in your own hands and there was a sense that it was yours, right?
00:17:25 ◼ ► And there was a time when, you know, the earliest sort of cresting of machine learning where there were a lot of people saying,
00:17:42 ◼ ► And we said, "Look, we think it's the -- we build products for ourselves, and we sure as heck know we want them to be private.
00:17:50 ◼ ► I think it's wonderful that it's resonating with more and more people because I think as people begin to understand that they haven't actually -- it's not too late.
00:18:08 ◼ ► Regardless, this is who we are. These are our values, and they're going to reflect themselves in our product regardless.
00:18:15 ◼ ► We spoke about this -- I know I asked about this a few years ago, and I think it might have been the year that Mike Rockwell was on.
00:18:23 ◼ ► But basically, my question was, isn't this -- you know, if the idea was five, six years ago that to get really, really great machine learning backed features,
00:18:38 ◼ ► I think in hindsight now it's clear that, no, it's just that the early companies that were having success with machine learning backed features were doing it in the cloud.
00:18:50 ◼ ► And my question was, isn't this an amazing amount of distributed computing when you have these chips in phones and iPads and now Macs, these Apple silicon chips with incredible performance,
00:19:01 ◼ ► these incredible neural engine components for machine learning, and you have -- JAWS, I'm sure you can help me out here, how many of them in active use?
00:19:21 ◼ ► But if you've got a billion of these devices with a powerful neural engine distributed, that's a tremendous amount of cumulative machine learning power.
00:19:35 ◼ ► And here's the phrase I distinctly circled and underlined and highlighted in my notes from the keynote.
00:19:47 ◼ ► Which emphasizes both the features, right, you get this cool stuff, but also implicitly, privacy.
00:20:00 ◼ ► So a lot of the features we launch are a lot of the training that is done is done centrally, but not done centrally on customers' data that we grabbed,
00:20:21 ◼ ► So when you talk to Siri now, or when you write your handwriting, or when you hold up your camera now and it interprets text that it sees on your whiteboard,
00:20:36 ◼ ► But when we trained the thing that could recognize that handwriting, we did a lot of data gathering. We went out and had a lot of people write.
00:20:45 ◼ ► You know, a lot of people write in a lot of languages and a lot of different handwriting styles.
00:20:50 ◼ ► And then we got very, very good at synthesizing data, taking a little bit of data and saying, let's stretch it, let's contort it,
00:20:58 ◼ ► let's extract elements of style and use that same style to generate even more text so that we could amplify the data sets we have for training.
00:21:07 ◼ ► And so, it's not that we still don't need a lot of data today to train great models, but we don't have to take our customers' data to go do it.
00:21:20 ◼ ► And as you point out, there are techniques, one of them is called private federated learning, where you can, in a way that never has the customer's data leave their device,
00:21:29 ◼ ► do a kind of distributed learning exercise to try to figure out are there words that people are typing on keyboards that we don't know
00:21:36 ◼ ► and that it would be good to have our language models understand. And so, there's some of that happening.
00:21:40 ◼ ► And there's also something that's personalization on-device. We have techniques, and actually there's some sessions at WWDC about it,
00:21:47 ◼ ► where you might ship a model, but then you want it to personalize just to that user, so do some local learning in addition to make the model work even better.
00:21:56 ◼ ► So, there are lots of different ways to slice this, but I think the key is you can do it all without taking the shortcut of taking your customer's personal information to build that intelligence.
00:22:09 ◼ ► And we continue to explore all the techniques to do that while maintaining privacy, and the field is advancing rapidly in that regard.
00:22:20 ◼ ► One of the things, it's always hard to say, what is, alright, lots of great announcements on Monday.
00:22:33 ◼ ► So, go back one year. Here is my single favorite iOS 14, 14 from last year's feature, and it is the photo widget for iPhone.
00:22:46 ◼ ► I've got it on my second home screen. I see it every day, and every day it surfaces at least one photo from my photo library.
00:22:56 ◼ ► I don't know what the schedule is and when it decides to change it, but I've never had to configure it. I don't have to say, "Oh, I'd like a new one every six hours."
00:23:03 ◼ ► I don't know. It's just every few hours, I go to that screen and I see a photo, and the quality of the photos it chooses, and the fact that they're not ones that I even hearted.
00:23:16 ◼ ► I can't tell you how many times one's come up, and I'm like, "Why didn't I heart that?"
00:23:21 ◼ ► Maybe I imported it back before a Photos app had the heart button. I don't know, but this is the greatest picture of my son from 15 years ago.
00:23:36 ◼ ► I send these to my family every day. I'm like, "Oh my God." I've sent hundreds of texts to my family with this.
00:23:53 ◼ ► I'm right there. Same thing, same thing. Every day, "Ooh, look at this one came up," because you get the whole set of featured photos, and I'm with you.
00:23:59 ◼ ► I'm sending them to friends. I'm sending them to family. I'm like, "Oh my God." I always put the date, July 2011.
00:24:08 ◼ ► It's really just—and it really is a great way to—all right, I don't know. Let's say I have 20,000 photos in my library. Probably we have more.
00:24:17 ◼ ► Whoever goes back and looks through 5,000 photos from the year 2006, I don't. But it's uncanny.
00:24:26 ◼ ► It's like having a personal assistant who you trust with your family photos to just cheer you up every day with this great photo.
00:24:34 ◼ ► Well, Craig and his entire team, they've been great because we spend so much time, and we talked about it earlier, talking about what amazing photos we take with the iPhone.
00:24:44 ◼ ► And they are amazing. But they're only useful if you go back to look at them. And these guys, Craig and his photos team, have engineered so many great features for going back and everything from memories to the photo widget to different ways to highlight these photos so that you actually get to do what you took the picture for, which is enjoy it.
00:25:01 ◼ ► Enjoy that memory and enjoy it throughout the years. And I'm with you. I'm just—it is life-changing.
00:25:07 ◼ ► You know, one of the things that I've found is that one photo that comes up in the widget will pull me in, and then I'll play a memory back because then you want more sometimes, you know?
00:25:21 ◼ ► And one thing we found in the past that you play these memory movies, and it would be showing you a bunch of photos, and then you go, "Oh, no, I want to look at that one again. Like, it'll swing by." And then you're like, "Let me look at that."
00:25:34 ◼ ► And you kind of couldn't, and you'd, you know, you'd seek back. It was like video. It was hard because then you want to send that to the family and so forth.
00:25:42 ◼ ► And so it's those kinds of experiences that we all had over the last year that had us saying, "You should just be able to pause the memory you're watching and swipe back and look at the one you want."
00:25:54 ◼ ► You know, because I think we're all just going through this experience of kind of rediscovering our photos, of taking these journeys through the photos that were kicked off sort of unexpectedly by the widget.
00:26:08 ◼ ► And so, adding this, you know, what's going to be the hit for next, when we have this conversation next year that we'll look back? I don't know.
00:26:15 ◼ ► But I do know that the ability to just be in control when you're watching those memory movies and go back and sweep forward and pick one and share that is certainly something born out of this very experience we're talking about.
00:26:31 ◼ ► And I hope it's another way that makes people feel like all those great photos you're taking, you enjoy.
00:26:37 ◼ ► And I'll say, I don't know if you guys find this, I take more, having realized now what kinds of old photos I'm seeing that I enjoy, now when I'm out just taking a walk with the family, I think like, "I'm just going to take a picture of this because someday I'm going to love when it comes and shows it back to me."
00:26:54 ◼ ► Right? The fact I'm getting that reward out of having those photos come to me is making me think to take more photos. And so it's a virtuous cycle, I think. It's really rewarding.
00:27:06 ◼ ► One of the little things I noticed in the keynote was, I think it's State of the Union too, actually, which is, I always say, every keynote at WWDC I get a couple texts from people saying, "Hey, isn't this a developer conference? What's going on in the morning?"
00:27:21 ◼ ► And it's like, how many times do I have to say this? The morning is like the consumer keynote. It's like the roadmap for the year and the State of the Union or the platforms, State of the Platforms, I forget. It's the developer keynote, right?
00:27:32 ◼ ► But both of these, and it's both great. But I noticed this year that a lot of the presenters were showing examples of photo memories, and it was them, right?
00:27:44 ◼ ► It was, that's you rock climbing. Wow, you're nuts for doing that. But that's you, right?
00:27:52 ◼ ► Yeah, Chelsea. I learned something new about Chelsea as we were preparing for the show. She's an incredible rock climber.
00:27:59 ◼ ► Yeah. I mean, it is developer conference, Jon, but what you call a consumer keynote is just a keynote.
00:28:06 ◼ ► Right? For us, it's aimed at both consumers and developers. You have to tell the story.
00:28:12 ◼ ► You have to tell why are these things going to be useful to people. So many of them require developers or allow developers to take advantage of them and bring goodness to them to take them even further.
00:28:26 ◼ ► But you have to explain why does the consumer want to do this in the first place, of course, to make sure that developers are then going to be interested in doing it.
00:28:33 ◼ ► So we get very excited about those keynotes. And we, again, truly, we don't look at them as a consumer or developer keynote. It's for both, right?
00:28:44 ◼ ► We're just excited to launch this platform, right, that I think for the developers are some of our most excited, deep, appreciative users to see like what's new, what have we done, and where are there new, exciting things for developers to participate in.
00:29:03 ◼ ► And so to lay out the whole vision of that release and say this is what everyone's going to be running and what we as developers can all be a part of, I think sets the stage then to go a level deeper in the afternoon.
00:29:15 ◼ ► But I don't know how you'd tell the story about how developers could participate in SharePlay without first showing what SharePlay is going to be like for the people who use it, right?
00:29:25 ◼ ► You tell that story and the developers are like, wow, cool, what could I do with that? And then we get to the afternoon, we get to dive in a level deeper and say, hey, this isn't just about watching television together or listening to music.
00:29:36 ◼ ► Like, you could build a shared whiteboarding experience, you could do a co-gaming experience, or there are many, many things you could do.
00:29:43 ◼ ► And we get to talk about the technology. So I just think it's a sensible way to lay the common foundation for us all then to have the deeper conversation.
00:29:52 ◼ ► Yeah, I guess that's another way. I didn't mean to totally separate consumer from developer, but just sort of if you have that developer mindset, you've got to wait till the end of day on Monday, right?
00:30:03 ◼ ► Wait until you've seen them both. And it's sort of like, here's the feature, and then in the afternoon, now let's turn it upside down, take the screws off, show you how it works, and show you how you can do this too, right?
00:30:16 ◼ ► But it's like, we're not going to show it to you in the morning and then take the screws off and start putting SwiftCode up on a slide.
00:30:22 ◼ ► Well, and as Joss knows, he and I do a little bit of a tug-of-war sometimes where I want to open the hood right there, and he's like, well, hold on.
00:30:31 ◼ ► We do have several million people watching this who might not be interested in seeing you turn those screws.
00:30:51 ◼ ► There's reasons more than COVID that you're not sitting right next to each other right now.
00:30:59 ◼ ► Here's my spitball for the what will we be talking about next year from this year that people are like, oh, this changed my daily life, and that's live text.
00:31:09 ◼ ► Craig, you already mentioned it. And people keep texting me things from after Monday where they were impressed right away.
00:31:18 ◼ ► They're like, I can't wait to use this. I totally need this virtual whiteboard type thing. I totally need this text from the photos.
00:31:24 ◼ ► I've had like three people text me like, whoa, wait, this actually applies to all the photos I've already taken? All of them?
00:31:43 ◼ ► Absolutely. It's amazing. One of the people working on the feature, and you know, it's integrated into Spotlight as well.
00:31:53 ◼ ► And he was, when we do these demos, and it's pretty dangerous, people are doing most of their internal demos with their own photo library, so you really get to know them.
00:32:04 ◼ ► I mean, literally, you'd find the name, like, you know how the dog tag would, you know, the dog was posing, and the little dog tag is there with like Spot on it?
00:32:15 ◼ ► And he searches for Spot, and it comes up, and you look down, and you're like, oh, there it is. It's, you know, dangling on this picture of the dog.
00:32:22 ◼ ► Text is everywhere in our photos. Like, you just realize that it's in the name of the restaurant where you took this photo that you were posing outside of, and there it is.
00:32:32 ◼ ► And so, it's really going to be useful looking backward. You know, it's also great, heck, I used it yesterday where I just was taking a screenshot and wanted to copy the text right off the screenshot into some notes.
00:32:48 ◼ ► Right, so text is all over the system now, and it's just, you know that feeling when years ago, you could, on Mac UIs, you didn't used to be able to just select text off of user interface elements.
00:33:01 ◼ ► You know, it was like, that was, that was, it was the editable text, and then UI text might as well just been pixels on the screen.
00:33:08 ◼ ► And then when Mac OS X came along, like text basically became an object you could interact with. It was so liberating.
00:33:14 ◼ ► Well, now that's what's happening with images of text, and already in living on the feature, I'm surprised how liberating that is, how much text has been, you know, rendered into a web page, into an image or something that you wanted to get at.
00:33:26 ◼ ► And now, you can just select it. So, this isn't, you know, this isn't just about the ability to recognize text, it's the way it's been integrated into the experience.
00:33:38 ◼ ► Just any web page, just select, any photo you already took, just do it. It's, I think it is going to be really one of the big ones when we look back next year.
00:33:51 ◼ ► I noticed during the keynote, too, you humbly did not point out the degree of difficulty of several of the examples. Like, for example, the phone number of the restaurant that was behind the subject, which was blurred because there was a guy at the restaurant who was the focal point of the image, and it was a little bit out of focus and nailed it.
00:34:13 ◼ ► The handwriting examples were not really, like, from calligraphers, they looked like real handwriting. And the pictures of the—you always had them at, like, a skewed angle, and, you know, it's like 30 degrees off, and it's like, nope, just works.
00:34:27 ◼ ► Just run your finger over them on iOS, run your mouse over them on Mac, it'll select the text, it'll get it.
00:34:35 ◼ ► You know, our team—the legacy of this feature comes from going back to doing handwriting recognition, you know, we were doing Chinese handwriting recognition, we were doing scribble, and we realized people would write at crazy skewed angles, generally.
00:34:52 ◼ ► So, we started, like, text just doesn't come in perfectly aligned in the real world. And the team put so much effort into handling skewed text, rotated text, text that's like bent around, you know, the kind of marquee, right?
00:35:07 ◼ ► You want to be able to read that. There's a lot of work that went into that. And I mentioned that when we gathered data, we also synthesized text, so we took handwriting, but then we trained our systems by taking that handwriting and skewing it and rotating it
00:35:21 ◼ ► and doing all the things that happen in the real world, and so we were able to make the system robust to real world text. And it works. It's really pretty remarkable.
00:35:31 ◼ ► Well, and we talked about the power of Apple Silicon, and that's one of the things that, again, we—Craig's team is able to take advantage of the neural engine, you know, to make that all happen, you know, instantaneously.
00:35:42 ◼ ► It's interactive speeds, because if you think about it, yeah, you could have in the past run in the background overnight on your photo library and done a bunch of analysis, but in this case, you can load a web page up or take a screenshot, and you want to instantly start selecting that text.
00:35:56 ◼ ► And it's because the neural engine is so amazingly fast that we can instantly recognize there's text on there and instantly convert it so as you just click and drag, it works. I mean, this is—it is incredibly liberating for us to have that kind of power in the silicon.
00:36:10 ◼ ► And again, it's all on device. The other new on-device intelligence that is new this year is that for, I think, seven languages—I forget, but for a lot of users in a lot of places around the world, Siri processing is now on device.
00:36:26 ◼ ► And that's not just a privacy feature. The demo Eric gave during the keynote, you know, all in one shot, one take, one request after another, to be honest, is not the sort of performance that you could get out of Siri before.
00:36:44 ◼ ► I mean, it is— The speed is awesome. We're all living on it, obviously, and it is remarkable. Yeah, it's, you know, not only do you not have to care if you're on the network for those kinds of requests, but the responsiveness is a huge win.
00:37:02 ◼ ► And that's generally true of on-device machine learning. You know, you don't want to deal with, "Is my network flaky? How's my connection right now?" You want it to work wherever— I mean, these are mobile devices, right? You want them to work well wherever you are, and getting Siri to be local in this way is fantastic for the user experience.
00:37:22 ◼ ► Yeah, I'll tell you, Jon, I'm a huge Siri user. I use it for everything. I use it for dictation. I use it for constantly reminding myself to do something at a particular time. You know, it changes the way I use it. I use it even more because it's so predictable now in how long it's going to take.
00:37:38 ◼ ► Because as you know, if you get a bad network connection, you know, it reflects badly on Siri, but it wasn't really Siri. It was the fact that Siri had to send a bunch of stuff up to the cloud, get it converted back to voice, do all a bunch of stuff with it, and your network connection could really slow you down.
00:37:51 ◼ ► Now that it's all local, it's just like Eric showed. It's just phenomenal, and I find myself using it more and more. And I'm very excited. It's actually near the top of my list, too, of things I'm excited about.
00:38:07 ◼ ► I just think of the last two years where if you listen to Siri's neural text-to-speech, like this incredibly high-quality voice, it felt like a moonshot a couple years ago to do that locally, to not have to send what you wanted to say to a server to synthesize it at that quality level.
00:38:28 ◼ ► And the A&E let us do that on device, and now to be able to also use that same A&E on the front end of the request to understand what you're saying and then to run the network to not only understand it in terms of turning it into text, but to do the natural language processing to understand what that text actually meant in terms of intent.
00:38:48 ◼ ► I mean, it's been just an incredible leap forward in the last year. So, it's really exciting where we're going with on-device learning, for sure.
00:38:57 ◼ ► I'm stealing this observation from a good friend, but my favorite example of the difference that on-device Siri makes is that now you can use it to turn airplane mode off and back on.
00:39:17 ◼ ► That was slipped into Eric's demo. He just casually turns airplane mode on, and then he keeps talking to Siri. And I think the observant in the audience probably caught what was happening there.
00:39:33 ◼ ► So, still on the privacy beat, the app privacy reports. Now, this is the thing, there's a footnote that says this will be coming later in the year.
00:39:42 ◼ ► And you guys have switched to a sort of, hey, some of these features we're announcing will roll out after the initial release of the new OS.
00:39:51 ◼ ► I mean, iOS 14 is up to iOS 14.6. There is a lot of action post .0 on your annual schedule these days.
00:40:01 ◼ ► The app privacy reports are, to me, it's like, yes, I have wanted to know this forever.
00:40:12 ◼ ► If I'm using some app and it has a bunch of these frameworks that are embedded from marketing partners, whatever they call them, how often is it calling home?
00:40:23 ◼ ► And is it like 13 of them? Is it 12 of them? I mean, and that seems like what these app privacy reports are going to show users.
00:40:32 ◼ ► That's absolutely right. And this isn't the kind of thing that we expect all billion-plus users to activate and analyze.
00:40:41 ◼ ► But we know there are a lot of people who do want to know, and to give them the power to, you know, the transparency to see what's happening, to look into those things, I think is going to be really important.
00:40:56 ◼ ► And I think it does help keep everyone honest, for sure. And now you can do it without maybe trying to run some kind of, you know, other more invasive VPN or something.
00:41:09 ◼ ► There are tools in the past, which you can still use, but that you're giving up a little bit more than that to try to get this insight.
00:41:14 ◼ ► Now, just flip a switch. It'll start capturing all that information. You can export it to a file and run analysis on the traffic if you want, or just view it through the UI.
00:41:29 ◼ ► Next on the list is one, and again, I feel good about this. I feel like I hit the lottery this week because a lot of the things I've complained about got addressed.
00:41:38 ◼ ► But I was so happy to hear about the mail privacy protection, which is that Apple Mail is going to give users the ability to say, "No, I don't want to be tracked.
00:41:52 ◼ ► I don't want marketers to know when I've opened a message, how many times I opened the message, what was my IP address when I opened the message, because the image loads remotely and your IP address can backwards geolocate you."
00:42:11 ◼ ► And I'm curious if you guys got any pushback internally to Apple, because Apple sends marketing emails.
00:42:17 ◼ ► Is that something that you guys had resistance to internally, or was it like, "No, no, everybody's on board. This is nonsense that it's taken root in the industry that your email openings are available to the sender."
00:42:31 ◼ ► John, first of all, when you wrote about it a couple months ago, we got pretty excited because we thought, "Oh, John's going to like this one."
00:42:38 ◼ ► Occasionally a customer will write something like this, and then like a week later we'll announce it, and they'll say, "You're welcome."
00:42:44 ◼ ► And I'm like, "Well, I know we're productive, but did you think..." So yeah, that one was already underway.
00:42:51 ◼ ► Yeah, but I will say, John, when it comes to privacy, there's two words that we're always telling ourselves, and it's the pillars of people's privacy, which is transparency and control.
00:43:02 ◼ ► You want users to know what's happening to them, and you want the control to be able to say whether that's okay or not.
00:43:08 ◼ ► And this is a classic example of that. Most users had no idea whatsoever that that email that they were opening was not only telling the sender that they opened it, but sending back their IP address,
00:43:22 ◼ ► which again allowed them to potentially add that to other activities they were doing online with that IP address, and allowed them to get back to location, as we talked about.
00:43:33 ◼ ► So for us, it's just that straightforward. It's like we want transparency and control. We say privacy is not an end, it's a journey, right?
00:43:42 ◼ ► And this was one of the things on that journey which so much needed to shine some light on over, again, transparency and control, and I think the team has again nailed it.
00:43:52 ◼ ► And we felt like users should have a choice. So we do, when you first launch mail, we tell you there's the Existences feature, here's what it'll mean.
00:44:00 ◼ ► Do you want your mail to be fetched directly, which has these consequences, or do you want it fetched with these privacy protections?
00:44:07 ◼ ► And we'll let customers make their choice. But as JAWS says, prior to this, when I've talked to people and said, "Well, by the way, did you realize that?"
00:44:17 ◼ ► They're just like, "No way." Right? And you brought it up, Jon, in your site, but by and large, the average consumer just doesn't know, so we think we'll give them a choice.
00:44:29 ◼ ► And I think part of it dates back, I mean, I'm sure you guys have both been on email forever, too, you know, at least, I mean, early 90s for me in university.
00:44:39 ◼ ► And if you told somebody back then, in the early days of internet email, that it would someday be possible that when you open an email, the person who sent it could tell that you opened it.
00:44:51 ◼ ► They would have been like, "No, that's not even possible." But it's like email, it's sort of evolved in this, with a utopian mindset of what would be cool without considering the weight, how could this be abused?
00:45:06 ◼ ► I mean, spam, of course, is the most famous example, where at some point somebody figured out, "Hey, for like $10, I can send a million emails. Why don't I just do that?"
00:45:16 ◼ ► And it's like, well, here we are now, we have all this spam. And tracking sort of went that way, too, where it used to be that email was plain text, then it got to be rich, and then it was like, well, it would be cool if you didn't have to send all these images, we could load them remotely.
00:45:29 ◼ ► That would be a cool feature. It is a cool feature, it makes email look a lot better. And then there's this whole dark side to it, and like a cottage industry.
00:45:37 ◼ ► I think there's such a history of that with internet technologies. I mean, even cookies, right? I mean, you probably remember when cookies were first proposed and what the examples were, what they were for.
00:45:47 ◼ ► I'm all but certain that the people who first thought of cookies never conceived that they would be used for cross-site tracking in the way they ultimately were.
00:45:57 ◼ ► But the technology gets used both for what it was intended for and many other things. And I think the evolution of, "Oh, yeah, HTML, that'd be great. Let's put that in emails." Whoa, who would have thought?
00:46:11 ◼ ► But this is a lot of what we have to do every year to continue to protect privacy is root out these cases where maybe the way the technology has ended up being used is not fitting with customers' expectations and make sure they have an understanding and the ability to affect the outcome.
00:46:30 ◼ ► Is there another side to mail privacy protection in addition to trackers? Is there something where, like if you're using iCloud, or would it work with other email providers, where it also protects your IP address as the originating, you know, like the way in your headers it says "received from this IP address," and it has this whole chain of the mail servers as it goes from A to B? Or is it just about trackers?
00:46:53 ◼ ► It's just about trackers. So, the actual SMTP protocol as the message that is sent is relayed from server to server and the chain there. Our mail prevention is not about that. It's about you having received the mail, reading it. Yeah.
00:47:13 ◼ ► So, there's iCloud private relay, but is there also something separate from iCloud private relay where Safari is going to do IP address masking, or is that related?
00:47:24 ◼ ► Yeah, yeah, actually, so you can see that there's this relay technology that we were finally able to get deployed at scale, and we take advantage of this even for mail tracking prevention because the way that we're fetching these images that are referenced in emails you receive is through a relay.
00:47:42 ◼ ► There's a use built in now to Safari, whether or not you use iCloud+, where your contact to known trackers occur through the relay. So, just, you know, you contact CNN.com, say, and let's say CNN.com includes references to known trackers, and they attempt to do accesses.
00:48:11 ◼ ► Safari, by default, as part of intelligent tracking prevention, now uses the relay to make sure that your IP address isn't going out in those tracker requests. So, that's awesome, and that's for everybody using Safari.
00:48:24 ◼ ► And then, with iCloud+, you have this additional level of all your other traffic on Safari going through the multi-hop relay.
00:48:35 ◼ ► I've seen a couple people who, I know it's not a VPN. This is a very different, and it's Safari specific, but can you talk about the differences between iCloud+'s privacy protection for Safari versus using a VPN?
00:48:51 ◼ ► Yeah, I think, of course, VPNs are sometimes used in a corporate context. You're connecting to your company, and all of the traffic, sometimes they can be configured as a full tunnel or just to access your corporate resources.
00:49:05 ◼ ► In those cases, your traffic is, all those accesses are essentially going through a centralized point, which is that of the VPN operator. And so, the VPN operator knows who you are, and they know everything you're accessing.
00:49:20 ◼ ► And so, as VPN technology also then was brought to market as a more commercial solution for consumer access, the same fundamental thing applied, which is, you know, you were paying for a subscription, they knew who you were, and they're routing all of your traffic, so they know everything you are accessing.
00:49:39 ◼ ► So, you're putting a lot of trust in this single centralized entity to understand all the sites that your system is accessing. With iCloud Private Relay, we use a two-hop solution that ensures that no single entity, not even Apple, sees both who you are and what you're browsing, right?
00:50:01 ◼ ► That there's one hop of encryption that goes to Apple where where you're going is encrypted. We forward that to someone else who now doesn't know who you are, but they do know where you're going, and they route the traffic.
00:50:15 ◼ ► And so, both of the hops have different information. They aren't joined together. So, you don't have to make this difficult trust decision around a single centralized entity to get the protection that you can browse without revealing your IP address to the desk, either a centralized entity like your VPN provider, or, of course, to the destination that you're browsing.
00:50:37 ◼ ► And so, in that sense, it's quite different than VPNs. The other difference is many people, many VPNs can tend to obscure completely where you are. Like, maybe someone wants to get Netflix, but they're not in a country that has this content available on Netflix because Netflix has regionalized things.
00:50:59 ◼ ► iCloud private relay isn't meant to obscure what region you're in. In fact, normally when you're doing normal browsing, it can be useful that you get regional level understanding of where you are, that the site that you're accessing knows that you're in the San Francisco Bay Area.
00:51:17 ◼ ► They don't have to know where your IP address or where you are, but it's useful for them to find the right service to know regionally where you are. We're not obscuring that. That's not the goal.
00:51:29 ◼ ► In a related area, and you brought this up, and to me that's one of the big things. Like, if you work for a company, I know that Apple has VPN services that you guys connect to when you want to work from home, which, guess what, a lot of every Apple employee was doing for the last year.
00:51:48 ◼ ► And if you work for a company, you implicitly trust your employer. You can trust their VPN. If you're a consumer, there are trustworthy consumer VPNs. There's a lot of apps in the App Store you can choose from, but you are putting your trust in them, and I feel like some people don't quite understand that.
00:52:05 ◼ ► And then on a related issue, a couple of years ago, when iOS added some parental controls, but also adjusted the rules for some of the apps in the App Store that were geared towards families, but were sort of using like enterprise technologies, things that were, I forget the exact API, but it was sort of like
00:52:29 ◼ ► MDM. And the accusation against Apple was, oh, they put the kibosh on these apps in the App Store because they were adding this feature. But you guys weren't selling that feature. This wasn't like, hey, give us $5 a month and you can do this. This was just added.
00:52:49 ◼ ► So there's sort of like a dot dot dot profit, you know, sell your underpants or whatever the meme is. It's like, I'm not quite sure where you're going with that. It seems like Apple's explanation that MDM was never intended for this, and Apple wasn't comfortable with everything that your kids do on their devices going to these third parties.
00:53:14 ◼ ► And I saw this year, this is one of the things that I saw is new in this year's OSes, is proper APIs for third parties to create family control types. I forget the marketing name for it.
00:53:33 ◼ ► Yeah, I mean, there's three separate actual frameworks involved in a privacy preserving way of implementing these screen time kinds of facilities because there are lots of things that we think can be really valuable.
00:53:50 ◼ ► Someone wants to implement some scheme that says, hey, you want to configure homework time and if you spend this much time doing homework, then you get this much time and you know, the game or whatever it is. I mean, we don't have a service for doing that kind of scheduling, but it seems like a great opportunity for someone to provide a solution to families that do.
00:54:13 ◼ ► But we wanted to do it in a way that wasn't saying, you now know every app the child is using at all times. We wanted to give an API that could provide that kind of control, that lets you build that kind of facility without violating their privacy in that way.
00:54:30 ◼ ► And so, we actually have ways that in your app, in the control app, when the parent say is picking, oh, I want them to do their homework in these apps, the identity of the apps that are being picked are hidden from the management software.
00:54:47 ◼ ► They're abstracted into an anonymous token. So, it puts up a system chooser, you say, oh, this app, this app, this app, we hand back to the app some opaque identifiers, they can in turn tell the framework, you know, disable these apps or activate them at this time or, you know, tell me how much time they spent in these things, but they don't, they can't know strictly what they are.
00:55:08 ◼ ► And so, you're now able to provide all the control, all the interesting kinds of support that the parent might want, but without just suddenly saying, here, it's open season on understanding how the child is using every aspect of the device.
00:55:23 ◼ ► Yeah, it seems like a great solution and it, you know, seems to be what you've been saying all along about it, increasing the ability for parents to have control over screen time and what kids are doing and when.
00:55:35 ◼ ► Next major topic, and man, am I happy about this, iPadOS 15. I've had some critical remarks about iPadOS multitasking in the past, I don't know.
00:55:47 ◼ ► You know, I don't remember any of those. I don't remember the repeated floggings. I have no recollection of those.
00:55:55 ◼ ► It was the first beta I installed on one of my devices, and I have to say, a lot of my complaints are fading from my own memory. I really, I will tell you, I did almost all of my prep work for this show on my iPad, running iPadOS 15 with two apps side by side and with a lot of different apps.
00:56:17 ◼ ► And I really like what you guys came up with, and to me, the biggest change is that so many of the controls are now visual, and maybe it's because I'm a visual person, but things that used to be gesture only are visual.
00:56:32 ◼ ► Here's a three-dot button at the top, a multitasking button, and it does the same thing every time you click it in every app, and it's very obvious what it's going to do, and even if you want to go side by side, when the current app goes to the side, you still see it over there, right?
00:56:47 ◼ ► And it gives you this visual context, a sort of spatial awareness of where this is. I'm just curious what aspects of that you guys like best, because I'm sure you've been using it for a while.
00:56:59 ◼ ► Yeah, for sure. Well, I'd say when we did the previous round of multitasking, we were, it was also the time when we were really bringing drag and drop pervasively to the system.
00:57:13 ◼ ► And so, a bunch of things were converging for us, where we were bringing out the dock where you could access apps, and we were doing drag and drop as something we felt could become a really dominant way of interacting with iPad.
00:57:27 ◼ ► So we leaned really heavily on that. But you're certainly right that drag and drop is essentially a hidden capability. There's nothing there that's saying drag this here to do that.
00:57:39 ◼ ► And so, this year, we really were focused on saying, let's make sure there are those breadcrumbs at every state of the operation. There's something telling you you want to do something? Oh, there are these dots.
00:57:49 ◼ ► You tap on the dots, here are the things you can do. You tap one of them to say I want to go into split view, great. Animate it out, let you pick your app. Right over here is still the app that's moved aside. You can see it, you tap it, it'll come back.
00:58:02 ◼ ► You know, making, having that visual breadcrumb at every step, I think there are just certain large portions of our user base where that's a really, really important element of the design to have those visual breadcrumbs.
00:58:13 ◼ ► And so, we have them. We also made sure that there were some cases in our old design where you could do something in one direction, essentially, and then the trip back that was like a break in the cemetery, like it was hard to get back out in the way you'd expect.
00:58:28 ◼ ► We were able to clear all that up. And we also introduced this new idea for us called the shelf, which gave you a place if you were working on something, had a window up there that you could, you know, drag a draft down there and get back to it later, or something
00:58:45 ◼ ► I think we only found in experience, but certainly something I experienced is at some point I'd open up a couple of Safari separate spaces or windows, or I'd open up mail messages, long mail messages I wanted to read in separate windows.
00:58:58 ◼ ► And then I'd complete, they were out of sight, out of mind. I just completely lose track of them. And so, the shelf plays this great role that as you launch an app, you know, you bring up Safari, it'll show you kind of the last Safari window that you accessed,
00:59:12 ◼ ► but it'll also show you at the bottom all the other ones you have open. So if you're like, no, no, I didn't want this one, I wanted one or the other, others, now they're really easily accessible. You know, you can tap and open any one, you can say, oh, this was, you know, I was doing that a couple days ago, I don't care about this anymore, flick, delete it.
00:59:27 ◼ ► So, so again, it's a visual breadcrumb presence of mind about what's there. And then finally, quite honestly, this is something we've wanted to do for like four years now.
00:59:37 ◼ ► In the multitasking overview, you know, when you see all the apps in the grid, just being able to manipulate the pairings, unpair this, drag these two things together. It's just one of those things that you can see it, you know what you want to do, why not just be able to do it right there in the overview.
00:59:52 ◼ ► And so I think that's really powerful as well. So I feel like this all comes together in a really coherent set this year. And then we added keyboard bindings for all of these operations as well.
01:00:02 ◼ ► So you can keep your hands on the keyboard and create a split view, you know, going to split you spotlight to pick the app and boom, you've just brought up a second app and it's scriptable via shortcuts.
01:00:12 ◼ ► So I feel really, really good about this. And I want to let you know that you have been recorded saying that you like this year's design. So this is this is going in the permanent record.
01:00:30 ◼ ► I feel like I'm back in high school, I had a lot of times I was told stuff was going on my permanent record.
01:00:37 ◼ ► I there were two of the new features for iOS last year were widgets on the home screen anywhere and the app library and they were absent from iPad OS.
01:00:48 ◼ ► And it seemed a bit curious, but my take now seeing this is those were parts of this larger story of multitasking and the way that the home screen is part of the multitasking right like on on on both iOS and iPad OS, the home screen is how users interact with apps, right?
01:01:10 ◼ ► That's where they are. That's where new apps appear. That's how you delete it if you want to delete it. And of course, it's how you tap you tap it to open it. And that's the multitasking interface, you slide down or you know, hit the three buttons and now you're back on your regular home screen.
01:01:24 ◼ ► There's your apps, there's the one to just tap it and it'll fill the spot. But is that is that sort of the thinking that this let's not just shoehorn these two iPhone features in let's do this right.
01:01:40 ◼ ► Well, I think we certainly wanted to do it right. And we knew that doing it right for iPad was not going to be just doing exactly what we done on iPhone when it came to widgets.
01:01:52 ◼ ► And we certainly knew that doing widgets right on iPhone itself was a big important job. And so an app library as well, there wasn't a just ported over kind of thing. Like when we did app library, we made it part of the dock and it has a bit of a different flavor that makes sense for for iPad.
01:02:12 ◼ ► And when we did widgets this year, we realized we want to do this bigger class of widgets because you've got this bigger device. Sometimes having a widget that can really take advantage of that space was important.
01:02:24 ◼ ► And so it really made sense for us to focus. You know, a year before in iOS 13, we'd actually brought widgets in a way to the iPad home screen as a as a column that you could lock there and so forth.
01:02:38 ◼ ► So we felt like we were in a good place on an iPad and wanted to take a year where we could really focus on iPhone, then take what we learned out of that and then really do a great job on on iPad.
01:02:51 ◼ ► And so sometimes sometimes we'll do that. You know, if you try to cram too much in and in one point, you know, I think I think you you avoid the you miss out on the opportunity to learn and do a better job on the next round.
01:03:05 ◼ ► One of my favorite little things, and it's been a bigger unscratched itch since the tremendous trackpad mouse support from mid mid cycle last year and the magic keyboard as you know, as a peripheral you can connect your iPad to.
01:03:24 ◼ ► But the the fact that when you're on the home screen and you have a keyboard connected at the moment, you're not just holding the iPad, you got a keyboard, you can now use the arrow keys to just move the focus around the apps on your home screen.
01:03:36 ◼ ► Arrow arrow, you get a little, you know, outline around it, hit return, and it opens. And it's a little thing, but it felt like the sort of little thing that should work right like and that's always been to me the hallmark of the best parts of the Mac is, is that
01:03:53 ◼ ► the way you learn you go from being a simple user to an expert user is you figure out the idioms and then you think, well, I've never tried this, but I think, you know, like you're in a list like, shouldn't I just be able to type j to jump to JAWS in the list of I'm in, it's like, Oh, that works.
01:04:12 ◼ ► Yeah, I can just type J doesn't it. And it's always felt to me once I got used to using a keyboard with my iPad that it seems like I should be able to just use the arrows to move around and now you can and it seems like that is probably related to the fact that you can now use a game controller to control your iPad.
01:04:28 ◼ ► Not just to play games. Yes, I mean, you're right that there was a, there was a broad push to really take our whole keyboard focus handling and bring much of our learnings and expectations from the Mac and bring them thoroughly to iPadOS this year.
01:04:49 ◼ ► So you could be in the mail app and tab out of the search field and down into the message list and then tab over into the main body of the reading part of the view and so forth. So all of that sophisticated keyboard focus handling is was a focus.
01:05:05 ◼ ► This year, we wanted to bring that to the home screen. And as you point out, we also for gaming, you know, we know if you have a game controller in your hands, you'd like to just keep it in your hands and be able to launch the game, you know, pick the game launch the game, exit pick another game.
01:05:25 ◼ ► And so allowing all of that manipulation to occur with the game controller without putting it down reaching over and doing this, I think just makes a much, much more enjoyable experience with the game controller. So those are two things that really came together in a nice way this year.
01:05:45 ◼ ► is universal control, which was again, a scene stealing demo like, Whoa, this is crazy.
01:05:56 ◼ ► And but then it's like you, you know, it's like any good detective movie, you know, and it's like you get you see you see this unveil in Act Two, and then all of a sudden, your mind goes back to Act One and you're like, Oh, that's why blank and it's like, I see this universal control and and it's like, Oh, this is no wonder you know, and you like the the the sort of membrane that the cursor has to pass through, you just have to give a little bit of control.
01:06:25 ◼ ► You just have to give a little bit of an extra push to go across from like your Mac to the iPad. And it's like, Oh, that is totally just like the way the iPad circle cursor sort of encompasses the buttons that it passes over, or the you know, it takes the shape of the thing that it's hovering over, there's this sort of liveliness to it.
01:06:48 ◼ ► And it really just sort of makes sense with the way that iPad got this truly rich, terrific first class trackpad mouse pointer support. I'm curious how long I know you don't like to talk about how long you've been working on something. But when did that idea like that? It's such a crazy idea, but it now it seems like Oh, why wasn't this there all along?
01:07:10 ◼ ► Well, I do think it is one of those things where, to some extent, we've been working on it for 10 years, because the foundation that we've been building on, that it's built on all those pieces came a lot of threads got pulled together to to then finally realize that experience.
01:07:27 ◼ ► You know, with all the continuity experiences, the fact that our devices generally, that are signed into your account, know about each other, that they're via Bluetooth low energy can talk to each other. The fact with peer to peer Wi Fi, we can quickly establish a high speed Wi Fi link between your devices.
01:07:47 ◼ ► You know, this is something we did for airdrop years ago, right and had that as a foundation. But then you need the iPad, of course, to actually support a trackpad in order to make this meaningful, right? So that that set the stage.
01:08:01 ◼ ► Once we had all of those pieces, we we knew that we wanted it to be a zero setup experience, right that if you're going to do something like this, if there was this friction where let me bring up system preferences and lay out my stuff, you know, then you're probably not going to do it a lot of the time.
01:08:23 ◼ ► And, you know, we evaluated lots of crazy ideas about how do we use ultrasonic or something to locate where the things are, but there was a really kind of intuitive leap we were able to make where we said, well, you know where the iPad is relative to the Mac.
01:08:41 ◼ ► That's you're going to move the cursor toward the iPad, you're telling us something when you do that, can we take advantage of that to say, Oh, okay, and then give you this indication that says, we think you might be trying to do this, are you and take advantage of the visual language
01:08:59 ◼ ► we built in an iPad to kind of reflect that and make a really natural way to say like, pull it through. But what you've done is told us where it is and confirmed your intent, all in a way that just felt like the most natural way to perform the operation in the first place.
01:09:18 ◼ ► And so, so that all came together just, just beautifully. So it's one of those things that, you know, years of putting all the pieces in place made it possible and now we're finally able to capitalize on it all.
01:09:32 ◼ ► Yeah, the continuity stuff, it's the rare times when I copy something from my iPhone, and I can't just paste it on my Mac, or you know, it doesn't take the first time, every once in a while, it always feels like, Oh, that's crazy, you know, because I'm so used to it. And it's so much more convenient.
01:09:49 ◼ ► Like, I don't, I don't want to open this web page on my Mac, I don't want to airplay it. I just want to paste it into this mail message that I'm writing on my Mac, and I just want to copy it on my phone and do it and it's like it just feels it.
01:10:05 ◼ ► It's I know it's fancy technology. I know there's a lot going on, but it feels like the most natural thing in the world, even though they're not exactly the same computer.
01:10:21 ◼ ► It's very, it's a very Apple feature. And this just feels like, again, no pun intended, continuation of continuity. In the interest of time, I don't, there's a lot I could say about Mac OS Monterey, but I wanted to spend more time on iPad OS, because it's a bigger, it's a bigger year for iPad OS, because it's it's so much new stuff.
01:10:41 ◼ ► But I did want to talk about shortcuts for Mac, which is something everybody has been, you know, like, hey, well, you know, shortcuts is clearly a big deal.
01:10:51 ◼ ► The enthusiasm of users, like the Reddit groups and stuff for shortcut users is like, again, that to me is classic Apple of getting users who are maybe not quite full-fledged, Swift or Objective-C developers,
01:11:08 ◼ ► but who have that sort of tinkering mindset. And look, I've got this itch I want to scratch where I want to build up this system where when I wake up in the morning, I want my shades to go up, I want my lights to go on, I want some, you know, morning music to come on, and I want it to be just my way.
01:11:31 ◼ ► Shortcuts for Mac, that's the one thing I've poked around at so far in Monterey, and I really like what I see so far, because it really does.
01:11:41 ◼ ► I know, you know, it's what you said, and I'm not accusing you of ever saying things that aren't true, but it feels like a Mac app. It really does.
01:11:57 ◼ ► Right, which is, and it really does, and that to me, it's like I said, like with the, it's like SwiftUI is this UI component that's like the M1 to hardware, where it's like, hey, it does look a lot like the iPhone and iPad, but also these UI elements act like Mac elements.
01:12:14 ◼ ► But then the other factor is the, well, wait, what if they just move shortcuts with the abilities of iPad and iPhone over to the Mac? It's going to be missing out on all these things that make a Mac, the Mac from a tinkerer's, scripter's perspective.
01:12:29 ◼ ► And no, it's all the stuff you like. It's AppleScript. It's shell scripting. You can have actions that just run a Perl script if that's your thing.
01:12:38 ◼ ► Yes, there is a conversion story for it. Yeah, the future is shortcuts. Automator has not been put out to pasture yet, but if you just drag an automated workflow onto shortcuts, it'll do its best job and convert an awful lot of them into working shortcuts.
01:12:59 ◼ ► I'm curious on your perspective on, because to me, it's been a thing, and again, I know I'm dating all of us, but my mind goes back to HyperCard.
01:13:11 ◼ ► And it's sort of been a missing level of the tinkerer's level scripting on a Mac, and it seems like that shortcuts is bringing that back, but in a way that works everywhere.
01:13:28 ◼ ► And it fits with the system in a way that HyperCard never did back in the day. HyperCard was sort of this, many good things and fond memories people have of it.
01:13:37 ◼ ► It was like you sort of left the Mac and you went into HyperCard world, and instead of dealing with apps, you were dealing with stacks.
01:13:43 ◼ ► Whereas shortcuts, one of the great things about it is it feels totally integrated with your system.
01:13:49 ◼ ► Like I have these little stupid shortcuts that I've made for myself, and it just feels like I've made my own little custom app to just do my little stupid John Gruber thing.
01:14:00 ◼ ► I'm curious what you guys think of how many users are actually engaged in using shortcuts.
01:14:09 ◼ ► Well, I mean, there's a pretty large community and a really passionate community around shortcuts on iOS, and I think it's an astute observation that the shortcuts, having a community there that, where the focus is make something that hits the right sweet spot for iOS,
01:14:40 ◼ ► If you'd started again around, let's think about the next generation of automation on the Mac, you run the risk that it wouldn't make the transition the other way to iOS, and that you'd end up with something that was not as simple as it could be.
01:14:57 ◼ ► By starting with iOS and pushing the kind of capability there, really developing something that's very capable and yet really simple, and then taking it to Mac and then letting it spread its wings to tie into all of the different system capabilities the Mac has,
01:15:19 ◼ ► I think was a really great path to end up with something that really was worthy of replacing Automator, that was really a big step forward and not just another iteration, but a big step forward.
01:15:33 ◼ ► And so I do think there's a lot of people now who they have their iPhone, they have their iPad, they have their Mac, they're into automation, they shouldn't have to learn two different technologies in order to do it.
01:15:57 ◼ ► And the team's done a great job, and it's also been a fantastic internal case to push us on various elements of SwiftUI.
01:16:07 ◼ ► We've had so many internal adopters now SwiftUI all doing quite different things with it, and all of that is making sure that we're stretching the framework in a lot of great ways.
01:16:19 ◼ ► And so I think all the rest of the developer community is going to find SwiftUI to be that much more capable this year because of how much all year long we've been adapting it to meet all this next set of needs.
01:16:35 ◼ ► John, I'm getting word from some engineering project managers that if we don't get Craig back, we're not going to ship on time.
01:16:51 ◼ ► Do you feel like you guys, you released it two years ago, and it was very nascent. It was like, this is the future of UI code, and now it's really something. You built this great shortcuts app, mostly using SwiftUI.
01:17:07 ◼ ► Do you feel like that's a change from you guys, the way you usually release stuff? A lot of times you like to make a big surprise.
01:17:15 ◼ ► Here's this whole thing, and it's this whole SwiftUI framework, and it's four years later all at once.
01:17:23 ◼ ► Or is this an important part of getting developers into it where, look, we're going to show it to you in a very small state, and we're going to grow this, but we need the feedback from outside Apple to know which direction to build it.
01:17:38 ◼ ► Yeah, I think I certainly feel good about the way we launched it. We went in eyes open about the path we were on.
01:17:46 ◼ ► We wanted to provide direction for the community. We wanted to get something out there that could be useful in certain domains, and get a lot of feedback, and then be in a position to drive it forward to maturity.
01:18:04 ◼ ► While we still had some great alternatives out there for developers. I mean, UIKit and AppKit are still very strong alternatives.
01:18:11 ◼ ► So, I do think this was the right way to take a technology like SwiftUI and mature it in public a little bit.
01:18:20 ◼ ► And I think it's helped accelerate it and make sure it's hitting the mark. Even if we kept it internally kind of under wraps,
01:18:27 ◼ ► I don't think it would have matured quite the way it has, having the full community able to use it and give us feedback.
01:18:37 ◼ ► But I think it's in great shape right now. I certainly, in all of my weekend projects and so forth, I can't imagine using anything else at this point. It's really fantastic.
01:18:49 ◼ ► And the flip side of that modern day HyperCard story is, to me, Swift Playgrounds. And that it's the other aspect, like I mentioned, with HyperCard you could become an expert and it didn't roll you into becoming an expert Mac developer in MPW or Code Word or whatever the environments were at the time.
01:19:08 ◼ ► Whereas, if you get to the limits of Swift Playgrounds to make real apps using SwiftUI and you get to the limits, you can roll over to Xcode and it, literally, your project comes with you and you're not starting in Xcode, which is huge and does all these things.
01:19:27 ◼ ► From scratch, you're starting with your knowledge of Swift and SwiftUI and a project that's already working under your belt. And that, to me, is actually totally new to have this ramp from beginner, like a student level, learning the program to roll your project into the most professional IDE for all of the Apple platforms.
01:19:50 ◼ ► Yeah, absolutely. And I think the support to build SwiftUI apps now in Swift Playgrounds is going to have two audiences. It's going to have the person who was learning to program and they wanted to take the next step toward real app development and you can go very far with Swift and SwiftUI in there.
01:20:09 ◼ ► But there are also going to be professional developers and hobbyists who want an environment to lightly try out ideas and they're pros, but it's such a pleasant environment to just do some prototyping, maybe build a small app, and then they can be moving back and forth between Xcode, which they use for their day job, and SwiftUI, I mean, and Playgrounds because we have this compatible project format that's really simple.
01:20:38 ◼ ► So, I'm super excited in bringing real development to the iPad and being able to do it with these really modern, really intuitive frameworks with this pretty smooth ramp from moving around the little character bite around the maze all the way up to building real apps in SwiftUI.
01:21:01 ◼ ► All right, final question. There is, in my mind, a sort of, and maybe part of it is the last 18 months, too, the fact that the last two WWDCs have been remote. They're not personal.
01:21:16 ◼ ► But to me, there's a sentiment among third-party developers that the symbiotic relationship between Apple as the first party and developers as the third party, and that the apps developers make for these platforms make the platforms more appealing to users, and the work that Apple does to make the platforms keep moving forward enables developers to better make these, you know, it's symbiotic.
01:21:43 ◼ ► And to me, there's a sense among some third-party developers that Apple's view of the relationship is less symbiotic and a little bit more one-sided, and instead of "Thank you, no thank you," it's "You should thank us."
01:21:59 ◼ ► I know you two don't feel that way, but I'm just curious, like, at the end, what is your message to third-party developers who are concerned that Apple's lost sight of that symbiotic Apple benefits from third-party developers, too?
01:22:17 ◼ ► Quite honestly, I'm baffled by it. I mean, as I hope you know, I and my team and everyone I deal with at Apple believes that there is an absolute symbiotic relationship.
01:22:34 ◼ ► I mean, what my teams work on all year long are platforms for our developers. What we try to create making iOS an appealing platform is a thing that people want to own and then want to use other apps developed by our developer community.
01:22:48 ◼ ► So, honestly, I don't get it. You know, I mean, as you say, it's an interesting time, and there's a lot of funny stuff in the air, and so nothing's changed over here.
01:23:00 ◼ ► I can say that, and we do love our developers, and they're a huge part of why we do what we do every day here, and so it's honestly a little bizarre, some of what I've read just lately, but I hope it turns out to be a minority viewpoint, because it's certainly not founded in reality.
01:23:23 ◼ ► John, at some point, if we haven't introduced you yet, I'd love to introduce you to Susan Prescott. You saw her in the keynote. She's our new VP of Developer Relations.
01:23:32 ◼ ► She is unbelievable in bringing an energy to what we do in Developer Relations, and I'm very excited for that.
01:23:40 ◼ ► And if we were here in person, we'd have that conversation with Marco as well, because I'm sure we'd be having that conversation.
01:23:46 ◼ ► We're pretty excited about what we're doing for developers. I think this is something crazy. Don't quote me on it, but I think it may be our 38th developer conference.
01:23:54 ◼ ► You know, that doesn't happen on accident, right? This is a very important thing to us. Developers are very important to us.
01:24:00 ◼ ► The amount of work that Craig's organization does to create technologies that developers can take advantage of, because we all do win. It is symbiotic, right?
01:24:08 ◼ ► And that is the way we keep looking at it. There's no change in that, and so we're very excited about what we have for developers this week at the conference.
01:24:18 ◼ ► 200 sessions, you know, we've had that platform State of the Union, which is the official name, as well as our keynote, which was aimed for consumers and developers.