0:00:00There is a way to get John Siracusa to leave his house. For those of you who remember, in November, I traveled up, I made a multi-hundred mile train trip to go see Marco and try the Vision Pro prior to release at a lab. And I said to John in advance, "Hey, I know you're not going to the lab, but wouldn't it be neat if you came down to New York and visited with us? I think that would be a lot of fun." And John told me to go outside and play hide and go screw myself.
0:00:28Those were his words exactly.
0:00:30Yeah, those words precisely. And so I thought there was no possible way to get John to leave his house, even for his beloved friends of years and co-workers, of years and years and years.
0:00:41But it turns out, John, there is a way to get you out of the house, and I am happy to report the ATP reunion, for the first time since 2019, is a go. John, what's going on?
0:00:51We are going to WWDC.
0:00:54All of us?
0:00:55At Apple Park, all of us.
0:00:56All of us.
0:00:57All of us.
0:00:58All of us.
0:00:59All of us.
0:01:00All of us.
0:01:01All three of us.
0:01:02Indeed, I am extremely, extremely, extremely excited. I cannot overstate how excited I am, especially to see the two of you fellas, but also to be able to go to Apple Park.
0:01:10The only place I've ever been on Apple Park is the visitor center, the public visitor center that you don't need any special privileges to get to.
0:01:17I'm excited that we are press for the purposes of this event, which I'm really, really excited about.
0:01:22I swear I have never done a press WWDC. I don't think so.
0:01:28And I've been through my badges to see if I can find a press badge. Maybe I have, and it's just been so darn long and I don't recall knowing me.
0:01:33That's probably true. But one way or another, it's happening again, and I am so excited.
0:01:38So I am glad that you are willing to make the trek instead of just a couple of convenient hours down the interstate and/or train from Boston to New York.
0:01:48You're just going to cross the country instead, and I appreciate the effort.
0:01:51Yeah, I mean, I have done press WWDCs. We've all attended many WWDCs, but I've never been to one at Apple Park, and neither has Casey.
0:01:58So that's what I'm most excited for, is to be officially allowed onto Apple's campus.
0:02:04And I think they don't let you take real cameras, which is kind of a bummer, so I'll just take lots of pictures with my phone. But whatever.
0:02:10I'm excited to do it. It'll be a fun experience. Yeah.
0:02:13Looking forward to not looking forward to taking the plane flight, but you know, you do what you have to.
0:02:19Alright, let's do a little follow-up. I wanted to briefly call attention to, there's some new Vision Pro content.
0:02:26What? Really? Yes, there is. There's new Vision Pro content. Who'd have thunk it?
0:02:31So there's three things I wanted to call everyone's attention to, two of which are from Apple and one of which is not.
0:02:37First of all, there is a new sizzle reel. I don't know if that's the bestest way of describing it, but basically it's like a three and a half minute video that gets you interested in the Vision Pro.
0:02:48And previously, I hadn't watched it in a couple of months at least, but previously my recollection is, or the way I remember it was, it showed like a little bit of the kids playing soccer, rhinoceroses.
0:02:59It showed the tightrope walker, and I think it showed a very brief bit of sports, if memory serves. I don't entirely remember.
0:03:08But one way or another, there's a new one, and I think it's really well done. It's still, it's a little jumpy for my taste, just a touch.
0:03:15It's not like that MLS thing from a month or so back where it was way too jumpy. It's just a touch jumpy.
0:03:21But the soundtrack is excellent, and I really, really like it. Again, three and a half minutes. You can find it in the Apple TV app.
0:03:28And since it's new, it is kind of front and center. I know I was complaining and moaning about the information architecture last week.
0:03:37But in this case, it's pretty good, so you should check that out.
0:03:39Additionally, there's a new, I believe it's the Adventure series. This is the one that had the tightrope walker on it.
0:03:47There's a new episode of that all about parkour, and it's three, I believe Brits, based on accents, although who knows.
0:03:55But anyway, it's three Brits doing all various and sundry stunts across Paris.
0:04:01And it's really, really well done. It's like 12 to 15 minutes long somewhere in that neck of the woods.
0:04:06And I really enjoyed it. It's not earth-shattering, but it's really good.
0:04:11And I got to tell you, no spoilers for the 12-minute video, but there's a point at which the three gentlemen are trying to jump from one rooftop to another.
0:04:20And they position the camera such that if you want, you can look down and see how tall it is.
0:04:26And I got to tell you, when it's 3D and immersive, it looks scary as hell. So it's pretty, pretty cool.
0:04:34And I think both of these, I mean, altogether, these two things are worth literally 15 minutes of your time. I really do think it's pretty cool.
0:04:41And then finally, What If has finally launched. I had planned to do the whole darn thing earlier today and report in on it.
0:04:49Unfortunately, they seem to release on Pacific time, so it wasn't available until my afternoon.
0:04:54And I've only had the time to do the first 10-ish minutes, but it was very, very cool.
0:05:00The premise here, no spoilers, is you're in the Marvel Cinematic Universe, and what if things weren't the way they were in the movies?
0:05:09What if things went a little awry and different? And it's your responsibility as part of the story to try to fix all this.
0:05:16And so the portion that I've done so far, again, the first 10 minutes or so, is they teach you how to cast some spells.
0:05:23They interact with you, so it begins as fully immersive, you can't see any of your own environment, and then it converts to augmented,
0:05:32such that there are a couple of characters in your space. And I don't know if it was an extremely happy accident or if it was deliberate,
0:05:38but I was doing this in my office, and I had spun my chair around so my desk and monitors were behind me.
0:05:44And my office is, I don't know, 12 feet by 12 feet, something like that, so it's something like 4 meters by 4 meters.
0:05:49Not terribly big. Yeah, maybe that's right. I don't know, it doesn't matter. Not big is the point.
0:05:54And there were a couple of characters, one is designed to be floating in the air, and sure enough, he was floating in the air,
0:05:58and his feet were kind of inside my guest bed, which is behind my desk. But that's fine, he's magic, so whatever.
0:06:04But then the other character was designed to be on the ground, and sure enough, he was on the guest room floor.
0:06:12Like he was standing on the floor, there was a little shadow below him, and it looked just spot on.
0:06:16And then they had you do some, you go back into an immersive environment where all you can see is what they're showing you,
0:06:23and they have you do some spells where you're using hand tracking in order to do stuff.
0:06:28And there were three or four that I learned, and one of them didn't really work that well, but the rest were spot on.
0:06:33And again, I've only done the first few minutes, but it's really slick, and it's free.
0:06:37So definitely, definitely check all three of these out. Again, that's the new sizzle reel.
0:06:41I think they just call it immersive, or like immersive demo or something like that.
0:06:45And then the parkour, the adventure series, the second episode about parkour and what if by Marvel.
0:06:53And I've put a link to the what if thing in the show notes, assuming I can dig one up, I should be able to.
0:06:57I don't think I can link to the other ones in the show notes, but if I find a way, I will do so.
0:07:02Alright, the low storage 13-inch iPad Pros have 12 gigs worth of RAM chips in them,
0:07:10but they don't use 12 gigs of RAM? Jon, what's going on here?
0:07:15This was discovered by the iFixit folks at Teardown.
0:07:19Their lead Teardown technician, Shahram Moqarti, found that readers apparently,
0:07:24meaning like people who saw the YouTube video, I don't know how they spotted this,
0:07:27but they apparently spotted two 6 gigabyte RAM modules on the 256 gig and 512 gig 13-inch iPad Pro.
0:07:33I looked at the Teardown video, I looked at the Teardown on their website, I'm like,
0:07:36"How did they spot that these were 12 gig modules? I can barely make out the part numbers.
0:07:41They're so blurry from compression." But whatever. Anyway, Shahram says,
0:07:45"Our chip ID confirms this with high certainty. Two 6 gigabyte LPDDR5X modules produced by Micron for a total of 12 gigabytes of RAM."
0:07:54Now, to remind you, Apple says this machine has 8 gigs of RAM, because remember,
0:07:58if you want to get 16, you've got to get the higher storage. They say it has 8 gigs,
0:08:02but there's apparently 12 gigs on the chips there. So why does Apple utilize only 8 gigs of RAM?
0:08:08iFixit says Apple has never done this before, as far as we know.
0:08:12Someone asked if it's possible these RAM chips are defective and some of the RAM is disabled for some reason.
0:08:16The folks at iFixit says that, "My understanding is that if this were the case,
0:08:20they would receive a different part number and be labeled as 4 gig,
0:08:23but I don't think that's how LPDDR5 is manufactured anyway.
0:08:26There's very little doubt that there's 12 gigs of RAM."
0:08:29Another possible question, "What if those are the only LPDDR5X modules they can get their hands on,
0:08:34but as soon as they're able to put in 4 gig ones, they will, so just to avoid problems,
0:08:38this initial batch is artificially limited?"
0:08:40The iFixit answer to that was that these modules have been in production since 2020,
0:08:44according to the spec sheets, so it seems unlikely based on how long they've been around.
0:08:48Super weird, because why would they do that?
0:08:53If you follow the Twitter thread about this, you can see people throwing out a bunch of theories
0:08:58and them basically getting shot down by iFixit.
0:09:01Why would they use bigger RAM chips? Could they not get smaller RAM chips?
0:09:05And if they couldn't get smaller RAM chips, why wouldn't they just free up the whole 12 gigs?
0:09:09The other theory is like, "Oh, they're reserving a certain amount of RAM for LLM stuff."
0:09:14Why would they only do it on this model and not all the models?
0:09:17Because I think this is the only one that has more RAM and still all the other ones have the amount
0:09:21that Apple says based on the teardowns, but this one has this particular variant
0:09:26of this particular model has 12 gigs instead of 8. Super weird.
0:09:30But maybe this is the side door for Apple finally putting more RAM in its products.
0:09:34They'll just install it and not enable it for you. But it's there, you just can't use it.
0:09:38How is your M4 iPad Pro treating your eyes?
0:09:43Because apparently it's not all roses and pansies.
0:09:47I mean, it's good for me, but Ben writes in to say,
0:09:51"I'm upgrading to an M4 iPad Pro from the 2018 iPad Pro.
0:09:55I noticed immediately that my eyes seemed unable to properly focus on the display,
0:09:59resulting in eye strain, fatigue, blurry vision, and even headaches.
0:10:02I couldn't use this display for very long before the symptoms reappeared,
0:10:05so I went down a rabbit hole researching."
0:10:07This is kind of like Marco with the Zhiyun Pro thing, but even worse. Ben continues,
0:10:11"It seems like I'm not the only one experiencing this, though I have yet to determine the exact issue.
0:10:15It might be PWM, which stands for Pulse Width Modulation."
0:10:19And by the way, when I follow these links to look at the research that he was doing,
0:10:23everyone just says, "Oh, it might be PWM. I think you have a PWM."
0:10:26Yeah, it's probably PWM. People will come into a forum or a Reddit or whatever and say,
0:10:30"Hey, I just got a new M4 iPad Pro, and the screen hurts my eyes.
0:10:34What do you think the problem is?" And people would say, "Yeah, it's probably PWM."
0:10:37And I was like, "Are you going to explain what PWM?"
0:10:40I mean, I guessed it was Pulse Width Modulation just by knowing the term or whatever,
0:10:44but when you're helping somebody, don't just say, "Yeah, you probably have PWM,"
0:10:47because they don't know what PWM is.
0:10:49Never mind that the term Pulse Width Modulation doesn't make much sense.
0:10:51I bet if you hear this now, you're like, "Well, I know what Pulse Width Modulation is,
0:10:54but what does it have to do with screens and why would it be hurting their eyes?"
0:10:56So we'll get to that in a second.
0:10:57So anyway, Ben says, "It might be PWM, though I've never known this to be a problem,
0:11:01and I have been using LG OLED TV as well as OLED versions of the iPhone Pro
0:11:05for years without any issue, or maybe tandem OLED is misaligned if there is such a thing.
0:11:09I ended up going to the Apple store and compared my device with others.
0:11:12Mine appeared to be slightly different, as if the HDR was turned on all the time.
0:11:16Overall, the display was always just too much,
0:11:19as best described as basically yelling at me all the time.
0:11:21Since I was still within the 14-day return period,
0:11:23they switched my device for a replacement, which seems to be much better now,
0:11:26although not perfect.
0:11:27My question is for Jon, are you noticing any eye fatigue with the new Pro,
0:11:30especially compared to the 2018 version?
0:11:32If so, do you expect this could be improved through software updates?"
0:11:34So here's the research and what PWM is talking about.
0:11:38According to these links that Ben provided that we'll put in the show notes,
0:11:42the way this OLED and some other OLEDs handles brightness,
0:11:48there's two ways OLEDs can handle brightness.
0:11:51One is they can send less voltage to the pixels, and they're not as bright.
0:11:56So for example, if you dim the brightness of the screen,
0:11:59how does it do that?
0:12:00One way you can do it is you can just send less voltage to the screen,
0:12:03and it gets dimmer.
0:12:04But if you only use adjusting the electricity going to the pixels
0:12:09to control brightness, apparently when you get to low brightness levels,
0:12:12you lose a lot of the color saturation, too,
0:12:15and it looks kind of dingy and gross.
0:12:17so they tend to not want to do that for the lower brightnesses. The other way you can control
0:12:23brightness on an OLED is you can have the screen be at maximum brightness briefly and then go off,
0:12:29and then max brightness and then off, and the longer the "on" period is, the brighter it is.
0:12:35So they show a little graph and this is the pulse width modulation of saying,
0:12:39"If you just pulse the screen, pulse, pulse, pulse, pulse, pulse, the faster the pulses
0:12:43go the brighter the screen. If you go pulse, pulse, pulse, it is dimmer because the light is on
0:12:49less of the time." You don't notice this because Apple screen, this TANO OLED, pulses at 480 times
0:12:55a second, which is a pretty high refresh rate if you remember from the CRT days. Like, "Oh,
0:13:00my screen looks flickering at 60 hertz. I can see the flicker, but at 85 I can't see it anymore.
0:13:04At 120 I definitely can't see it. At 480 I can tell you I cannot see. Well, it's flickering."
0:13:11But the complaint about the M4 iPad Pro TANO OLED is that it uses pulsing to control its brightness
0:13:20through its entire brightness range apparently. Even at maximum brightness it's still pulsing,
0:13:25as opposed to some other OLEDs which will use pulsing down at low brightnesses,
0:13:29but once they get to high brightnesses they will do that by keeping it on all the time but
0:13:32just sending less voltage. I don't know if this is just the way TANO OLEDs work. Is this the way
0:13:37Apple is choosing to make it work? But some people report that this bothers them. I don't
0:13:42personally see how it could because I am not aware of anyone who would notice flickering at 480 hertz.
0:13:49We're not even talking about motion here. We're just saying like put on just a full field,
0:13:52you know, red slide. I can't see it flickering at 480 hertz. If you had asked me whether this is
0:13:59flickering to control brightness, I would have said no because I can't see it. But maybe people
0:14:04with very young or better eyes than mine can see it. I don't know. Or maybe it's... Anyway,
0:14:09follow the links, decide for yourself. And by the way, I would say do iPhones use this or do they
0:14:15use voltage regulation? I honestly couldn't tell you because it just looks like a screen to me and
0:14:20I can't see whether it's dimming or pulsing at 480 hertz. Some of the OLEDs that are out in the
0:14:26market pulse at even higher rates than that. So yeah. Is this an issue? Is this something
0:14:31people are imagining? Are some people just uniquely sensitive to it? All I can tell you is that
0:14:35my old man eyes don't see this and aren't bothered by it. But we'll see. We'll see if Apple does an
0:14:41update. We didn't put this in the notes for the last episode, but there is some kind of
0:14:44actual software error with displaying HDR video where some like highlights are getting blown out
0:14:49that Apple... Uncharacteristically, Apple immediately acknowledged inside a software
0:14:53fix was coming for. So it is possible that maybe whatever issue people are complaining about here
0:14:58will also be wrapped up in that fix, but we'll see. Michael Thompson writes in with regard to
0:15:03trillions of operations per second measurements or TOPS measurements. Michael writes, "I found
0:15:08this article on the Qualcomm website that suggests that the TOPS measurement they use for their NPU
0:15:12performance is based on 8-bit integers. In the paragraph headed 'precision,' they state,
0:15:16'The current industry standard for measuring AI inference and TOPS is at 8 precision.' The context
0:15:25here being whether or not the new surface line and the... What is it? The copilot plus PC or whatever
0:15:31it is? Line of PCs. Are they or are they not actually faster for neural related things than
0:15:38Apple stuff? And so I don't recall what was... Does this mean they are faster than I presume?
0:15:45Well, I mean, so this matches what we saw. The idea that on this website they're saying Qualcomm
0:15:49is saying the industry standard is 8-bit precision, right? And what we've seen is Apple is using
0:15:55int8 and so is Qualcomm and so is Microsoft and everybody who's current talking about their
0:15:59current line of products, they're all using int8 precision. And again, this is saying,
0:16:03how many 8-bit things can you process at once, right? If I ask you how many 16-bit things you
0:16:09can process, it's half as many, right? And so the number is smaller. Previously, unlike earlier,
0:16:15you know, Apple Silicon stuff, Apple was using 16-bit stuff. And so their numbers were half as
0:16:19big, but with their new stuff, Apple is using numbers that are twice as big and they're using
0:16:24int8 and so is Qualcomm. So the answer is everyone's using int8 now. Is that because
0:16:29int8 is more representative of the actual jobs we're asking our NPUs to do? Maybe, but you know,
0:16:34whatever. The industry has decided when measuring tops, we're going to use int8 precision. That may
0:16:40become less relevant if it turns out that the things we ask our NPUs to do involve 16-bit or
0:16:4732-bit values, and it doesn't really matter how fast they can do stuff on int8 things,
0:16:50but I would trust that int8 is actually a relevant measure right now. So the answer is,
0:16:54you know, the Copilot Plus PCs and the Snapdragon X elite thing has 40 tops. The M4 has 38. Those
0:17:03are both int8 measures. That means they're essentially comparable. Eric Jacobson writes
0:17:08in with regards to iCloud Drive and node modules. So if you recall, this was with John Sun,
0:17:16who basically nuked his MacBook Air by trying to sync the node modules folder through iCloud
0:17:22Drive. Eric writes, "I haven't used it since I don't use iCloud Drive, but there's a project
0:17:26that will add a no sync directive to every node modules on a file system. I imagine it might need
0:17:32to be rerun whenever a new project is kicked off." And we'll put a link in the show notes to
0:17:35nosync-icloud. Yeah, and I tried to look at the code to remind myself how I think you make like
0:17:40a directory with a dot nosync extension that has the same name as the other one. Like that's the
0:17:44way you signal to iCloud Drive not to sync the directory or something like that. This is a node
0:17:49module itself, so you can look at the code, but unfortunately the documentation is all in,
0:17:52what are we going to say here? I'm going to say Chinese? Yeah, something like that. Yeah,
0:17:56so the documentation is in Chinese and I can't read it. But the source code is not in Chinese
0:18:00and I still couldn't quite make any sales of it. But yeah, I think it's just making dot nosync
0:18:05directories in the right places. And it's a node module that you can use and it will, you know,
0:18:08I think you just include it in your project and make sure everything nosyncs. So that's,
0:18:12you know, useful and helpful if you want to dare to walk that tightrope of trying to use node modules
0:18:17with iCloud Drive. Indeed, and then Eric continues, I do however use time machine and can attest that
0:18:22the Asimov utility works perfectly for excluding node modules and other dependency directories.
0:18:27Also it is a background service, so it doesn't need to be reinitialized. And we will again put
0:18:31a link to the show notes to Asimov. And also, John, I guess you wanted to call attention to the
0:18:36list of excluded directories, which I put in the show notes as well. Yeah, it shows what kind of
0:18:40things like when it says dependencies, what does that mean? Obviously it means node modules, but
0:18:43it has a whole list of all the different things that excludes things from Gradle, Bower, PyPy,
0:18:50NPM, Parcel, Cargo, Maven. I think CocoaPods might be, yeah, CocoaPods is in there. Like Marco,
0:18:55have you heard of any of those things other than CocoaPods? Zero, precisely zero. Flutter. Anyway,
0:19:01it's sad that I think I've heard of all of these. But I installed this and I ran it. It installs a
0:19:08little launch daemon thing or whatever. And it essentially does the... I forget what the time
0:19:15machine is. I think it's an extended attribute or something, but there's a way to exclude things
0:19:19from time machine. Maybe it just calls tmutil. Yeah. Anyway, excluding all these directories
0:19:24from your time machine backups can make your time machine backups go faster. What it's basically
0:19:28saying is you don't need to back up the dependencies of your code. If you're writing something in node
0:19:34and you use 17 node modules, you don't need to back those up. You get them from the internet
0:19:37anyway. You got them through NPM or YARMM or whatever. They're on the internet. Do not back
0:19:41them up. That's not your code. It's a dependency. You didn't write that code. It's just pulling it
0:19:45in. And there are tons of files. So if you can exclude those directories from time machine,
0:19:49it will make your time machine backups go faster. But who remembers, oh, what am I going to do? Go
0:19:53to options and time machine and drag the little thing in or set the extended attribute. I don't
0:19:57remember how to do this. This just runs in the background all the time. Looks for directories
0:20:01that fit this signature and excludes them from time machine. So I did that. I probably saved,
0:20:06I don't know, thousands, many thousands of files are no longer in my time machine backups
0:20:11because I ran this. I hope it doesn't have bugs and isn't excluding a whole bunch of
0:20:15important files from my time machine backups. But I've got multiple backup strategies. So for now,
0:20:20I'm trying the experiment of running this asmoff daemon in the background to see if it helps with
0:20:24my time machine backups. And I'm still not running iCloud drive, of course. Of course.
0:20:28- And then finally, this is actually, I should have moved this up by the other Vision Pro follow-up.
0:20:33Didn't think about it. But anyway, Jonathan Gobranson writes with regard to audio routing
0:20:37during Vision Pro guest mode. So if you recall, I was doing demonstrations for my mom and dad.
0:20:42And I noticed that when mom was on Mount Hood or whatever it's called, and I had her go fully
0:20:47immersive, that the crickets and whatnot were being routed through my iPad Pro, which was doing
0:20:54mirroring at the time. And so it doesn't really make for a very good effect if the audio was going
0:20:58through there. So Jonathan writes, "You can choose during setup of each guest user session whether to
0:21:04route audio to the Vision Pro or the iPad or whatever the case may be if you choose to mirror
0:21:09content." And we'll put a link to the Knowledge Base article. So what you do is you look up and
0:21:15you get the little green down chevron near the top of your view. Then you go into Control Center.
0:21:20Then you go back into the Mirror My View button. And then in there, there's a audio routing section
0:21:27that you can choose to push everything back onto the Vision Pro. Not entirely sure why this isn't
0:21:32the default, to be honest, because pretty much every time I've always wanted this. But here we
0:21:36are. At least now I know that there's a way around it. So good deal. We are sponsored this episode by
0:21:43Fastmail, not only a great email host, but my chosen email host since 2007. Obviously, long
0:21:49before they were sponsored, long before I was a podcaster, I've been a Fastmail customer and I've
0:21:53been very happy because hosting email, it's constantly a moving target. You don't want to
0:21:58do it yourself. You don't want to also use one of the big services necessarily because then your
0:22:03name's tied to them. I like having my own control. I like being able to have standardized hosting
0:22:09with a domain that I own. Fastmail is a great place to do this. They even now have domain sales.
0:22:15You can add or buy domains right through them and they set up all the records for you and everything
0:22:18so it works immediately and painlessly. But it's just a great email host. Fastmail has all the
0:22:24features you want under an email host. Of course, obviously it's IMAP hosting and they have all the
0:22:29basic default stuff that works with every email client. They also have all the modern productivity
0:22:33features you might want. Things like scheduled send, snooze, folders, labels, search, etc.
0:22:38They have all the great plans, all the great features. I love Fastmail. They now also have
0:22:42multi-user plans. So if you want to have your whole household have a plan, they have a duo plan
0:22:47for secure private email for you and a partner at a reduced price. And for just a little bit more,
0:22:51they have a family plan for up to six people. You can upgrade your entire family. And of course,
0:22:55in those family and duo plans, you have things like shared domains if you want. You can have shared
0:23:00or private calendars and address books. You can have like a family calendar all shared,
0:23:04hosted by Fastmail, all based on standards. It is wonderful. So I strongly recommend Fastmail
0:23:09and they are super private too. For over 20 years, Fastmail has been a leader in email privacy.
0:23:15They believe in working for you as people to be cared for, not data and products to be exploited.
0:23:21And Fastmail has better spam filters than everyone else and absolutely no ads. So it's super easy to
0:23:27get started with Fastmail today. You can download your data from wherever you are now. You can
0:23:30import it into your new Fastmail inbox. You don't need to leave anything behind when you switch.
0:23:35It's great. Try Fastmail free for 30 days and get 10% off your first year at fastmail.com/atp.
0:23:42Once again, free for 30 days, 10% off your first year, fastmail.com/atp. Thank you so much to
0:23:48Fastmail for being an awesome email host for all these years and for sponsoring our show.
0:23:52It is quickly approaching WWDC time, which means I'm going to be seeing you too soon. But
0:24:00nevertheless, we should talk about some last second predictions. And I guess this is most
0:24:07predominantly from Mark Gurman in today's episode. So I'm going to read a whole bunch of stuff. One
0:24:11of you guys feel free to pipe in and interrupt at your convenience. But here we go. Mark Gurman
0:24:17writes, "Apple is preparing to spend a good portion of its worldwide developers conference
0:24:22laying out its AI-related features. At the heart of the new strategy is Project Gray Matter,
0:24:28a set of AI tools that the company will integrate into core apps like Safari, Photos, and Notes.
0:24:33The push also includes operating system features such as enhanced notifications.
0:24:37Much of the processing for less computing-intensive AI features will run entirely on the
0:24:42device. But if a feature requires more horsepower, the work will be pushed to the cloud. There are
0:24:46several new capabilities in the works for this year, including ones that transcribe voice memos,
0:24:50retouch photos with AI, and make searches faster and more reliable in the spotlight feature.
0:24:55Faster would be great, particularly on my iPad, please, and thank you. They also will improve
0:24:58Safari web search and automatically suggest replies to emails and text messages. The Siri
0:25:03personal assistant will get an upgrade as well, with more natural-sounding interactions based on
0:25:07Apple's own large language models. There's also a more advanced Siri coming to Apple Watch for
0:25:11on-the-go tasks. Developer tools, including Xcode, are getting AI enhancements too."
0:25:16Let's stop here for a second and look at this list of features because we're always like,
0:25:19"How will Apple add AI sauce to all this stuff? What things will they talk about?"
0:25:22There was a big story, Gurman had it, and I think we might have mentioned on the show,
0:25:27like, "Oh, they're going to fix Siri." We were speculating months ago or weeks ago, whatever.
0:25:32"Is this the year that they're going to fix Siri with AI, or are they just going to add it to a
0:25:34bunch of other stuff?" And Gurman's rumor was like, "No, they're doing a Siri thing." So we
0:25:37can expect to see that. But here's some specifics, and the specifics seem not weird. I don't know.
0:25:45Sometimes these rumors aren't comprehensive. Very often, Apple emphasizes one or two particular
0:25:50things, whereas we just get a laundry list and we don't know which one they're really
0:25:52going to concentrate on, and it's going to be impressive. But let's look at some of these in
0:25:55turn and see how exciting they are. Transcribing voice memos. Apple's been doing transcriptions,
0:26:01for example, on voicemail for a long time now. Having transcription be better, that's good.
0:26:08Probably not going to really radically change people's lives since it's an existing feature,
0:26:12they're just making it a little bit better. Retouch photos with AI could mean a lot of
0:26:15things. That could be a headlining feature. That short description doesn't tell us, "Is this going
0:26:20to just be like, 'Oh, they're better at background detection for tearing people off,' or is it going
0:26:24to be like, 'Oh, they're going to really emphasize that they're doing the feature that Google and
0:26:27many others have, where if there's something in the background, a photo you don't want,
0:26:31you can remove it, remove a person from the background who shouldn't be there, remove a
0:26:36sign or a tree or something like that.' That could be a big headlining feature. Google has certainly
0:26:41had a whole ad campaign about theirs, and obviously it's every graphics application from
0:26:46Photoshop to Lightroom, Pixelmator and everything has features like this, and they've been touting
0:26:50them, and I think they're crowd-pleasing. - Well, but that's part of the problem. So far,
0:26:53voicemail transcription and fancy AI photo cleanup and removal, that's table stakes today.
0:26:59I'm glad Apple's getting there, they should, but neither of those I think is likely to make a big
0:27:07splash simply because other people have been doing that for a decent amount of time, and the rest of
0:27:14the industry is there. What I want to see from Apple is what we get excited about from Apple.
0:27:21I want them to blow us away with stuff that we haven't seen from everyone else. I want to see
0:27:25features that are not just playing catch-up with the rest of the industry. I want to see Apple
0:27:29leading the way, not following, and these are just following so far. - Yeah, I mean, well, as with
0:27:34all these things, we know that this has been done elsewhere and even on Apple's own platforms in
0:27:39various applications, but when people get one of these new phones or they update the OS or they
0:27:45see an ad on TV that Apple touts this feature of erasing a person, chances are good that they've
0:27:50never seen that before if they're not a tech nerd, right? And so as far as they're concerned, wow,
0:27:54this is amazing. Especially if they don't know it's even possible, like they're saying, "Oh,
0:27:58on my phone I could just tap a person and remove them?" And we all know that that's been around for
0:28:02a long time and they're playing catch-up, but A, they do have to play catch-up. We don't want them
0:28:06to not do this. And B, it can be just as impressive. It can be very impressive to people who haven't
0:28:10seen it before. So I just hope they do a good job of this because there is a lot of competition,
0:28:14again, even on Apple's own platforms in applications that you can get for the Mac and for
0:28:20the iPad and for the iPhone that already do the same job. I hope Apple at least matches them.
0:28:25Historically, Apple has kind of been... They want to make it simple, so there's not a lot of
0:28:30adjustability. And if it does a bad job, the Apple way is like, "Well, just tap on the person and
0:28:35remove them." And you'll tap on a person and it'll make a terrible mess of it. And you'll be like,
0:28:39"Is there something I can do to try that again and do better?" And the Apple way is no.
0:28:42Nope.
0:28:43You tap them and if it does a good job, good. And if it doesn't, oh, well. Whereas in Lightroom,
0:28:48there's a million knobs you can adjust and you can take a second attempt and you can mask
0:28:51differently. You do all sorts of things at a fancier application. So there's that. I really
0:28:57hope they do catch up and I hope they do a good job of it. The next item about improving Safari
0:29:03web search. How? I get. All right.
0:29:09Well, I mean, for whatever it's worth... Obviously, we don't know what this means yet. We might not
0:29:14even learn it in two weeks. But Safari had... Whatever Apple is doing, whatever their back-end
0:29:21logic is for autocomplete suggestions in Safari is basically a search engine. I can't even imagine
0:29:29how many millions and millions of web searches Apple avoids even making on the iPhone by people
0:29:36just tapping that first autocomplete result that comes up. So maybe it's just related to whatever
0:29:42the back-end is of that. And that could be one of those invisible things that everybody will take
0:29:48for granted and nobody will even notice, but we will appreciate it getting better. Like things
0:29:52like autocorrect. Well, notice if they screw it up, they've changed autocomplete many, many times
0:29:56over the years and when they screw it up, we all notice and we don't like it. So I really hope
0:30:00it doesn't start suggesting ridiculous things. I think that might all be local too. So maybe that
0:30:05is a place for LLMs to try to do better predictions. But anytime they're messing with an existing
0:30:10feature, you hope it's gonna be a big improvement, but there's the possibility, especially when you're
0:30:14throwing LLMs into the mix, it's a possibility. If the current strategy is like, look at your past
0:30:19history, look at the number one search results, they could even be hitting the Google backend
0:30:23for some of those things. It's a very straightforward non-complicated algorithm,
0:30:26but it's deterministic. And if they switch from that to like, let's just chuck it over the wall
0:30:31to the LM and see what it says. I'm concerned that there might be some wackiness there,
0:30:36but we'll see. More reliable spotlight features. Similarly, when you use spotlight on the phone,
0:30:42I don't know if it's using spotlight, spotlight, if it's the same technology as the runs on the
0:30:46Mac, but it's like, oh, I'm searching for stuff on my phone and it includes contacts and all your
0:30:51applications and files and things you've done recently. And yeah, you could probably throw
0:30:56LLMs into the mix there to handle, basically like, so you don't have to type things exactly,
0:31:00and it's better about synonyms and you can type in vague things, like type in some expression about
0:31:05what you want to do in settings and have the LM figure out the setting you want to go to.
0:31:09Potential for good, also potential for messing up spotlight on the phone.
0:31:13And then suggest replies to emails and text messages. This starts to get into the area
0:31:18where we thought like, will Apple go there? We haven't gotten to like chat bots yet, but
0:31:22obviously everyone else is doing this, so it's another catch up feature.
0:31:26But the idea that Apple would within the mail application or while sending text messages pop
0:31:31up a little clippy and say, it looks like you're trying to reply to your friend. You want me to
0:31:34write it for you? Which is what, I mean, just look at the Gmail interface, for example, every time
0:31:38I'm in Gmail, it's like, do you want me to summarize this email? Do you want me to just
0:31:42write the reply for you? Like Google's been pushing on this for years. It used to just be like,
0:31:46here's a canned phrase that is probably a reasonable reply to this email. And now it's
0:31:49like, you know what? I can just write the whole email for you. Like you don't have to click from
0:31:53canned phrases. You don't have to, you know, the auto-complete that happens in like Google docs
0:31:57and everything. It's just like, let me just write the whole thing for you. And that, we should find
0:32:02that link to Nevin Mergen's blog post. Nevin Mergen said he got an email from a friend that was
0:32:06written by an AI and he kind of flipped out about it, right? That seems like a less,
0:32:12that is not a conservative move to suggest to people that the phone will write their email or
0:32:19their text message for them. Summarizing, I can see it's like the phone is helping me. If I just
0:32:23want a summary, I don't want to read it all, let the phone summarize it. It's like, you know,
0:32:27asking the phone to help you deal with your stuff. But having generative AI write something that you
0:32:33will send as if you wrote it yourself is a step farther than Apple has gone. So we'll see how they
0:32:41present that. Again, obviously everyone else is doing this. It's not new. I'm not saying Apple's
0:32:44going to be the first one to do this and they're going to get in big trouble or something. It just
0:32:47doesn't, it's not a very safe thing to do. So I'm wondering how daring Apple will be in like
0:32:53their little presentation. Maybe they'll just be like, and it can even suggest a reply for you. And
0:32:57some happy person who, happy text will tap a thing on their phone screen. And they'll be so delighted
0:33:03to see the three word reply come and they'll hit the send button and they won't talk about it
0:33:07anymore. But I don't know, it wigs me out a little bit too. - I mean, keep in mind, like we're probably
0:33:13not that far away from, I mean, people are probably doing it now. Where like you're using your AI to
0:33:20compose an email that will be then read by the recipients' AI. And we will all just be sending
0:33:26even more garbage crap to each other and having it be processed by even more garbage AI junk on
0:33:32the other end. And we will finally reveal email to be the useless thing that it has been most of
0:33:36this time. - It's not useless. And I feel like in a business setting, this is most useful because
0:33:40there's a lot of boilerplate and niceties in business that would help. But like we already
0:33:44have auto, like I mentioned autocomplete, like Gmail forever has had a thing where you start
0:33:47writing something and it guesses what you're probably going to say for these next three words
0:33:51and you can autocomplete that. But I feel like that kind of piecemeal autocomplete, even if it
0:33:56is LLM powered, that piecemeal autocomplete at least forces you to read everything. Whereas having
0:34:01a button that says, "Compose a reply for me," invites the possibility that you will not read
0:34:06what it generated because that will take too much of your time and you'll just hit send.
0:34:10And now we're sending messages that we're not even looking at to give a thumbs up or a thumbs down on
0:34:14because it just takes too much time. And that's just, I think it's a waste of people's time
0:34:17because if you don't read what it wrote, maybe it didn't write what you wanted it to write. And
0:34:20now you have an even more confusing email thread. And I mean, again, we'll figure this out as a
0:34:25culture, as the technology advances, but I'm just saying character for Apple, for Apple's thus far
0:34:30very conservative approach to AI, suggesting replies to emails and text messages seems like
0:34:36a big move for this particular company. Yeah, agreed. Continuing from Mark Gurman,
0:34:43"One standout feature will bring generative AI to emoji. The company is developing software that
0:34:49can create custom emoji on the fly based on what users are texting. That means you'll suddenly have
0:34:53an all new emoji for any occasion beyond the catalog of options that Apple currently offers
0:34:58on the iPhone and other devices. Another fun improvement unrelated to AI will be the revamped
0:35:03iPhone home screen that will let users change the color of app icons and put them wherever they want.
0:35:07For instance, you can make all your social icons blue or finance related ones green,
0:35:10and they won't need to be placed in the standard grid that has existed since day one in 2007."
0:35:15All right. So that's a lot of stuff here. The generative AI emojis, obviously they're not
0:35:19emojis because emojis are like Unicode things. There's a defined set of them. You can't just
0:35:22make up new ones. I mean, you can't if you're the Unicode consortium, but Apple can't, right? So
0:35:26what it's actually saying is kind of like on Slack and Discord and all these other things where you
0:35:31can sort of generate custom stickers or I think they call them custom emojis or custom reactions
0:35:35or whatever. The point is, this is going to be a new graphic that they'll generate on the fly for
0:35:39you that will be sent as an image because they can't just send it as a Unicode code point because
0:35:44these, they're not dynamically adding things to Unicode, right? That's not how this works.
0:35:47So they're kind of generating stickers for you. And I guess they're going to like, you know,
0:35:53they have a bunch of like Mr. Potato Head style building parts they feed into the LLM and then it
0:35:57can do like sentiment analysis to figure out what kind of emoji we'd want. I have to say, like,
0:36:01there's a lot of emoji, especially for things like face reactions and stuff like that.
0:36:05I guess there's ones that aren't there. Like you never try to summon someone emoji and you're
0:36:09shocked that there's not like a watermelon emoji or something. I can't think of one that doesn't
0:36:13exist. Sorry. But frequently I go into emoji and I'm like, "Oh, surely there's an emoji for this."
0:36:18And like, there's no ice cream cone. Like, again, I don't know if that's real or not, but like I'm
0:36:21shocked by what's not there. So that's nice that it'll generate to it, but it'll have to send it as
0:36:25an image because it can't send it in the other way. And I do wonder, this is another place where
0:36:30like, I feel like Apple is taking a risk, even with a very, very limited image generator that's
0:36:35trained on their own source, you know, trained on all their own source for emojis that Apple makes
0:36:39and can generate new ones based on that and some other information. There's the potential to
0:36:45generate a little image that maybe is embarrassing to Apple as a company. You know what I'm saying?
0:36:49So, and there's a potential to generate images that just look ugly or not up to Apple standards.
0:36:56You know what I mean? Like, this is, that's a weird thing to be touting. And the coloring icons,
0:37:01that's, you know, another, I'm not going to say, I keep wanting to say a bridge too far. It's not
0:37:05a bridge too far, but it is a bridge. Like you're going to take developers icons and draw all over
0:37:11them. They've done this before. They used to add the shine to your icon if you didn't opt out. Do
0:37:14you remember that? Like the gloss? But now they're like, we're going to change the color of your icon?
0:37:20Like, I'm sure these companies that are like so proud of their branding and change their icons or
0:37:24whatever, like that now the user will be able to say, you know what? I want Instagram to be green.
0:37:29And the OS will be like, sure, I'll take that Instagram icon and I'll make it green. We can
0:37:33do that. That's weird. I mean, you can make little color code things on your home screen, I suppose.
0:37:38But yeah, I can't imagine that's the whole story to that one. I mean, first of all,
0:37:42like a lot of companies I think that have a problem with just like trademark issues with like,
0:37:46what if you color my icon green? First of all, it's not my brand identity. Second of all,
0:37:51it might look like my competitors or it might infringe someone else's trademark or something.
0:37:55Like I could see there's, I could see like big companies having a big problem with that. But
0:37:58users are doing it. And by the way, there is a precedent for this in Mac OS back in the day,
0:38:03someone else will look up what version of Mac OS or the operating system for Macs used to do this.
0:38:08When you added a label to something, you know, you can do labels in Mac OS 10 or in classic Mac OS,
0:38:12you could add labels in one version of Mac OS or system six or whatever it was called back in it,
0:38:18whichever version I'm thinking of. I think it might've even been system seven. When you apply
0:38:22to label it, what color the icon like, so if you probably green label, it would like green tint the
0:38:28icon by basically making the black pixels green or the black pixels red. I think it did not look
0:38:32good. It was not a good look that didn't last for a very long time. Certainly the Mac OS 10 era that
0:38:38has never been the case, but yeah, the user deciding to color icon, like that was not on my
0:38:45list of things to think of they were to be doing. And I know this is like unrelated to AI, but let's
0:38:48just say iOS 18 change next week, we'll probably have many more previews of WWDC features, but
0:38:53what an odd, what an odd thing to do. What is this? Have people been clamoring for this?
0:38:59I want to change the color of my icons. I mean, changing icons. They added that while ago where
0:39:04you can pick from different icons and blah, blah, blah, but even that, remember how conservatively
0:39:07they added that? Like it has to pop up a dialogue in people's faces so you know when it's happening
0:39:11so apps can't masquerade as other apps. - I literally just implemented that in the
0:39:15rewrite yesterday. (laughs) - Yeah.
0:39:17- No, but I mean, there's clearly, there's a very large market out there. We've talked about it
0:39:23before. Like there's a very large market out there for these home screen customization apps that use
0:39:27all sorts of tricks and hacks and frankly kind of bad hacks to get people to be able to customize
0:39:33icons for their apps and make like their aesthetic home screens that they want. That's a huge demand,
0:39:38huge market. I think Apple wants to try to address some of that demand without having all these huge
0:39:43hacks if this is indeed what they're doing here. The problem is unless you can have arbitrary icons
0:39:49for apps, like specify your own image as the user. - As you can with shortcuts.
0:39:54- Right, like if you can do that, then all these apps that sell like these, 'cause you know,
0:39:59they sell like theme packs. Like people aren't just drawing it themselves. Like they're using
0:40:03apps that have like cool theme packs of here's like a bunch of popular icons for all the popular
0:40:09apps that you know you have like Instagram. Plus also here's a bunch of like random ones you can
0:40:13apply yourself to whatever else you need them to be, you know, like kind of generic ones.
0:40:17That's a huge market and so if we're just talking about like you can tint it seven different colors,
0:40:23that's not gonna get rid of this market at all and you know, some people will use it. That'll
0:40:28be great but I can't imagine that being worth the hassle and the trouble that they might get
0:40:32in with other companies. So I have a feeling this is like, you know, sometimes a rumor comes out
0:40:37and we're just kind of like huh? And then you know, whenever it's actually announced, we see,
0:40:42oh there's important detail X, Y, and Z here that explain it better and that now it makes more sense.
0:40:48I think we're missing those details right now on this feature. I think there's something else here
0:40:52to explain this that we're not getting it. - I mean, it might just be like you mentioned
0:40:56those icon packs or whatever. Part of the limitations of customizing your home screen
0:40:59using shortcuts to replace the app icons, first of all, it's super cumbersome as is doing anything
0:41:03in home screen because you have to hide the existing app and make the shortcut to the app.
0:41:06You know, it's just, it's a hassle and second, if you get one of those icon packs, maybe it doesn't
0:41:09have an icon for the app that you want to change but if the OS says, okay, well if you're making
0:41:14a little region of green icons and you can use icons from the Steam pack, you got, oh but there's
0:41:18no green icon for this app, well just take the actual icon for the app and tint it green and
0:41:22now it fits in. So maybe it's like, you know, a catchall for if there's, if, because people
0:41:27aren't going to draw their own custom icons and if they can't find the one in a pre-built icon pack,
0:41:32now you can just at least make the icons match. I don't quite understand the idea to color code
0:41:38everything but there's that and yet the other part of this is being able to, you know, this does not
0:41:42put them in the standard grid. It's basically just to be able to leave space. You don't have to make
0:41:45spacer, clear spacer icons anymore. I'm assuming that's what they're talking about here is like,
0:41:49now you can arrange things on the home screen and leave gaps if you want to. I still think doing
0:41:56anything on the home screen on the iPhone is incredibly painful. We've talked about this
0:41:59for years and years. I think it will continue to be painful. I really wish there was a nice interface
0:42:05where you could mess around with this stuff and do it without destroying anything and either commit
0:42:10or roll back all your changes as opposed to the current system which is just pure chaos and it's
0:42:14like the hardest game on the entire operating system to try to rearrange icons on your phone
0:42:19without screwing everything up and just throwing your hands up and saying forget it, I'm hiding
0:42:22everything into the app library. Yeah, I wonder, I feel like I heard this from Jason maybe. I heard
0:42:30it somewhere. Maybe it was Mike on upgrade. I forget where I heard it but somebody pointed out,
0:42:33well maybe there will be like layers and that makes me think of, you know, how that was upgraded
0:42:39in SF symbols. You can have like different layers and you can color them and you know in different
0:42:43ways and whatnot. I think that makes some amount of sense but that's not going to fix like the
0:42:49instagrams of the world so I'm not sure or maybe it'll be opt-in for developers but again I don't
0:42:54see. VisionOS has this right? Don't they have layers in their icons? Yes, they actually kind of,
0:42:58I think they require it with VisionOS. They made that a new like hey for iOS 18 if you provide
0:43:03a layered icon, our new coloration API will respect that like as you sound like SF symbols
0:43:10where we can colorize parts of it. We'll see. It just seems to me like a weird place to be
0:43:15spending resources. I think the arrangement part is a smart feature. They should have that. People
0:43:20have wanted that for a long time. People have been doing it with spacer icons. It's annoying.
0:43:23The place where they really should have spent resources is how hard it is to rearrange the
0:43:26home screen. We'll see if they do that. Yeah, that's the only way I really wish I could still
0:43:31interact with my phone via iTunes which I know is now finder but that was so much better was,
0:43:36you know, doing home screen rearrangement. And even that was terrible but it was better.
0:43:39Yeah, it was terrible but it was way better. All right, finishing up with Mark Gurman. A big part
0:43:44of the effort is creating smart recaps. The technology will be able to provide users with
0:43:48summaries of their missed notifications and individual text messages as well as web pages,
0:43:53news articles, documents, notes and other forms of media. There's also no Apple design chatbot,
0:43:57at least not yet. That means the company won't be competing in the highest profile area of AI
0:44:02and the version that Apple has been developing itself is simply not up to snuff. The company's
0:44:07held talks with both Google and OpenAI about integrating their chatbots into iOS 18. Apple
0:44:12ultimately sealed the deal with OpenAI and their partnership will be a component of the WWDC
0:44:17announcement. A couple of quick thoughts here. First of all, I think we all are in a group chat
0:44:21that oftentimes will just pop off at a time that you're not paying attention and so it is not
0:44:26uncommon. Particularly, I find that I'm in a group chat with a couple of guys that are constantly
0:44:32looking at cars that they may or may not ever buy. One of them in particular is obsessed with finding
0:44:38a affordable 911 and those are mutually exclusive terms. You cannot find a 911 that is affordable,
0:44:43that's not a pile of garbage. And so anyways, I will often come back to my phone or computer,
0:44:48what have you, with literally 40 or 50 messages, most of which I don't particularly care about
0:44:52because 911 is not really my cup of tea. And so if there was a way to summarize that, I'm all in,
0:44:57sign me up. And then additionally, I mean here it is that Germin is saying OpenAI is the winner and
0:45:03like we discussed I think this past week, that's going to be a little dicey. So I wonder how they
0:45:08spin this. Well, in typical Germin fashion, there's also no Apple designed chatbot, at least not yet.
0:45:14Apple designed? Do they mean Apple's in-house LLM that they've been working on? Is that what
0:45:18he's saying there won't be any of? Because I bet whatever, like they say they did a deal with OpenAI
0:45:23but he doesn't say, okay, well what is that deal, how is that deal going to manifest? Is it going
0:45:26to manifest by iOS 18 having a chatbot that is powered by OpenAI? And will that not be Apple
0:45:30designed or will it be OpenAI designed? Will it be OpenAI branded? These are all, I guess,
0:45:35questions that he doesn't have the answers to yet. But yeah, the news is they did a deal with OpenAI.
0:45:40There's also news this week that Apple is still talking to Google, obviously. So we'll see if the
0:45:49words OpenAI appear at WWC, if they're going to announce this as a partnership, they're just going
0:45:53to use them on the back end. Like I said last week, maybe they've been building features using
0:45:57the existing OpenAI API and now they just ink this deal so now they can officially do it in a released
0:46:02OS instead of just doing it in dev builds. But I'm just not quite sure how this is going to manifest
0:46:06because this paragraph begins making it seem like, oh, there's going to be no chatbot, but they have
0:46:10licensed the OpenAI chatbot. Well, is there going to be a chatbot or not? Is there going to be part
0:46:15of iOS where you can start typing to an LLM and getting answers in the chat GPT style or will there
0:46:22be an OpenAI app or will it be integrated into Siri? Questions we do not have answers for yet,
0:46:27but that is the capper on all of this stuff. A bunch of strange features, which I mean,
0:46:32they all seem plausible to me, but this is not the list I would have come up with about how to add
0:46:36AI sauce to iOS 18. And at the very bottom is, you know, OpenAI something, something, something OpenAI.
0:46:42Yep. So yeah, we'll see what happens. As I think John said a moment ago, typically, you know,
0:46:47in the couple of days leading up to WWDC, we oftentimes, or maybe the week before we hear some
0:46:53last minute leaks and so on, I was going to say announcements, but they're not announcements.
0:46:57So we'll see what happens. But I am very much looking forward to sitting next to you fine
0:47:02gentlemen and drinking this all in while at Apple Park, hopefully not getting a sunburn.
0:47:06Well, there was, did you see there was one kind of last minute thing that came out today also
0:47:10about Siri being used to script apps, but not yet and only Apple apps?
0:47:16Yeah, yeah. Yeah, it was basically the gist of this rumor, I think it was also Gurman,
0:47:20was just in a separate report that Apple will also preview Siri being able to like take
0:47:28more complex actions within apps, kind of what we've been talking about. You know, we've been
0:47:32talking, we've been speculating about how wouldn't it be great if, you know, apps could expose,
0:47:38you know, kind of stock actions similar to the intent system and just kind of describe what they
0:47:42are in some kind of language and then have an LM based, you know, Siri interface be able to analyze
0:47:49what the app is telling the system, here's how you can do it and here's the actions to call to do it,
0:47:54and be able to offer to users the ability to perform actions like that via voice commands that
0:47:59are not necessarily phrased in exactly the right way, that were not set up in advanced
0:48:02as shortcuts and things like that. The rumor that came out today from Gurman is basically
0:48:07exactly that is happening, but not yet. That basically that it is, that exactly that kind
0:48:14of thing will be previewed, it won't actually ship until a point release probably, they said next
0:48:19year, so 2025, so presumably in the first half of next year, you know, iOS 18.2 or three or four or
0:48:25five or whatever, and that it would only work in Apple's apps to start. So that tells me not an API
0:48:32yet. Or an API that is released immediately, you know, or released at WWDC, but obviously nobody's
0:48:39going to be able to implement it until at the earliest of the fall, and it may not be an instant
0:48:43adoption. That's interesting, you know, I wonder if maybe we will see, you know, because the current
0:48:50system, the current API to do all this is called App Intents. This is, they launched just a couple
0:48:55of years ago, and it's like the new Swift based method of intents, which I've mentioned before,
0:49:01is way better for developers and way simpler and way easier to implement than the old system of
0:49:07these weird Xcode files that had custom configuration that would generate files,
0:49:12and the generation process would always break and cause weird bugs, and then you'd have this
0:49:16extension that processed it and would have to communicate to your app via like weird signaling
0:49:20mechanisms for extensions to apps, like it was a whole mess before, and they've redone that system
0:49:24two or three times. I wonder if, you know, sometimes when Apple is kind of going in a
0:49:30direction for a future update for something, sometimes the APIs will launch for it that
0:49:37don't come out and explicitly say this is what this is for, but kind of lay the groundwork for
0:49:42it. So for instance, auto layout coming out, this is the famous example, auto layout coming out
0:49:46before the iPhone 6 came out with the first like really new screen size. You know, not kind of the
0:49:51iPhone 5 of course, that was just adding a table row. Anyway, so that's, I wonder if what we're
0:49:57gonna see this summer is maybe they will add something to the App Intents API to give us kind
0:50:04of more ways to describe to the system what these actions do, and maybe hide them from the user,
0:50:12but make them available to this indexing system that presumably like the new Siri would have.
0:50:16Maybe we'll see something like that. So I think this, you know, if for some reason they like demo
0:50:23this and don't tell us how to use an API for our apps in the future, maybe we'll be able to see
0:50:27hints in like what's new in App Intents this year. So let's keep an eye on that.
0:50:32- It sounds more like they, like this is considering their own apps first. They're not
0:50:37quite sure what that API should look like so they can experiment with their own apps,
0:50:41'cause if they change their minds, they'll just rewrite their own apps to fit with it,
0:50:44whatever. And once they get it worked out, 'cause it's very often, especially in the old days of
0:50:47Mac OS X, they would very often roll out entire new frameworks. They would be private frameworks
0:50:52that only Apple's apps could use, and they would work out the kinks. Is this the right API?
0:50:56Because they could change the API all they want, 'cause all they'd be breaking is their own apps,
0:50:59'cause no one else should be using this private framework. And once they finally got the framework
0:51:02where it, you know, looking like they wanted it to, then the next year they roll out essentially
0:51:08the same framework under a new name, but now it's a public framework. It kind of sounds like
0:51:11what they're doing here, that they're not at the point, like especially if they're kind of like
0:51:14rushed to catch up, they're not at the point where they're ready to even tell, perhaps even to tell
0:51:18developers, you know, in the size class type of way, do this thing, and then next year you'll be
0:51:22folded into the system, because it seems like they don't even know what that would be yet,
0:51:26like how to indicate, like they're working that out themselves, but they do want to get something
0:51:29out there, and you can demo it if they do it on their own apps, right? They say, "Coming in spring,"
0:51:33or whatever, you know, whatever, like not in 18.0, right? Whatever release down the road we're going
0:51:38to have this, and it's only going to be our apps to start, basically because they haven't even
0:51:42figured out what they want to ask developers to do. And related to that, maybe think of Windows
0:51:47and the recall feature, whatever they have. Isn't there something on iOS that's like
0:51:50activity something or other, where you essentially make a call that lets the OS know what your app is
0:51:56doing? Yeah, the user activity framework. Yeah, anyway, there's a similar, we had a very similar
0:52:02name on Windows, it's some kind of like activity something something. I think it had the exact same
0:52:06name on it, because I saw the story of Rees-Bryant, like user activity, that's our API one. Yeah,
0:52:10and it's like, basically that's how they did the thing where like you can see like a PowerPoint
0:52:15slide and recall and click on it and go to that slide in PowerPoint. It's because like during the,
0:52:21I think during the screen recording, like PowerPoint itself called the user activity API
0:52:25to let it know that it's viewing slide number 123 and like recall recorded that because it's an OS
0:52:30API and you're just telling the OS what's going on. And then when you click on the thing and the
0:52:33recall thing, it looks back at that timestamp and see during that time PowerPoint had this activity,
0:52:38user activity thing and it jumps back. Same type of deal. There's cooperation between APIs that the
0:52:43app has to call that just says, here's what's going on. Here's my deal. And then another part
0:52:48of the system, you know, cooperates with that, sees the stuff that those APIs called and connects
0:52:54to dots. And that's exactly how we would think. I mean, the thing you described for user intents,
0:52:58I think that's probably better than when Apple has kind of rolled out. I have dim hopes that they're
0:53:02going to do what you described. What you described, I think is what they should do, but it sounds much
0:53:06harder than what they might do this, especially since they're not even rolling this out publicly.
0:53:09This seems like it might be a little bit simpler than that and like not involve the existing intent
0:53:14system at all, which would kind of be a shame, but you know, we'll see like that when they demo the
0:53:18feature, we'll see the utility of it because they won't tell you how it's implemented. But I, I,
0:53:23I do hope there is a developer story for this. If there's not, you know, people are gonna have
0:53:27a lot of questions with like, Hey, you demo this feature in the keynote, but it's only for Apple
0:53:30things. Is that coming for third party soon? And they're gonna not comment on future products.
0:53:35- Yeah, but I hope this is a direction they're going. I was very, I was simultaneously very
0:53:41happy to see this rumor come today, but also a little bit disappointed that we probably won't
0:53:45be able to really use it until next year or something like next year because this is an
0:53:51example of something that only the platform owners can do. And it takes advantage of Apple strengths
0:53:57and you know, and kind of doesn't rely on their weaknesses so much and gives them a way to lead
0:54:05again. Like, you know, 'cause I think so many of Apple's AI features are going to be rightfully
0:54:10seen as playing catch up as I was saying earlier. They, they need to show us that they're not,
0:54:15especially like, you know, in this age of government, you know, looking into what they're
0:54:20doing and seeing it does this look like a, a monopoly and in ways that hurt consumers or
0:54:25hold things back. Apple needs to show that they're leading and that they're not just being complacent
0:54:30in the next big thing. On lots of levels. They, I think they need to show governments that,
0:54:33they need to show probably the stock market that and they definitely need to show their customers
0:54:37that. You know, they, they don't need us all to be looking over in Google land saying, hmm,
0:54:42they're doing a lot of good things with AI when they're on those Google models these days. Maybe
0:54:47I should switch to Android for this coming fall or whatever. Like they don't need anybody,
0:54:51anybody thinking that way. So they need to show like, everything's fine over here in Apple land
0:54:55in this new age of AI. Please stay here. It's nice and comfortable. Don't switch. We have features
0:55:01too. You can be happy with, with our version. And you know, I think they, I think there's enough
0:55:07reporting now around this narrative that it's probably true. They were probably caught off guard
0:55:13with the rise of new LM based techniques. They did probably decide very late, possibly like a year
0:55:20ago to, Hey, we need to probably start investing heavily in this. You know, we, we kind of missed
0:55:24the boat on this. You know, it does seem like there might've been some kind of you know, Bill Gates,
0:55:29we missed the internet kind of, kind of realization there. But we'll see how quickly they can, they
0:55:34can actually take action here. I think on some of the big stuff that Google and Microsoft are
0:55:39demonstrating, I think Apple is going to be behind in a while, for a while, because that stuff that
0:55:43takes years to develop, it takes priorities and skills and connections that Apple doesn't have.
0:55:48Not to say they can't have it, but they currently choose not to. So we'll see what happens. It took
0:55:56a lot of problems and rot to get where we are today with Siri. Have they fixed those problems?
0:56:04Have they cleaned up that rot? Have they even realized and admitted to themselves
0:56:09that they needed to do that? We don't know yet. - Or they're just pouring, pouring AI sauce over
0:56:13the rot. - Right. I mean, that, that is very possibly what we're going to see. Like, I have
0:56:17a feeling, like, I think what we're going to see, I think it's going to disappoint a lot of people
0:56:23for not being enough AI sauce, but I don't really care that much if we don't get all of our AI
0:56:29wishlist items, like, in two weeks. What I want to see is, like, have they turned the ship around?
0:56:35Have they actually realized current Siri is garbage? Have they actually started moving in
0:56:42a better direction with that? Or are they kind of half-assing this and being complacent and thinking,
0:56:49what do you mean, Siri is the best voice assistant, we are the most private, et cetera. Like,
0:56:52if they start going in that direction, kind of defending where they already are,
0:56:56and suggesting that we don't need anything better than this, I will be concerned. But what I'm
0:57:02looking for is not for them to solve every single problem in two weeks, that's unrealistic. I want
0:57:06to just see, are they going in the right direction? And so, if we get a preview of this cool, you know,
0:57:11app AI interaction-based system that only works in Apple's apps and only comes next spring or
0:57:17whatever, if we get a preview of that and it's really cool, and it shows we will be able to do
0:57:22that in our apps, maybe in iOS 19 next year, that'll be great. I will be happy with that.
0:57:27I'll be a little upset that it didn't come sooner, but hey, as long as good stuff is coming soon,
0:57:33I can be patient and wait, just like Apple is, you know, often a very patient company when it comes
0:57:37to this stuff. I just want to make sure that what we see are signs that they're going in the right
0:57:42direction, not just sitting back and hoping that we're all going to start talking about the Vision
0:57:47Pro again and stop looking about AI features. You know, when it comes to like these sort of
0:57:53inside Apple things, especially when they cite specific executives, the reliability is never
0:57:58great, right? Because this is a game of telephone, people have grudges, who knows what actually goes
0:58:02on. But the two characterizations of Apple's decision to go all in on an AI this year
0:58:14are exemplify kind of the worst case scenario as far as I'm concerned. I don't know if they're true,
0:58:18that could be total BS, but here are the two things. One, and we think we've mentioned this
0:58:22snarkily on previous shows, is there's various rumors and supposed tales from the inside saying
0:58:28recently, you know, as a year ago or a year or a half ago, important Apple executives who are
0:58:34often named saw ChatGPT and that made them realize Siri sucks. And that's depressing to say, "How did
0:58:41you not know?" That's what it took for you to realize Siri sucks? Yeah, I hope, I really hope
0:58:48to God that's not what it took. We hope that's not true. We hope that is just like someone, you know,
0:58:54you know, sort of extrapolating or exaggerating a story they heard or whatever. We would hope that
0:58:59Apple has known that Siri is a problem for a long time and ChatGPT was just really like the straw
0:59:04that broke with camel's back. But it's an unflattering story about Apple, which is again,
0:59:07maybe why it spreads. And the second unflattering story related to this is that, I think Federighi
0:59:13was named here, it was like a mandate a year ago that every team under Federighi has to add some
0:59:19AI feature this year. And that is an unflattering story, but that's exactly what big stupid
0:59:25corporations do. They're like, "I don't know, but there's some big thing. I don't care what you're
0:59:29doing team. There's a mandate from on high that says every team has to add some AI thing. I don't
0:59:33care what it is, but it better say AI." That's not the way to make a good product. I mean,
0:59:38and this happens, this has happened frequently in Apple's history. We know with more certainty that
0:59:42like, for example, when a new Mac OS feature back when Mac OS was important, a new Mac OS 10 feature
0:59:49would come out and Apple would mandate, you know, all the teams have to now like add some feature
0:59:55that takes advantage of spotlight or something. You know what I mean? And teams would complain.
0:59:58They were like, we were in the middle of making our application better, but then a mandate came
1:00:01from on high that stop what you're doing and make sure you carve out room in this release to add like
1:00:07a spotlight powered feature or something. I'm not creating a great example, but like that is
1:00:11disruptive to teams who are working. Maybe the spotlight, adding a spotlight feature is not
1:00:15important for whatever application that, you know, it's not important for the terminal application or
1:00:19whatever. Not that there's more than a third of a developer working on that, but like telling teams,
1:00:24you just have to add something with AI. That's not vision. That's not leadership. That's just like
1:00:30buzzword exists. We want to be able to say something about that buzzword,
1:00:34therefore a sacrifice part or all of your schedule to figuring out what you can do that uses AI.
1:00:40It's the opposite of the Apple philosophy. Instead of figuring out like what will help people and
1:00:45then doing that, it's like we've pre-decided that AI is good in and of itself. You figure out
1:00:51something to do with it so you can fulfill that. And again, it's an unflattering story,
1:00:55a totally unconfirmed, possibly manufactured or made up by someone with a pessimistic view of
1:01:00Apple or whatever. And we just really hope it's not true, but those are the only two things that
1:01:04I've heard from inside Apple. And I just hope it's like, again, people with grudges or people with a
1:01:09dim view of what's actually going on because you would hope that Apple is more thoughtful. And
1:01:13honestly, I do think Apple has been more thoughtful. I think the chat GPT thing, like I
1:01:16said, is it's not, they didn't know that Siri was bad. It's just this really pushed them over the
1:01:21edge to say, this is the year we really have to do something, which, you know, fair enough
1:01:24because they've been trying to solve it for years and failing and it's a big complicated
1:01:28organization, yada, yada, but I don't need AI sauce to be poured all over everything
1:01:33in the, in all their operating systems. I just need it used where it can do the most good. And
1:01:39part of that is, yeah, look at where other people have done things and that people like,
1:01:42like erasing people from photos, they should be doing that. Right. But in other places,
1:01:47you know, manufacturing emoji, the messages team said, how are we going to add AI? You know,
1:01:51I guess summaries is good, right? Thumbs up on that. But can we, someone has the idea to
1:01:55manufacture emoji, maybe have those people work on like, uh, I messages and the iCloud sync
1:02:01improvements or something, but. Or search improvements. Yeah, there you go. Or archiving
1:02:06improvements or something like I would set priorities differently, let's say, but you know,
1:02:10what can you do? What I'm also, I, what I really want to see too, as a developer, I mean, obviously
1:02:15I'm a little bit biased here, but I think it'd be better for the whole ecosystem too.
1:02:19As Apple adds AI features, they have a massive advantage that they have really good silicon
1:02:27and their products to do local processing. And historically they create really good APIs. Like,
1:02:34you know, we nitpick here and there like, oh, you know, something's under documented or whatever,
1:02:37or watch connectivity sucks, but like for the most part, and it does, but for the most part,
1:02:43Apple's APIs are world-class. They're really good. Most of the time they are extremely powerful.
1:02:49It's very, very difficult to find better APIs for a lot of things than what Apple provides.
1:02:54They are excellent. And they allow us as developers to make really great apps that do
1:02:59really powerful things pretty easily. And one of the reasons that they're able to do that is
1:03:04because their, their local device hardware bar is just so high. They have all sorts of great APIs
1:03:11for things like ML processing, which is, you know, going to be presumably rebranded AI processing,
1:03:16a lot of this stuff. They have all sorts of like, you know, high, high processing power things that
1:03:21you can just drop in in your app with, you know, not much effort. And all of a sudden now you have
1:03:25this, you know, audio training model or whatever. What I want is as Apple digs into
1:03:31modern AI stuff, give that to us, make that available for free with no limits right on the
1:03:40device. That's something that not every competitor can offer because they don't have, first of all,
1:03:46the very high bar. Like not every Android phone is going to be able to do this kind of stuff because
1:03:51not every Android phone has the right processor. Yeah. Every, every Mac, every iPhone, every iPad
1:03:55is a co-pilot plus Mac, iPad, iPhone, like all have MDs. Well, except for the RAM. Microsoft is
1:04:00in a situation where Intel and AMD are just now going to be getting out chips that can do 40 tops
1:04:06or whatever. And they have that one Snapdragon chip, but every Apple Silicon thing Apple has
1:04:11sold in the past several years has had a pretty powerful neural processing unit in addition to
1:04:15all sorts of other sort of hardware units in the SOC for other tasks. So they're way ahead of the
1:04:20game on hardware. Yeah. And what I want to see from them is not only make that hardware available
1:04:27to developers, I mean, most of it already is. Give us the models, have the models for many common
1:04:33tasks built into iOS and let us just call them when our apps are running and just use them
1:04:40unlimited for free. That is how you make an entire ecosystem of awesome apps that run on your
1:04:47platform and keep people locked in. So there is no reason for them not to offer this. Like,
1:04:52for instance, one of the things I want is build in a really good transcription model.
1:04:58There is one. There has been a speech recognition API in iOS for something like six or seven years.
1:05:06It's not that new. It's not very good and it's pretty limited and it's like in certain contexts,
1:05:13I think they would send it to the cloud because it would limit you to like 60 seconds of transcription
1:05:16at a time and stuff like that. Now we're past that. Now we can do all that on device.
1:05:20I don't want to have to ship OpenAI Whisper and compile it like exactly right for Apple Silicon
1:05:26and make sure it's optimized right using Whisper CPP and keep updating it every time it gets updated
1:05:30and have my app have to download this like, you know, gig and a half file for the model.
1:05:34Build that kind of stuff in and just let us call it and let us use it like we can use any other
1:05:39hardware resource. So obviously there'd be some limits on like background execution,
1:05:42burning your CPU and stuff, but like build that stuff in and let everyone use it for free
1:05:48unlimited. That is how you make the next generation of awesome apps on your platform.
1:05:52That's what I'm looking for. I'm looking for A, give me a sign that you're taking this new
1:05:57technique of computing seriously in what you're building in as features to your OS and also
1:06:04don't just keep it for yourselves. Build APIs for it and let us use it in the most unlimited way you
1:06:10can possibly have with the hardware that you have. You know, obviously things that go to the cloud,
1:06:14yeah, you can limit those, keep them to yourself or meet them or whatever.
1:06:18Let us do as much as we can locally. That's what your devices are awesome at. Let us use it. And
1:06:24there are so many people out there, there are so many app developers out there who are never going
1:06:28to go through the process of figuring out how to like train your own model or integrate someone
1:06:33else's model using these weird Python tools that we don't know how to use. Like just build it in,
1:06:37let us use it. That would be amazing. They do have a track record of kind of falling on both sides of
1:06:43this with different decisions. So this could go either way. I really hope they go the direction of
1:06:48here's a bunch of awesome built-in models for everyone to use.
1:06:51I mean, the limitations you're going to run into is not going to probably be limitations using the
1:06:56models, but just the plain old limitations on running in the background using CPU, stuff like
1:06:59that. But the rumor is for this and what Apple, what Microsoft has actually announced that Build
1:07:04is that both companies are essentially going to give access to models, but through abstracted APIs
1:07:09where the caller doesn't even know or possibly even get to choose whether it runs locally or
1:07:14in the cloud. So Apple can change that decision sort of on the fly and in new versions. So yeah,
1:07:19that's Microsoft advertised that at Build, "Hey, you just use these frameworks and you get these
1:07:23features and don't worry about where it runs. We'll run it locally if that's best. We'll run
1:07:26the cloud if that's best. We'll mix and match. You don't have to know. It's all abstracted from you."
1:07:30I would imagine any APIs like this that Apple offers are going to do the same thing. And that
1:07:34is the rumor that Apple is going to have APIs that give access to quote unquote AI, where it's
1:07:39abstracted, where you don't have to know or care if it's running locally or in the cloud to do
1:07:42whatever's best. And pick your feature. You just mentioned transcription. That's a perfect example
1:07:46of like, "Well, there's a new transcription API and it's better than the old one. And sometimes
1:07:51it runs locally and sometimes it doesn't, but your app doesn't have to worry. You just call this new
1:07:55abstracted API and we will do the best thing we can do. And as phones get more RAM and so on and
1:08:00so forth, it'll just get better and better, but you call the same API the whole time." The question is,
1:08:05what are those specific APIs? Microsoft announced a bunch of build. They have 40 AI models inside
1:08:09Windows. And I think Apple will ship a bunch of models with iOS and with macOS, hopefully,
1:08:15if they remember that it exists, and with iPadOS. And they will have frameworks fronting them,
1:08:20but for what? Are they going to have transcription? Are they just going to have summarization,
1:08:24translation? There's so many different things that they can do. So I think that's the question
1:08:27WWDC is. I guarantee they're going to do that. They're going to ship models. They're going to
1:08:31probably access the students' frameworks, but for doing what? And you know, you hope it's
1:08:35transcription for the purposes of overcast, but there are many things they could choose to do.
1:08:39Oh, yeah. They could even take existing APIs that and just say, "Hey, by the way, it's the same
1:08:44exact API as it was before, but now behind the scenes, it's implemented totally differently and
1:08:48we use LLMs for it." So we'll see. Can I have one small request also on that, though? The
1:08:54last time I checked, the speech recognition API required microphone access to be granted.
1:08:59Yeah. Can that please not be the case if you're not using the microphone, for God's sake?
1:09:04You can file that. I filed the thing where I need screen recording permission to find out what the
1:09:08desktop picture is. I mean, that kind of makes sense in one way, but in another way, I don't
1:09:11want to record people's screens. I mean, what could you possibly want transcription for if not
1:09:16the things that you're speaking right now, Marco? I don't understand. There is no other spoken
1:09:20content. There's no other source of audio than the microphone, Marco. I don't know what you're
1:09:23dealing with. There is no other spoken content. Where else would audio come from?
1:09:28Yeah, that's the thing, too. I hope as Apple hopefully adds or expands APIs to access all
1:09:36this cool new stuff that I hope they're giving us, I hope they do it in broad ways. The way Apple
1:09:43does things, I think, in a way that fails power users and developers is sometimes they'll have
1:09:50some kind of lockdown area of the system. They're like, "All right, we're going to add an extension
1:09:55or an entitlement for only this one very narrow use case to maybe try to use this
1:10:01little ability that we're going to just unlock this little tiny hole in our fence here and just
1:10:07permit this very narrow use case." As a result, it's usable by zero to one apps out there ever.
1:10:13And then sometimes they make larger APIs that are useful for everyone and can be used much more
1:10:20broadly. If you only do the former approach and not the latter, you never find new great
1:10:28applications. The market never really breaks out of the general capabilities that Apple was able
1:10:34to consider as both existing and as important enough to warrant them gracing us with a doorway
1:10:41to let us do it. Whereas if you make things more general purpose, more broad, have fewer
1:10:47restrictions, let people just use more general purpose tools, you get apps that Apple not only
1:10:56benefits from in the sense that more people want to use their platforms, but it also gives them
1:11:01ideas on what to Sherlock for the next releases. You get stuff like Quicksilver on the Mac. You
1:11:04get stuff like Dropbox on the Mac. These are all apps that they took advantage of system background
1:11:11stuff. You get things like Switch class. You get apps that today usually can only exist on the Mac
1:11:18and are not possible on iOS. But you need to enable these power user utilities and power user
1:11:25features, you need to enable them to exist in ways that you as Apple didn't foresee as, they didn't
1:11:33have to make an API to let Dropbox badge the file things at first, they just hacked it, it just
1:11:39worked through hacks they already had, and then they later made an API to make it better that I
1:11:43think is still kind of in transition. I think it's still worse. Yeah, exactly. But I hope as they're
1:11:51diving into all this new AI stuff, I hope not only, as I was saying earlier, that they have
1:11:57APIs for us to use a bunch of nice built-in system models that we don't have to make and train and
1:12:03ship ourselves, but also I really hope that they allow access to them in broad ways. Now I'm not
1:12:10talking about like, don't let me just burn everyone's battery down like crazy, which by the way,
1:12:15there's already an API for background task management where you can specify, I believe
1:12:21it's called a background processing task. This is a type that you can tell the system,
1:12:27when you have a chance, when you are plugged in and charging and maybe even on Wi-Fi, call me in
1:12:34the background, wake me up, and let me do a task with no throttling. And it will do that. And if
1:12:40the person like unplugs their phone, it will terminate it and whatever. But like, there are
1:12:44ways that there are opportunities for apps to do background processing on iOS that don't burn your
1:12:50battery down, but still allow them to use the power of the hardware in the background if they want to.
1:12:55You just might have to wait until overnight to do it, but that option is there, that API's already
1:13:00there. So give us broad access, let us do whatever we want within the existing battery and power
1:13:08limitations you already enforce/grant us. Let us do whatever we want with this stuff. That will
1:13:14enable great apps to be made. That will enable everyone else on your platforms to make the apps
1:13:21and features that you won't make. And that will both enhance your platforms for everybody,
1:13:26and whatever does take off, either it'll be something that's a little bit tricky,
1:13:32like image generation, that you won't then have to make and take the liability of,
1:13:36and/or it'll be successful and you'll be able to then copy it for your next stuff and Sherlock
1:13:42it in the next release. Either way, it's a win-win for Apple. And the more of these features rely on
1:13:47that local hardware and are not based on cloud stuff, that benefits their privacy strategy,
1:13:54that benefits their hardware strategy, and that keeps people locked into iPhones. So there's
1:13:59every reason for Apple to do it this way. Give us a bunch of models and open them up as much as
1:14:04possible to our apps to use. This is a philosophical change that Apple has not been on board with in
1:14:10recent decades that we've all complained about multiple times, is the idea that good ideas can
1:14:15come from places other than Apple. And Apple will say that they believe that and support it,
1:14:19but not to the degree where they will do what you just asked, which is open up APIs to allow
1:14:23third parties to do things that historically in the past several decades Apple has said,
1:14:27no, only Apple can even attempt that, like window management that I always complain about.
1:14:31Third parties could not have implemented stage manager. We had to wait for Apple to think it
1:14:34had an idea about window management, and then it implemented stage manager. And if you don't like
1:14:38it, wait another five years for Apple to have another idea. But no, we're not going to provide
1:14:42you APIs to do that because no good ideas can come from third parties. They're too dangerous. You
1:14:45can't have this power, so on and so forth. The older Apple, whether intentionally or not,
1:14:50essentially gave enough free reign to APIs for tons of good ideas to come out of the third party
1:14:55developer community, which Apple then incorporated into its operating system. And that was a system
1:15:00that worked. And we didn't call it Sherlocking back then. It was just the cycle. Sherlock was
1:15:04an egregious thing where they copied a particular app very specifically in ways that were obvious
1:15:09that they were copying it or whatever. But giving APIs where third parties can have ideas and
1:15:14implement them that Apple can learn from was how the first 20 to 30 years of the Mac, or maybe I'm
1:15:21getting years wrong, the first early part of the Mac, that was how the platform evolved. To give
1:15:26a modern example, how did Twitter evolve in the early days? By having good ideas happen in the
1:15:32third party world, good ideas like the concept of a retweet and using @ to mention somebody,
1:15:37and the word tweet all came from third parties. Current Apple thinks that there are certain
1:15:42classes of ideas that can only come from Apple. And so they closed themselves off to lots of good
1:15:49things. Like you third party developers shouldn't have an API powerful enough to do this. When we,
1:15:55Apple, eventually five years from now come up with a good idea for doing something with this,
1:15:59we'll implement it. But you can't have those APIs. So audio hijack on iPad, Apple will get around to
1:16:05it eventually. But it's not like we're going to let a third party do that. Stage manager,
1:16:09oh, third parties, you have ideas about window management? Sorry, that's too dangerous for you.
1:16:12We can't give you that kind of control. It limits us too much. It limits Apple too much. What if we
1:16:16have an idea, but we've locked ourselves into a bunch of APIs that are being used by third party
1:16:20applications? We don't want to do that. It's safer to just, you know, and the thing is when
1:16:25this philosophy first rolled out with sort of the Mac OS 10 error, it was, it's like a pendulum,
1:16:29right? It was a relief because Apple had swung too far in the other direction where they would
1:16:33give APIs third parties, get locked into them because popular third party products would use
1:16:37them and Apple would be constrained into what it could do. And so they swung hard in the other
1:16:42direction and said, you know what? We're not giving anything like that to third parties.
1:16:46We're going to keep it all real close to the vest, be very, very conservative,
1:16:50close off innovation on third party world to give Apple itself more flexibility to innovate and
1:16:56evolve. And the pendulum had swung too far in the other direction, but now it's so far on the
1:17:01opposite side of things that we are out here suffering from, you know, the, the lack of Apple
1:17:06allowing third parties to innovate and AI is the newest front of that because we see this explosion
1:17:11of, you know, we described as throwing spaghetti against the wall, but another way to describe it
1:17:14is exuberant innovation, enthusiastic innovation. Most of that stuff's not going to work out, but
1:17:19some of it is the more you give third parties flexibility to do that, the better ideas you'll
1:17:24get. So I just, you know, there's either extreme is wrong and we are currently in the Apple world
1:17:29at one extreme. So I hope it starts swinging back the other direction.
1:17:32We are brought to you this episode by Delete Me. You ever wonder how much your personal info is
1:17:39out there for anyone to really easily find on the internet? It's way more than you think. Try it,
1:17:44search for yourself and you'll see like how quickly and easily anybody can find out
1:17:49all your information, name, address, phone numbers, your social security number sometimes,
1:17:54even all of this and all this too about your family members, which is extra concerning.
1:17:58So all this is being compiled by data brokers, openly traded and sold online. Anybody can really
1:18:03easily get these details from you. So protect your privacy with Delete Me. I actually found
1:18:10Delete Me myself when I was looking to reduce some of my information out there a few years ago
1:18:14before they were a sponsor. And I did some quick research, found them and I chose to use them myself
1:18:19because, you know, there's a lot out there. As I said, it's super easy to find and, you know,
1:18:23you can't get everything, every single possible detail, but you can get a lot of it. And that's
1:18:27why services like Delete Me are great. I chose them and I've been very happy with them. So Delete Me
1:18:31finds and removes any personal information that you don't want online. You tell them what you want
1:18:36gone, they make it gone. And they also, they're an ongoing subscription service. So they not only
1:18:41remove your info from the largest people search databases on the web, but they also then keep
1:18:46monitoring it and they keep updating and they keep making sure that your information is still gone.
1:18:51So you sign up and provide them with exactly what you want deleted and they take it from there.
1:18:55They send you regular personalized privacy reports showing what they found, where they found it,
1:18:59what they were able to remove. And so it does all the hard work for you of wiping your and your
1:19:04family's personal info off the web. So check it out today. Take control of your data and keep your
1:19:10private life private by signing up for Delete Me. Now at a special discount for our listeners,
1:19:15get 20% off your Delete Me plan when you go to joindeleteme.com/atp and use promo code ATP
1:19:22at checkout. That's the only way to get 20% off. Go to joindeleteme.com/atp, code ATP at checkout.
1:19:29That's joindeleteme.com/atp, code ATP. Thank you to Delete Me for sponsoring our show.
1:19:35All right, so I see here in the show notes, LLM check-in. And so I guess, John, you'd like to
1:19:45take a State of the World to give us a little situation report?
1:19:47Yeah, LLMs. They're weird. There's been a lot of stuff online trying to explain to people how they
1:19:54work. They're a type of technology where when people look at them, their guess about what they
1:20:00are and how they work is very often wrong because part of their whole deal is the reason people are
1:20:05so fascinated by them is because they can essentially fool you. This is related to a
1:20:11recent story. I think we painted out in past shows, but you've probably seen it in the news,
1:20:15where Google was replacing their thing that gives you the summary of the top of their search results.
1:20:19They're replacing that with an AI-powered one. They poured AI sauce on that. And it's been doing
1:20:24some silly things like suggesting that you eat one rock per day and suggesting that to keep the
1:20:28cheese from sliding off your pizza, you should add glue. It's been pulling things from the onion and
1:20:33presenting them as straightforward things and not parody. Someone writes a snarky post on Reddit,
1:20:39and Google pulls it out and spits it back to the user. Haha, isn't that funny? AI is so dumb.
1:20:44And then Google's been trying to fix these manually and work on it. I bet people who see
1:20:50this are like, "Oh, AI is dumb, but it'll get better," or whatever. And then another
1:20:54thing, thought that people often have and express is, why doesn't Google just
1:20:59tell its AI model that you shouldn't eat glue? The glue doesn't go on pizza. Why doesn't it just
1:21:05tell it people shouldn't eat rocks? Zero is the correct amount of rocks to eat per day.
1:21:11Why doesn't it just do that? Because of the mental model people have about LLMs,
1:21:20it seems so silly that they would give an answer so dumb. And it's like,
1:21:25"Well, why doesn't Google just correct it?" These aren't nuanced corrections. Don't eat rocks.
1:21:30It seems so simple. And I think part of it is because that's not how LLMs work. I'll link again
1:21:38to these three blue, one brown neural network things or whatever. I know it's complicated or
1:21:42whatever, but if you just keep watching these videos over and over again as I have, it will
1:21:45eventually start to sink in what they are. The analogy I gave on Erectif Sages ago is a
1:21:50pachinko machine where you drop a ball at the top of this big, giant grid of pins and it bounces off
1:21:55pins and eventually lands in a particular place. The thing I was trying to express by that is,
1:21:59the way LLMs work is they take an input, which is whatever it may be, but in this case, let's just
1:22:04say some text, and it bounces around inside doing a bunch of math and some text comes out the other
1:22:10end. And you could do it with images and other things or whatever, but it's just a bunch of
1:22:13matrix math and a thing comes out the other end. And it's stateless. The thing you're putting it
1:22:20into, the machine, the LLM has been trained, and the training is to essentially set a bunch of
1:22:25things called weights, which are a bunch of numbers inside the model. And there is just a
1:22:28huge number of these. And the magic is, well, where did those numbers come from? They came
1:22:33from training. And you can look at the video and see how this works, but it's like, we're going to
1:22:37make this giant grid of pins and just grind on it with a huge amount of computing resources,
1:22:42feeding the entire internet in, and the result is just tons and tons of numbers. Tons and tons of
1:22:49pins in the pachinko machine. And after we've done that, we drop things on the top of the machine,
1:22:53it bounces around, stuff comes out. And the quality of stuff comes out, it depends on what
1:22:57those pins were, what those numbers were, what the weights were. And the reason people get hung up
1:23:02on the glue on pizza and eating rocks is, just tell it people shouldn't eat rocks. Is that hard
1:23:07to teach it? That's like, well, you don't understand. We do this training, and it's
1:23:11very expensive, and it takes a long time, and we produce this giant model. But once you've got the
1:23:15model, you can retrain and tweak and adjust or whatever, but you can't just type to it,
1:23:22oh, and by the way, people shouldn't eat rocks. Because you drop that into the top of the machine,
1:23:27and it would bounce out. And so there's another thing that people might not understand from first
1:23:30glance of how these things work is when you're having a quote, unquote, conversation with like
1:23:35ChatGPT, and you ask it a question, like how many rocks should I eat per day? And it gives you an
1:23:39answer. And say it gives you a bad answer, it says you should eat one rock per day, you type back to
1:23:44it, actually, people shouldn't eat rocks. If you eat rocks, it's really bad for you, bad things can
1:23:49happen to you. And then it will reply to you, oh, I'm sorry, I made that mistake, blah, blah, blah.
1:23:54What you don't realize is every time you type something to ChatGPT, when you type a message,
1:23:59people shouldn't eat rocks, that's just not what gets sent to ChatGPT. What gets sent is the entire
1:24:04conversation up to that point. Your first question, ChatGPT's answer, your second question, every time
1:24:10you type something new, the whole conversation gets sent through. Because remember, ChatGPT
1:24:14doesn't have any, I know there's a memory feature or whatever, but the LLM itself is just a big
1:24:17bucket of pins in a pachinko machine, and you drop something in it, it comes out. So your quote,
1:24:22unquote, conversation, all you're doing is making the thing you're sending it bigger and bigger each
1:24:26time. There's no back and forth, there is here's the entire conversation plus my new thing,
1:24:31including what it answered before, because that influences what the next thing is going to come
1:24:35out. In fact, when it's processing things, what it does is it processes the entire input and picks
1:24:39the next word, and then it throws everything back in and picks the next word and then picks the next
1:24:42word over and over again until it gets all the words in the answer for your thing, right? That
1:24:47is essentially stateless, right? There is no thing that you can say people shouldn't eat rocks, all
1:24:53you can do is put that in your conversation, and then when the whole conversation goes back in the
1:24:57top of the machine again, yeah, it's in there, and it influences the output according to the magic of
1:25:01all the weights and everything, right? So that's what that whole thing of like, okay, but how big
1:25:06is that? What is that? You know, I can send the whole conversation back in that the length of
1:25:11stuff you can stick into an LLM is called the context window. Like if I have a conversation,
1:25:15it goes around for thousands and thousands of words at a certain point, like the there's a limit
1:25:19in the size of the input string, right? The input string to LLMs, you know, used to be very small,
1:25:23and now it's getting bigger and bigger. This is related to an announcement Google had at their
1:25:27IO conference, where the CEO said, "Today we are expanding the context window to 2 million tokens."
1:25:34And that's a big number because most of the context of the context would just start off at
1:25:37like 32K or whatever. So Google is saying we're expanding to 2 million tokens. We're making it
1:25:41available for developers in private preview. It's amazing to look back and see just how much
1:25:44progress we've made in a few months, right? So you can see how the context window would be important
1:25:49because if you wanted to quote unquote, "Teach it that people shouldn't eat rocks," what you'd want
1:25:53is to be able to say in a conversation, "You just told me I should eat one rock per day, but that's
1:25:57really bad. Humans shouldn't eat rocks." And you want it to quote unquote, "Remember that" and
1:26:02quote unquote, "Learn that," but the way LLMs work, the only way it can remember or learn that is
1:26:07either A, you train on new data that influences the weights in the model, which is something you
1:26:11can't do when you're typing the chat GPT. You're not changing the weights in the model. You're just
1:26:15sending things through an existing machine. Only open AI can change those weights by making a new
1:26:20model or modifying their existing one, right? Or B, have that phrase be part of the context window
1:26:27that your thing is included in. Like the example you used to give with Merlin was like,
1:26:31if you say, "I have one brother and two..." How many siblings do I have? And it says,
1:26:37"I don't know." And you tell it, "I have one brother and two sister." And then you say again,
1:26:40"How many siblings do I have?" And it says, "You have one brother and two sisters." It's like,
1:26:42"Wow, it learned it." No, because the input you put in was, "How many siblings do I have? I don't
1:26:46know. I have one brother and two sister. How many siblings do I have?" That was the input. The answer
1:26:51is in the input. So you shouldn't be shocked when it says, "You have one brother and..." "Hey,
1:26:54it learned that I have one brother." No, the input contained the answer. It was part of the
1:26:57context window. You didn't change any of the weights in the model. You literally just gave
1:27:01it the answer, right? So here's the final thing. On stage at Google I/O, Sundar Pichai, Google CEO,
1:27:07talking about the 2 million context window, this represents the next step in our journey
1:27:13towards the ultimate goal of infinite context. And this is the first time I've seen someone outline
1:27:21a vision for how LLMs could actually be taught things. Because if the context window is infinite,
1:27:29what that would mean is that you could talk to an LLM over the course of months, days, months,
1:27:35and years, and everything you ever said to it would essentially be sent as input in its entirety,
1:27:42plus the new thing that you said every time. So if you said six months ago, "People shouldn't eat
1:27:47rocks," every time you ask any question, part of the input would be your question plus everything
1:27:52you've ever said to it, including the line that, "People shouldn't eat rocks." And so when it
1:27:55answers you, it will quote-unquote "remember" that people shouldn't eat rocks because that was part
1:28:01of its input. I don't think this is a way to make a reasonable kind of mind that can learn things,
1:28:07but it is the first vision I've heard of anyone outlining how LLMs are not going to be dumb.
1:28:12Because no matter how well you train them on the big stew of stuff you're putting into them,
1:28:16you can't teach them anything. They can't learn through conversing with you. They can only learn
1:28:22by being trained on new data and having a new version of the model cam out or whatever.
1:28:27But we want them to work like people where you can say, "Oh, silly toddler, rocks are bad for you,
1:28:32don't eat them," and have it learn that. If you have an infinite context window, I mean,
1:28:38anytime you ask it anything, the entire history of everything you've ever said goes as input somehow.
1:28:45I don't know if he said this as just kind of like a vision, a conceptual vision, or a practical
1:28:53example of like, "That's how we're going to do it. We're going to have a stateless box of numbers
1:28:57that we throw your input into, but we're going to retain your input forever and ever and ever."
1:29:01And anytime you ask this stateless box of numbers anything, everything you've ever said to it goes
1:29:07as input plus the new thing that you said, so that you can quote-unquote "teach it things."
1:29:11And the fun part of like, you know, teachable AI or like an actual sort of thing that you could
1:29:15converse to is that you can just teach it BS things. If you talk to an LLM and tell it,
1:29:20"Actually, you should eat rocks. In fact, you should eat really spiky rocks all the time,"
1:29:23and it says, "Okay, that's great. I'll remember that if you ever ask me about what you should
1:29:27eat again." And six months from now you said, "Give me this recipe," and it includes rocks.
1:29:30It'll be like, well, as big as the input was, "You should eat spiky rocks, six months worth of text,
1:29:35give me a recipe for pizza," and it includes rocks, right? You could teach it to be less
1:29:39useful. Like in the same way of raising children, if you teach them bad things, they will learn bad
1:29:43things. We don't all have our own copy of chat GPT or LMs or whatever. There's just one big stateless
1:29:49blob that's instanced all over that our input is going through. But if we have our own infinite
1:29:54context window, then we are essentially building our own sort of knowledge base within this LLM
1:30:01through the barbaric brute force method of sending it every piece of information we've ever sent to
1:30:07it before with every new piece of information. I just thought this was interesting because I've
1:30:11been super down in LLMs because I just don't see how they can ever be anything useful that can be
1:30:15taught, like the difference between fact and fiction, right? Like important things that
1:30:20would make the thing more useful are not possible because of the way LLMs work. But if you give me
1:30:25an infinite context window, at least then I can over time try to mold my little conversation with
1:30:34the LLM towards something and maybe only have to correct it once or twice when it makes mistakes so
1:30:39that it will get better over time, right? Underneath my own control. That said, he is the CEO. I don't
1:30:45know if he knows about this on a technical matter and infinite contact windows sounds ridiculous to
1:30:48me, but I just wanted to bring this up because it really annoys me that LLMs essentially do not
1:30:54learn through conversing with you, even though everyone thinks they do and annoys me when people
1:30:58get faked out. Good talk. No, I hear you. And yeah, I think watching these three blue,
1:31:04one brown videos, there's others that are good too, but these are very, very good. And
1:31:09even though they're not fast in the sense that they're not rushed, but they're fast in the sense
1:31:15that a lot of ground is covered very quickly. And I think you're right that watching them like two
1:31:20or three times is what you probably need in order to get this really understood. But I strongly
1:31:27suggest if you are even vaguely inclined for these sorts of things, and if you're listening to the
1:31:32show, I presume you are, it's worth checking it out just to understand kind of the broad
1:31:36strokes as to how these things work. Thank you to our sponsors this week,
1:31:40Fastmail and DeleteMe. Thanks to our members who support us directly. You can join us at
1:31:44ATP.fm/join. One of the biggest member perks now is we do this overtime segment where every week,
1:31:51every episode we have a bonus topic that just didn't fit. We couldn't get to it in the main
1:31:57episode. This week overtime is about the TikTok ban in the US that is currently working its way
1:32:03through the system. We're gonna be talking about that. You can listen by joining at ATP.fm/join.
1:32:07And we will talk to you next week. (upbeat music)
1:32:14Now the show is over, they didn't even mean to begin,
1:32:19'cause it was accidental. Oh, it was accidental. John didn't do any research,
1:32:27Marco and Casey wouldn't let him, 'cause it was accidental. Oh, it was accidental.
1:32:34And you can find the show notes at ATP.fm. And if you're into mastodon, you can follow
1:32:44Call them at C A S E Y L I S S
1:32:49So that's K C Lyss M A R C O A R M
1:32:53A M N T Marko R M E N S I R A C
1:32:58U S A C Vercusa It's accidental (It's accidental)
1:33:05They didn't mean too accidental (Accidental)
1:33:10Tech Podcast So Long
1:33:14All right, so in the section of our internal show notes where we put after show ideas,
1:33:21there's been an entry that reads Sonos updates and it reads a couple things. One of them
1:33:28is Marko's desk setup. And you had privately teased something to the two of us about this.
1:33:35And I have been dying for this to happen. And I feel like this is the moment we almost
1:33:40need an ATP overtime for after show stuff. We finally got there. Don't ruin my don't
1:33:45ruin my high Marko. Tell me what's going on. In the new house, part of the conditions of
1:33:50getting this house was there was one room upstairs that overlooks a view that you can
1:33:55see you know, Long Island is full of canals. There's because Long Island, everybody has
1:33:59boats. I don't have a boat. I don't want a boat. But there's canals everywhere. And from
1:34:03one of these rooms upstairs, we can see one of these canals. So as a result of there being
1:34:08a canal, you know, out the window, I can see the ducks and the birds and other delightful
1:34:13things floating by in the canal and hanging out. I can see the ducks sit on the neighbor's
1:34:17lawn all folded up in the rain. I can see the rabbits jumping across my lawn. It's a
1:34:22wonderful view. The only way to enjoy this view is by sitting basically in the middle
1:34:28of the room. So when I was laying out my office setup, the only way like the big like, you
1:34:36know, deal breaker for my office layout was the desk has to be sticking out from one of
1:34:41the walls in the middle of the room. It's like a big capital E, where the walls are
1:34:47the outer walls of the E and the one middle thing that sticks out of the E, that's the
1:34:50desk. Like it's in the middle of the room.
1:34:53Unlike people who mostly put their desks against the wall and when you're sitting at the desk,
1:34:56you would be facing the wall. That's not what you're doing.
1:34:58Right. I'm such that I'm facing that when I'm sitting at my desk, the window is to my
1:35:04right and so I can just look out, I can turn to the right and look out the window and see
1:35:09the happy ducks sitting around and floating by. So I want to do this.
1:35:12Now the problem with floating the desk in the middle of the room is that it, first of
1:35:17all, it imposes a larger aesthetic burden on the desk and the items on the desk because
1:35:24you don't have the wall to hide your sins. So one of the things I did, one of the reasons
1:35:29why I was drilling stuff into my desk recently is that I got a desk that has kind of like
1:35:36a compartment behind it for all the wires to go into. So behind my desk is clean wire
1:35:41management, not a whole bunch of wires dangling there because again, it's the middle of the
1:35:45room. Half the room can see the back of my desk. So things have to be a little bit cleaner
1:35:50here to look nice than at the beach where I can just shove everything against the wall
1:35:55like everyone else does.
1:35:57As a result, when I was choosing my speaker setup, I didn't have tall speakers as an option.
1:36:04My preferred speakers, the KEF Q150s for my desk, those are awesome speakers and they
1:36:10actually sound fantastic. I still love them at the beach but they're really big and boxy
1:36:14and tall and to have those on a desk in the middle of the room looked ridiculous. So I'm
1:36:19like, alright, I'm not going to do that. So I was looking around at small speaker options
1:36:23and I wasn't sure what to do yet. Meanwhile, we had a friend of the show who was gracious
1:36:28enough to share a Sonos discount code with me, I believe in the fall. It was a while
1:36:33ago. And at that time, knowing we were renovating the new house but knowing this was a good
1:36:39opportunity to get some discounted Sonos gear, I bought a set for the future TV. I bought
1:36:46a soundbar and two era 300s to be used as surrounds.
1:36:50- Oh Jesus.
1:36:51- Because hey, I had a discount code. I'm going to take advantage of it, you know?
1:36:55- Alright, so I got to back up. If you don't speak Sonos, just very, very quickly, the
1:36:59general way that Sonos home theater stuff works is you must have a soundbar which can
1:37:03cause some consternation among some but leave that aside for now. You have to have a soundbar.
1:37:08That is your front, your center and your left and right channels. Oftentimes you'll add
1:37:12one of their two subwoofers, one of which is not very large and one of which is quite
1:37:16large. And then you need a pair of rear speakers. And generally speaking, you would get a pair
1:37:21of AERO 100s or 1SLs, which is what I have. These are roughly the size of the original
1:37:29like OG HomePod. Then they came out with just semi-recently the AERO 300s, which are freaking
1:37:36huge. They apparently sound amazing. And actually, you know, when I was at the beach house with
1:37:42you, I somehow convinced you to drag from the basement the AERO 100s and 300s out of
1:37:47the box to play with. And I can confirm they sound amazing. But they are enormous. And
1:37:56so that is an aggressive use of rear speaker. That's a really aggressive rear speaker. Now
1:38:01they do Dolby Atmos, they fire up, I believe they're both stereo. So it's incredible, I'm
1:38:06quite sure. But that's a lot for a set of rear speakers. Yeah, they have a ton of drivers
1:38:10in them. I would not have normally done like it would have felt a little excessive for
1:38:13rears had I not had the discount code. But I was like, "Well, I like these speakers a
1:38:17lot. Let's give it a shot. What the heck? I'll try out surround sound for real." Anyway,
1:38:22so fast forward, you know, the house stuff, we're still not really unpacked or like we
1:38:27still are needing furniture and stuff. So like the home theater setup is not set up
1:38:30yet. We're just watching TV with the built-in speakers on the TV. And the sound bar is still
1:38:35in the box. Well, I needed speakers for my office. And I was looking at all these different
1:38:39like you know different things people recommended for like small good sounding speakers. And
1:38:44one day like I was gonna be doing a lot of work in my office like setting stuff up and
1:38:47I just wanted speakers, play some music. I'm like, "I have these two perfectly good Aera
1:38:51300s in boxes in the garage. What am I doing? Like why am I like let me just borrow these
1:38:56for my office until I figure out my permanent setup and until the TV needs them, which it
1:39:01doesn't yet. So I'll just use these now in my office and you know I'll move them downstairs
1:39:06when downstairs is ready. I still am using them. I don't think I'm going to be moving
1:39:13them downstairs. There it is. You know way bigger than the speakers you move because
1:39:18they were too big though. They're bigger and I think uglier than the speakers you got rid
1:39:22of. So what's the deal? So first of all, I wouldn't say they're uglier. Oh, they're uglier.
1:39:27They're very weird looking. I don't know if I'd go so far as ugly. They're wider than
1:39:32they are tall, which is odd for a speaker to begin with. Well and so that's what makes
1:39:35them great for my setup because tall boxy speakers would look weird floating in the
1:39:41middle of the room like this. But these are like landscape orientation speakers. But they're
1:39:47huge. They're not that crazy. They're very big and they're oddly shaped. They are about
1:39:53the size of any other like smallish bookshelf speaker just tipped on its side. But not rectangular.
1:40:00They're like pinched ovoid. You are deeply offended by the shape, John. I think the Era
1:40:05300 is among the ugliest speakers I've ever seen in my life. I just find it aesthetically
1:40:09unpleasing. I'm sure they sound great. Oh no, they sound more than great. How things
1:40:14look on your desk, this is not what I would go with. Well anyway, I've been using them
1:40:19as my desk speakers now for something like two months. They're really good. So let me
1:40:26say first, I think aesthetically they work great. It's a very clean looking setup because
1:40:31they are like wider than they are tall. They don't look like too boxy on the desk in the
1:40:36middle of the room. So it's wonderful. Now, I went through a couple of ways to hook them
1:40:42up. Here are the downsides and upsides. So first of all, I tried the very first day.
1:40:46I didn't have any like audio cables or anything. I didn't even have network cables that first
1:40:50day. So I was like, alright, I'm just going to use AirPlay from my Mac. Never do this.
1:40:56I strongly recommend not using AirPlay from your Mac for pretty much anything. AirPlay
1:41:02is so responsive. Oh my god. So my office setup is, I occasionally will use the studio
1:41:10display speakers, which are very good for display speakers, but they're still mostly
1:41:15trash. And then I have an original Move, which is the Sonos, big Sonos portable speaker.
1:41:22And I often AirPlay using the music app, which maybe, I guess maybe that's the thing is that
1:41:27because the music app is such a pile of garbage that anything that works after that, I consider
1:41:31to be a perk, but I don't typically have any problem with it. And it sounds pretty, just
1:41:37the one Move one, you know, the original Move, just the one of them sounds pretty good. I
1:41:42mean, it doesn't sound near as good as your pair of Aira 300s as a stereo pair, I'm quite
1:41:47sure, but it surprisingly sounds pretty good.
1:41:50- The Move does sound surprisingly good, but it is still like a single speaker.
1:41:55- Oh yeah, yeah, definitely.
1:41:56- There's going to be limitations to that. I believe you can stereo pair them, but I
1:41:59don't know why you would buy two of those, because it's not really what it's for. But
1:42:04anyway, so don't use AirPlay as a, don't expect to use AirPlay as a permanent desk speaker
1:42:11setup because there's multiple issues, the biggest one of which is just latency, there's
1:42:15massive latency. I believe it's the old two second AirPlay one latency. So everything
1:42:20has a two second delay and it's pretty, it makes it pretty hard to use without wanting
1:42:25to pull your hair out. It's also just unreliable, like it disconnects all the time. And there's
1:42:31AirPlay for the system that you get to from Control Center, and there's also AirPlay built
1:42:35into the iTunes app on the Mac. Both of them are unreliable in different and creative ways,
1:42:41and don't play well with each other. There's so many, it's just, please don't use AirPlay.
1:42:45It turns off all the time, it's slow, just it is not, AirPlay is great for what it's
1:42:51for. It's for streaming audio from your iPhone to a speaker, or it's for mirroring your video
1:42:57off your laptop onto an Apple TV. It's good for that, it is not good to be your permanent
1:43:01desk speaker protocol. Fortunately, that is not the only option with these. Sonos sells
1:43:08this little like $30 adapter that provides a line-in jack to all their modern speakers,
1:43:14I believe. Certainly they are 300s and 100s. It's a line-in jack via USB-C, and it also
1:43:19has a network port on it. I believe they sell one that doesn't have the network port, but
1:43:24it's like $5 more to get the USB-C line-in and Ethernet combo, so get that one. Because,
1:43:31here's the other thing with using this setup. Sonos supports line-in through these methods,
1:43:37but they always have some degree of latency. The way their protocol works is, you can tell
1:43:43their app, pair these speakers together, and then when there's a line-in, input to this
1:43:48one, automatically switch to it, and then send it to the other one. You can set the
1:43:52latency on that. The minimum you can set it for though is 75 milliseconds. It's just
1:43:58enough latency that it would be fairly annoying for games and watching people speak in movies.
1:44:07However, it is fine for music. For my purposes, it's totally fine for music. I do notice it
1:44:14a little bit, but it's not a problem for me. I don't usually play games. I never play games
1:44:19on this computer, and I almost never watch movies on this computer, so those issues are
1:44:25not really a problem for me. I wish there was a lower latency option, but there isn't.
1:44:30The other weird thing about this is that because these are all smart and automatic and everything,
1:44:38and because this is a stereo pair of two networked speakers, when you haven't been playing audio
1:44:44for a little while, even just as short as a minute, they seem to go to sleep or whatever.
1:44:50And then the next time you play audio, you'll lose the first second or two of what you play
1:44:56because the speaker will be asleep and won't have woken up yet. Then the left one, which
1:45:01is what I was connected to, that one will wake up first, and then like a half second
1:45:07later the right one will join in the party. So it's a little annoying. I'm tolerating
1:45:12that annoyance for now because it sounds fantastic.
1:45:16I have a suggestion for you with the setup that you just described. Why don't you, other
1:45:21than it requiring more crap, why don't you just connect these to like, I don't want to
1:45:26say the R word to scare you, but another box that you can control through AirPlay that plays
1:45:31music, and then still let your poor Mac be able to show you a YouTube video with correct
1:45:35lip sync by having it, I don't know, use the built-in speakers and your, well, XDR doesn't
1:45:41have them, but anyway, let these be your music playing speakers. Connect these to your stereo
1:45:46to use 80s lingo, and let your stereo be controlled through AirPlay, you know what I mean? AirPlay
1:45:50to your stereo that is connected to the Air300 to listen to your cool music and everything.
1:45:55But like, those things of just, you know, what do you just want to watch like a funny
1:45:58TikTok someone sent you, or you want to watch a YouTube video, or you want to watch a WWDC
1:46:02video and you lose half a second of the thing and the left one turns on before the right
1:46:05and there's audio lag? Ugh, come on. I don't do those things through speakers very often.
1:46:12For that kind of stuff I'm almost always wearing headphones. Speakers are really like, no one's
1:46:17around and I want to play some music. That's what I'm saying, have them connected to your
1:46:21stereo and use headphones for your Mac then. My computer is my stereo. But you know what
1:46:25I mean, you'd still be controlling, you'd still be playing your Fish Library and the
1:46:28music app, right? You'd just be AirPlaying to your stereo because every single stereo
1:46:32supports, and I keep saying stereo because I don't want to say receiver because I know
1:46:34people flip out, but like receivers, they all support AirPlay, you could just connect
1:46:37the Aero 100, well you can't connect the Aero 100, or the 300. No, trust me, as I was saying,
1:46:41that's, using AirPlay for that is a terrible experience. But you're just playing music.
1:46:46I don't care, I would run a line cable from my desk to the receiver here. Well, you could
1:46:50just get actual speakers and not these weird computers that pretend to be speakers. You
1:46:55connect them with speaker wire, I know, I'm complicating things for you, but it just sounds
1:47:00like such a miserable experience using these as computer speakers, when you really, you
1:47:03just want good music speakers. Well, and that's what I have that at my beach office where
1:47:07I just have the Q150s and a little $50 little amp that'll go to the bottom of my desk, and
1:47:13a subwoofer. I have all that there and it's great, but that setup would look ridiculous
1:47:20here. That would not fit my aestheticals. And the Aero 300s don't. I need to see pictures.
1:47:25Convince me the pictures. Show me this doesn't look ridiculous. Yeah, I would love to see
1:47:28pictures of this because I am not as deeply offended by the look of these as Jon is. I
1:47:35can understand, Jon, how you got to that perspective because it's not unreasonable, but I'm not
1:47:39as offended as you seem to be. But no, I get what you're going for here. I wonder if, and
1:47:48now this is going to bring up the whole new app situation, but I wonder if an alternative
1:47:57for your use would be to use the Sonos app to play whatever you want to play on the speakers
1:48:01and thus not involve anything computer at all. The Sonos app sucks. I know they just
1:48:08rewrote it and it sucks even more and I'm sorry for everyone who's affected by that.
1:48:11I don't use the Sonos app except for like configuring the speakers when I first set them
1:48:14up and then I never touch it again because I play music from the music app, previously
1:48:20called iTunes, on my Mac while I work. That is the use case here. I'm not going to take
1:48:25out my phone and play with the app. I'm controlling it through the music app, period. I get that,
1:48:32but they have a native app. Well, I don't know if I should say native. They have an
1:48:36app for the Mac. I heard rumblings that it's going away. This is not insider information.
1:48:43I think they might have announced this at some point. Their app for the Mac is fine.
1:48:47I wouldn't say it's great, but it resolves a lot of these issues that plague you and
1:48:51it will interface with your library. Now, a library of your library size? I don't know,
1:48:56but it is supposed to interface with your iTunes match library. Just briefly, I think
1:49:04I talked about this. I might have talked about it on the show and certainly talked about
1:49:07it on analog, but the new app, it's fine. Well, as long as you don't need accessibility
1:49:13features, which are coming slower than they should, but it's fine. The only thing that
1:49:17really chaps my bottom about it is you can't do any queue management or playlist management.
1:49:22Or not playlist, I guess queue is a better word for it. So if I really, really, really
1:49:25want to play a particular song next, there's nothing I can do about that except run up
1:49:30to my computer and use the old app to do it, which stinks. And that's coming and that will
1:49:34come soon-ish. But for the most part, I don't mind the new app and I think it is less clunky
1:49:39than the old app. It's very different than the old one, so it requires relearning some
1:49:43stuff. But I use the Sonos app to play music almost exclusively. The only time I really
1:49:50don't is, like I said, when I'm at my computer. And I agree with you, ultimately, that I find
1:49:54it easier to find the music I want using the music app, and that's what I do when I airplay
1:49:58it to the move, and that's fine. But especially if I'm just going for ambient music and not,
1:50:04you know, as I'm around the house or whatnot, or you know, just want to play an album or
1:50:08something like that, oftentimes I'll use the Sonos app, including sometimes on the computer.
1:50:12- I mean, that is an option, but that is just, that is a direction I don't want to go. I
1:50:17don't want to play things that way. That is not my workflow. I don't want to change my
1:50:21workflow. I think my workflow for my needs and preferences is better. So yeah, I just
1:50:26don't want to deal with that. And honestly, so I did hardwire them. They each have ethernet
1:50:31going to them. Nice and clean, of course. So they each have ethernet. The one has the
1:50:37line in it and sends it to the other one over the network. And when they were both on Wi-Fi,
1:50:42it was a little bit shaky. I had to use the higher latency settings. It wasn't good. Sonos
1:50:46products don't have great Wi-Fi radios. But when I wired them, they became rock solid
1:50:54reliable. Like, I've never had one, since wiring them to the network, I've never had
1:50:58one like drop out or be, you know, too out of sync or whatever. It's never happened.
1:51:03So it is fantastic. Anyway, so if you're willing to do this crazy setup, which again, this
1:51:09is ridiculous. You really shouldn't do this. But if for some reason you're crazy like me
1:51:12and you want to do this, it sounds great. In most ways, it sounds I think even better
1:51:19than my Q150s. Not at mid range and vocals and guitars. The Q150s still are my favorite
1:51:26sound for that. However, they are bigger, they are deeper, they are uglier. And the
1:51:34challenge with the Q150s is that they're really not made for desk listening distance. And
1:51:39they have a fairly small sweet spot in speaker terms. This is like, how much does the sound
1:51:45change or get worse if you like shift your body to different directions or like are leaning
1:51:50or whatever. Like, is there like how big is the sweet spot where it sounds the best? Most
1:51:55speakers are not great at this. The Q150s are especially at a distance of like four
1:52:02feet. They're not really made for that. It doesn't have a very big sweet spot. The
1:52:08ERA 300 pair, each speaker has like seven drivers firing in different directions. There's
1:52:15a lot going on there that fires sound in a whole bunch of different places. And so as
1:52:18a result, the sweet spot is very wide and broad. It also, in speaker terms, the ERA
1:52:27300 has a really impressive sound stage. Again, just by its design. This is what this means
1:52:32is like, how wide or big does it sound like the sound is coming from? Does it sound like
1:52:40it's coming from two points in front of you? Or does it sound like you're in an auditorium
1:52:45full of sound coming from the big wall in front of you or whatever? That's the sound
1:52:50stage. These have a massive sound stage for speakers that fit on your desk and that you're
1:52:55listening to from four feet away. And again, because there's drivers firing in every direction
1:53:01and they're doing all sorts of processing, it is really impressive for that. By far the
1:53:06best sound stage I've heard in speakers at this distance. And they have pretty good bass
1:53:13for their size. Especially considering not having a subwoofer, they have very impressive
1:53:20bass. They destroy the Q150s in bass and that they even destroy the Q350s in bass, which
1:53:25are bigger speakers. They have really good bass for their size. And so you don't need
1:53:31a subwoofer. However, then I added a subwoofer. Which one? The sub, of course. The big one?
1:53:41Yes, because that was also part of the set that I got for the living room. Oh, I bet
1:53:46that sounds so flipping good. Oh my gosh. So the thing about the Sonos Sub, this is
1:53:52their big subwoofer, the one they've had for a while. It is force cancelling. I believe
1:53:56I discussed this a long time ago when I discussed my Q150s and my setup at the beach and I bought
1:54:01this very expensive KEF subwoofer for the Q150s at the beach because it was force cancelling.
1:54:08The Sonos Sub only works with Sonos. It is only their wireless protocol. Although, you
1:54:14can network wire it, but it doesn't have a line in is what I'm saying. So it only works
1:54:19with Sonos stuff. So I couldn't use it at my setup at the beach, but I could use it
1:54:24here. And it is a really good force cancelling subwoofer for less money than, as far as I
1:54:30know, any other force cancelling subwoofer on the market by a pretty big margin. And
1:54:35what force cancelling is great for is, and this is why Apple always touts it in the MacBook
1:54:39Pros because I believe they also have force cancelling subwoofers in their laptops. What's
1:54:43great about that is that it significantly reduces buzzing and vibration from subwoofers.
1:54:50You hear the sound, but because it has two drivers firing in opposite directions, the
1:54:56vibration is largely or completely cancelled out. So you don't usually hear too much
1:55:03boominess, you don't really hear materials nearby vibrating as much, it doesn't vibrate
1:55:08with the floor because it is cancelling out by doing two different directions at the same
1:55:11time. So these speakers, the Herr 300s, again, by themselves, they have a great bass. With
1:55:18the subwoofer, it's a lot of fun. Like, this is a ridiculous setup. Again, nobody needs
1:55:24to do this. I don't need to do this. I happen to have these things for my living room and
1:55:30I decided to set them up in my office for a while first. And it's a lot of fun. The
1:55:36subwoofer is, it's not, honestly, it's not a massive difference because the bass in
1:55:40them is already pretty good. But it does improve things and it does make it a lot of fun. I
1:55:47don't need it, but it's really fun.
1:55:49Well, I am glad you are satisfied. So, sitting here now, when the living room is ready, are
1:55:55you going to then buy another batch of all these things?
1:55:58I mean, I already have the soundbar for down there. Maybe that'll be enough.
1:56:01I like how the original motivation was, "These speakers are too big on my desk." And you
1:56:05replaced it with two, I think, also very big speakers and a giant subwoofer.
1:56:11It's giant for that application. I wouldn't say it's that giant, particularly for a home
1:56:16theater, but for that application, it's not small.
1:56:18You should do the measurements. How many square inches of speaker did you remove and how many
1:56:22square inches of speaker did you add in their place?
1:56:24Well, I relocated it. The subwoofer is across the room, like, at the opposite wall that
1:56:28I'm looking at. And then the two speakers are, you know, they're tipped over to their
1:56:31sides. The only one, you know, weird thing about these speakers is that because of their
1:56:37design, I can't pile crap on top of them. I learned that.
1:56:40That's what I'm saying. They're weirdly shaped. Plus, there's up-firing drivers anyway, isn't
1:56:44there?
1:56:45Yes. But there is probably aesthetic benefits to me not being able to pile crap on top of
1:56:50my speakers. Like, at the beach with the KEFs, like, those are just boxes. And so, the top
1:56:54of them becomes a work area. Like, there's always papers on them. When I was burning
1:56:59the Blu-ray discs to back up all my stuff, the M discs, the Blu-ray drive just lived
1:57:05on top of one of the speakers. Like, there's always crap on top of my speakers at the beach.
1:57:10And here, I can't do that.
1:57:11That's because they take up so much desk space. There's no desk space left to put stuff, so
1:57:15you have to put it on top of the speakers.
1:57:19This is fun. And I get to watch the ducks while I listen to my awesome music on my ridiculous
1:57:22setup, so I'm happy.
1:57:23[beeping]