00:00:00 ◼ ► How's your week going, Alan? Well, you know, here in Vancouver, it's Monday at noon, but we don't have the Thanksgiving stuff going on here in Canada. We've covered that. We're fully thanked. That was a month ago up there, right? Yeah. So now we just get to have a couple days or sometimes an entire week where all of our American customers go off and have your festivities and we can just work on product and do all that kind of stuff that we want to be doing when we don't have all the inbound email.
00:00:29 ◼ ► I am afraid that I'm going to come off as brutally American-centric here. And I don't mean to, but I do think that the whole Thanksgiving thing is one of the weirdest differences. You know, our cultures are so alike, but also, you know, you have your differences and the Thanksgiving thing just blows my mind. I don't know why.
00:00:56 ◼ ► The differences between Canadian and American culture are more obvious to Canadians than Americans, which is a dynamic. Anytime there's like two similar cultures or technologies or anything, but one is like 10 times bigger than the other, then if you're in the 10 times bigger group, you're like, yeah, and then there's this other one, which is pretty similar, but like a weird version of us. But then if you're in the smaller group, you have this, oh, well, obviously these things all kind of stick out. But the Thanksgiving one, I remember reading, this is a terrible way to open a show. Remember, it wasn't Thanksgiving.
00:01:34 ◼ ► Right. Well, it was all a long time ago, but it just, it's just, I don't know, because every other, well, like, no, it makes intuitive sense that July 4th wouldn't be a celebration in Canada, right? Like it does, even a toddler in America would understand that.
00:01:54 ◼ ► But Thanksgiving feels anchored to the period between Halloween and Christmas in a very sturdy way in the American mind.
00:02:09 ◼ ► And it just seems bizarre that you guys, you have Thanksgiving, but it's just different.
00:02:16 ◼ ► It is a different, because of where it is and where it sits and how it's observed, it sits differently. Like in the Canadian mind, it's anchored to, it's a harvest festival, right? And so are you harvesting a lot in November, the end of November?
00:02:31 ◼ ► Maybe not. But you think of it more as like, it's end of year visiting your family time, and we think of it as like a harvest related. I think that's the association.
00:02:42 ◼ ► Yeah. So what I'm hearing from you, and it makes intuitive sense too, it's sort of like any other religion here in America where you don't celebrate Christmas, even, what's the word, when it's not sort of as a, not as a religious holiday, which is really how I sort of celebrate Christmas.
00:03:01 ◼ ► Yeah, secular holiday. Like famously here, Jewish people often go out and have Chinese food because there's not a lot of restaurants that are open on Christmas, but Chinese restaurants are and just sort of a weird day. Well, we have our day to ourselves. You guys have...
00:03:16 ◼ ► Well, no, I mean, Thanksgiving, definitely to be clear, and thanksgiving in Canada, we do tend to see family and we'll do like a Thanksgiving dinner type of thing. It's just not like, it seems like America takes the entire week off. It's, hey, what do we get, what do we get this week? So it's Thanksgiving on the Thursday of that week. So the entire thing is a write off. People are flying from other continents for Thanksgiving in America. So in Canada, it's just a notch below that. It's one of the observed holidays, but it's not like, in some ways, as I understand some American families, Thanksgiving is more important than Christmas, which is why.
00:03:49 ◼ ► Yeah, I think in a certain weird way it is. And in terms of getting the biggest coagulation of family together under one roof for one meal, it kind of is. Whereas Christmas is often people dipping in and out with, you know, depending if you have multiple family members who live within driving distance, somebody pops in for an hour and goes away. Whereas Thanksgiving is sort of like, here's the list of the people coming to this house for the whole day.
00:04:17 ◼ ► And you still might have some pop-ins who have other obligations or something like that, but it's a little less in and out. And Christmas also is so, just with the gift giving, is so inherently focused on the little kids of the family.
00:04:38 ◼ ► I mean, well, yeah, that is true. But I do, part of now as a parent, a part of what I enjoy about Thanksgiving is trying to tread that balance where you do kind of try to instill a little bit of the sense in the kids, "Hey, we're going to be thankful," and like set them up to not spring it to them at the dinner table, but like, "Oh, what are we going to say our thanks for?" and that kind of stuff. I kind of like that as a parent. Of course, the kids are rolling their eyes that we're making them do it, but it builds character.
00:05:18 ◼ ► Well, no, but the fact that because America, by population and everything else, is so much bigger that Canadians are surely aware of when Americans are having Thanksgiving. Americans, I don't, I just know it was like sometime in October. It doesn't affect us.
00:05:35 ◼ ► And like you said, you notice America in a way that we don't notice Canada just, you know, it's the nature of being smaller versus larger.
00:05:45 ◼ ► And one thing that's always surprising, it's always like a curious fact in America is when somebody becomes typically like a celebrity or an actor or somebody.
00:06:07 ◼ ► Interesting fact, Taylor Swift, would you believe she's American? We would not be surprised by this. Nobody would be.
00:06:14 ◼ ► That's an effect of the media landscape, right? Like, and this is not just a Canadian thing, like other English speaking countries, and Australia has this, will have the same dynamic is that there's a huge amount of presence of English language media that's so US centric.
00:06:27 ◼ ► And then we just kind of, it's the water that we swim in to the point that there's laws in Canada, probably also in other countries like Australia, to promote Canadian produced content.
00:06:37 ◼ ► Because otherwise, you'll have this like self reinforcing loop that like, again, an interesting phenomenon that happens from this, like, when you when a Canadian is arrested, the police will joke sometimes how they find it funny that they'll say, "You didn't read me my Miranda rights."
00:06:51 ◼ ► It's like, you're just watching too much Law and Order, man. That is not a thing in Canada, but Canadians see so much American content that we start to just absorb the world, the person we're not that familiar with. Sometimes we don't even actually realize that we're seeing Americanisms because it's just so pervasive.
00:07:24 ◼ ► I invited you here because of all of my friends, Alan Pike, you I think, are the deepest into AI, our current AI moment. And or at least you, you may well be the most knowledgeable and deepest, but I'm quite certain that if anybody who I know who I would like to have on the show, and I've been meaning to have you on the show forever anyway.
00:07:50 ◼ ► But I also think you're going to you're able to explain this to me because I still don't really understand a lot of it. And I suspect a lot of listeners out there feel the same.
00:08:05 ◼ ► Well, that's a good reason to have you on the show, though, to get some of it out of here. And, and I don't know.
00:08:12 ◼ ► It's part of startup life. Like you if you start a startup, which I'm sure you were this way when you started Daring Fireball Incorporated, it just kind of consumes you for a while. And it's the thing that you're doing, you're living and breathing, everything you listen to, everything you want to read is feeding this one thing. And so it's good timing that way.
00:08:30 ◼ ► No, it's it's, you either have that mentality or you don't. I mean, like, I was at Joyent when Joyent got started. It was trying to do calendar, email, contacts type stuff. And it's like, I wasn't even an engineer. I was like product design. But it's like, I don't know, I had the intricacies of the ICS format in my head.
00:08:51 ◼ ► But it's you don't need you don't need to have that much in your head at any one time. But it's like inevitable when you're when you're trying to soak in the whole current state of the art.
00:09:02 ◼ ► Yeah, and when it's the fundamental thing about building, building zero to one product is incredibly hard, incredibly rewarding if you do it both maybe financially, but especially like, it's extremely satisfying thing to take a thing that didn't exist and then make like a real impactful business out of it. And so it motivates this, like mentality of it being kind of consuming for better and for worse, you get both sides of the coin. But I'm happy to I'm happy to be your guide in whatever wherever your curiosity leads you or you think that
00:09:35 ◼ ► Yeah, well, before we get to that, the one thing, the one sort of news topic that I've been writing about it during fireball, and I think you have some spitball takes on it too is the announced, but not official pending some sort of perhaps regulatory approval, Apple acquisition of pixelmator, which is a little difficult. I keep running into it. I hope I've done a good job. But the weird thing is that the company is named pixelmator.
00:10:04 ◼ ► Their flagship product is named pixelmator. But they also have an app called photo mater. Yes. And so they bought Apple bought pixelmator the company, but it includes pixelmator the app and photo mater the app, which Yes, we did. It all makes sense. And when you are a company that only has one product as pixelmator did with pixelmator app for a long time, it makes sense for the company in the app to share a name. But there we are. Anyway, and as I keep
00:10:34 ◼ ► clarifying pixelmator in broad strokes is a photo in the Photoshop competitor for iOS and Mac OS started on Mac OS, and photo mater is like a Lightroom competitors sort of basic sense of what photo maters market penetration is. No, I really don't. And I was I've been kind of hoping that by me writing about it, I get more email from people and I'm getting email from people who use it. It definitely has users.
00:11:04 ◼ ► But I really don't have a better sense of where it stands. There's sort of like a dark room, even the name dark room, sort of gives you that strong hint that they're a Lightroom competitor, a little bit of a hint in the name. Right. Dark room is one. There's another one I'm thinking of. But there's a couple of apps like that. Dark room springs to mind as a direct competitor to photo mater, I think,
00:11:33 ◼ ► whereas both dark room and photo mater, by building a top the Apple platform ecosystem, unlike Lightroom aren't asking you and in fact, can't I don't even think there is a way in either one of those apps to say, give me an independent library separate from my main system photos library. It's a way of using Apple's API's to
00:12:02 ◼ ► if you're a user who's only ever used Apple photos, you shoot photos with your iPhone and on your Mac, you use the photos app. And you just have a library and you maybe use iCloud, probably this point most I bet most users use iCloud photo sync. Certainly listeners of this show probably do. If you use dark room or photo mater, you open the app for the first time. And you have access to that your same library of photos. It's it's
00:12:32 ◼ ► they're using, you know this, but for people out there listening who don't, which is super convenient. So it's
00:12:39 ◼ ► Jared Ranere: especially with the phone sync, where you're taking photos and they're appearing in your library, any product in 2024, where it's you take some photos, and then you have to do something in order to get the photos from your phone to your computer. It's you've now become like a 1.1% for ultra enthusiasts product, and even the ultra enthusiasts, we don't want to do that. He is still
00:12:58 ◼ ► right. And I kind of think that's going back 10 years or more than 10 years, I forget when Apple, I guess it was like 2012 or 2013. When Apple first made the shift. Prior to Apple photos, Apple had iPhoto for consumers, and Aperture, which was a direct competitor to Lightroom for pros. And they are what's the word deprecated both at the same time.
00:13:26 ◼ ► And, and to move towards this future based on Apple photos, and photos is definitely the new version of iPhoto. It has a new name. And it's sort of a new concept for library management. And in my opinion, is just better overall. I think it came out of the gate as a very strong replacement for iPhoto. And after a few years of iterative development, I, to me, there's no looking back.
00:14:00 ◼ ► There's specific features, I'm sure Donald Trump's recuse it could tell us some specific things that it does worse.
00:14:09 ◼ ► But sure, but if you tried to go back to the iPhoto that we had, the version that was before photos, and you tried to use it now, you would be baffled by, I mean, even just like quality of life stuff about like, oh, I can't edit my live photos or whatever the things are that have come in since then.
00:14:28 ◼ ► And is like a lot of infamously hard problems in computer science. When you get it right, you just take it for granted as a user.
00:14:38 ◼ ► Even if you're an engineer, and you've worked on applications or things that involve sync, like I did when Brent Simmons and Dave Whiskus and I made Vesper, and we made our own sync engine.
00:14:57 ◼ ► And the sync between devices with Apple Photos through iCloud has been really, really good for years.
00:15:12 ◼ ► And every once in a while, I'll make a change on one device and expect to see it instantly or nearly instantly on another device. And it's not.
00:15:21 ◼ ► And then I look at the other device and it's paused for system optimization or something.
00:15:26 ◼ ► And it's like I can manually make it start the sync, but I don't even have to do that anymore.
00:15:34 ◼ ► But I think basically, I think that was one of the reasons that Apple abandoned Aperture was that it was just based on a, it really would have taken an awful lot of work to move it to this new back end infrastructure.
00:15:49 ◼ ► Well, they were probably, you know, as part of that photo library migration and moving to hit up all beyond the cloud, they needed to do this huge retool of the model layer, what I would call the model layers programmer, but like the back end of this whole thing.
00:16:00 ◼ ► And so if they're going to do that, which app are they going to do it first for? Obviously the consumer app.
00:16:04 ◼ ► And then if you think about it, well, how are we going to, if we wanted to have an Aperture successor, where like evolve Aperture, we would have to fork photos and then get photos working and then fork that and then have a shared code that did that for both platforms.
00:16:21 ◼ ► And then when push comes to shove is, well, what do you prioritize? We prioritize getting the consumer one out and then it's pretty compelling to say, well, let's try to add 80, 20, 80% of the more pro features to the photos app over time that Aperture might've had, which of whether or not they did that and is then kind of left up to history.
00:16:39 ◼ ► And I know by me writing about it the last week or two, I'm hearing from people who still say they miss Aperture and I can't blame them.
00:16:47 ◼ ► And anybody who's been using computers long enough and is even vaguely a power user has at some point lost an application, probably multiple applications.
00:16:58 ◼ ► Just companies go under apps, get acquired and get lost in the acquired company or the market changes or stuff happens.
00:17:08 ◼ ► And it kind of hurts more in a certain way in the back, in a petulant, selfish way when it's a company like Apple where they didn't go bankrupt.
00:17:26 ◼ ► And yeah, I got a free ride for a while on this startup that never was going to work out and they went bankrupt.
00:17:31 ◼ ► Then it's just kind of the shape of the world, but these billion trillion dollar companies have the ability to completely change the course of any given market if they decide on their whim.
00:17:40 ◼ ► And if you're, if you're creative profession, when Aperture was truly a professional tool or just your very serious hobby from pro serious prosumer to true.
00:17:58 ◼ ► It hurts to see that Apple still is committed to the video editing market with Final Cut and the professional audio editing market with Logic.
00:18:14 ◼ ► Would you think that were there photographers that were feeling that they're like, Oh man, I just really wish Apple would enter this market.
00:18:24 ◼ ► I don't actually, I'm not very close to how like music producers feel about Logic, but like those of us that edit podcasts and Logic have this love hate relationship where it's, this is a tool that is like very good in some ways and has a few janky bugs that they don't seem to be fixing.
00:18:38 ◼ ► And like you accidentally press some hot key and then your whole interface gets screwed up.
00:18:44 ◼ ► But if some company's entire livelihood was like, if Logic was completely independent company, I don't know, we can, we can speculate what it would be like.
00:18:52 ◼ ► But I'm not sure if there was a lot of photographers that were just wishing, Oh, they were so sad that Apple didn't have other than the people who miss aperture.
00:19:12 ◼ ► And Apple at times has like, I think it, I guess it was the Mac pro, like the last Intel Mac pro, the one where they debuted the current Mac pro.
00:19:41 ◼ ► I asked at the hit, they had a nice, they'd rented out a building across the street from the San Jose convention center to demo it, the Mac pro hardware.
00:19:52 ◼ ► And I asked if I could get on one and kind of ride it around as like a little sit down scooter.
00:19:59 ◼ ► And they said, no, you can't, but that you, but in terms of the weight, it would be fine.
00:20:13 ◼ ► Although if they had said yes, I might've tried it, but they had a demo station pulled up and I don't know, remember his name, but somebody who was, they, I guess they had hired him as a consultant, but he was like a professional movie score.
00:20:28 ◼ ► I don't even know if he wrote the scores, but he had something to do with big budget, major Hollywood motion picture scores and had this ridiculous logic program or, or, or project open with just a ridiculous number of tracks and was explaining that.
00:20:47 ◼ ► Until recently, a score like this would have had to have been broken up into several projects with fewer tracks.
00:20:55 ◼ ► And then it was a lot of work and just easy to screw up to get them to emit the final score and put it all together.
00:21:04 ◼ ► And now that it can all be one project, it's just, I think you just, it's life-changing in terms of the efficiency.
00:21:11 ◼ ► And I was there, Jim Dalrymple was still writing the loop website and Jim knows music and I don't.
00:21:18 ◼ ► And I looked at Jim and he was like, yeah, this is, this is, this is really impressive.
00:21:22 ◼ ► So I, you know, I think Logic has huge fans who are super serious users, but I kind of feel like podcast editing falls through the cracks as being so simple, technically, that, that it's not really an optimized tool for that.
00:21:38 ◼ ► It's just different enough from this, like what the, this Composers for Hollywood Hans Zimmer is doing something so different than what the podcast editor is doing that really it justifies its own UX, but it's not quite a large enough market to have a tool of the polish and quality that Logic.
00:21:58 ◼ ► And obviously there, there are smaller indie teams that are building tools in that space and we're all watching them and cheering for them to overtake Logic, maybe for that use case.
00:22:18 ◼ ► And in fact, when Lightroom and Aputure were at their peak competitive, I chose Lightroom, not Aputure, even though I tend to go the Apple route because I appreciate the Apple specific UI conventions and sticking to the platform.
00:22:54 ◼ ► And it's one of those things where in the alternate universe where Apple had stuck with Aputure, Moore's law would have just helped Aputure catch up.
00:23:01 ◼ ► And certainly the shift to Apple Silicon, I'm sure by today, if they're in the alternate universe where Apple stuck with Aputure, Aputure would be, would, nobody would think it was slow.
00:23:15 ◼ ► Like I tried it and imported some photos and it's just like going from one photo to the next.
00:23:34 ◼ ► If you're doing a thousand of something, the difference in between it being a 20 millisecond lag and a 200 millisecond lag, that's only, you know, 180 milliseconds.
00:23:44 ◼ ► But if you're trying to go like next and then a keyboard shortcut, next keyboard shortcut, that kind of workflow that it just can makes it feels like a totally different experience.
00:23:53 ◼ ► And I, you know, so I used Lightroom for a couple of years, but it was overkill for my needs.
00:24:03 ◼ ► And I eventually just sort of, it was like, you know what, I'm just going to switch the photos and I'm happier for it.
00:24:10 ◼ ► And it really did coincide with me shooting the tipping point where even at things like Christmas or Thanksgiving or birthdays or vacations, like I still have standalone cameras I use, but I use them less and less.
00:24:36 ◼ ► Like you said a couple of minutes ago, that whole, okay, step one, take your SD card and import your photos to your photo library.
00:24:50 ◼ ► But I kind of feel like the dark room, photo mature model of building on the photo library.
00:25:02 ◼ ► Some listeners are laughing that God, that was so long ago, but the backstory on that, cause there's gotta be, I wonder how many readers are listeners to get it way back when on this podcast.
00:25:22 ◼ ► And I literally, until that point had only ever read their name and I already had the app installed and was super impressed that two brothers had made this app that was like a full image editor on the Mac.
00:25:50 ◼ ► Turns out they're from Lithuania, but I knew they were from somewhere central to Eastern Europe.
00:25:58 ◼ ► And so I thought, I don't know, pixel mature, maybe that's whatever language they'd speak natively.
00:26:09 ◼ ► And at least a third of the time I try to say their name without even trying, it comes out wrong.
00:26:45 ◼ ► I mean, any kind of pro photographer has like clients and projects and you don't want to have all of, you don't want to, it would be catastrophic to mix photos from one to another.
00:26:58 ◼ ► In Final Cut Pro, you don't just, it's not like iMovie where you just like drop in stuff.
00:27:15 ◼ ► What they'll do with PhotoMator, which again gets complicated to talk about because the one that they have with photo at the start of the name isn't the one that's like Photoshop.
00:27:34 ◼ ► Every time you refer to the joke, you give yourself another year of, but yeah, so the two products.
00:28:01 ◼ ► And then of course, people who are fans of one app or the other have feelings about it and reasonably so because something that they've been depending on, especially if they're professional or even as a enthusiast,
00:28:25 ◼ ► Well, they have like a specific perspective on this having worked at Apple a long time ago and then having been an Apple developer for many more years than I was in the Apple ecosystem.
00:28:38 ◼ ► So I've seen so many of these, like some of the big, obviously the big acquisitions, Beats and PA Semi where this is like a giant strategic thing for the company with a ridiculous amount of money involved.
00:28:49 ◼ ► But then at the, well, I don't know, I'm not sure exactly what the size of the Pixelmator acquisition is, but it's like probably not at the, it wasn't announced, was it?
00:29:08 ◼ ► But I think I get the impression that they were doing fine financially, you know, as a standalone company.
00:29:20 ◼ ► And that's one of the things when these acquisition stories as like end users, we just see the announcement go by.
00:29:25 ◼ ► Often, if you're not in the industry or you're not like a founder, you haven't been involved in acquisitions.
00:29:30 ◼ ► Sometimes people won't consider some of the reasons why a company might get acquired. And one of the reasons could be they're failing, right?
00:29:36 ◼ ► And then so like an ecosystem partner like Apple who benefits from Pixelmator and maybe also Photomator continuing to exist might step in a situation like that.
00:29:45 ◼ ► They might step in a situation where, I have no information about this to be clear, but they might step in a situation where someone else is going to acquire the company where, and they would rather not, it be Microsoft owning them or whatever or some goal or something, Adobe maybe.
00:30:04 ◼ ► You can imagine it being a concern that maybe Pixelmator is like, oh yeah, maybe we'll add Windows support.
00:30:13 ◼ ► You can speculate a bunch of different reasons, but like strategically, I think for Apple, the biggest, like clearest reason to acquire a product like that is to continue to develop.
00:30:22 ◼ ► It gets big enough and the cost would be high enough, it's probably to continue developing and preserve or maybe enhance its position as a differentiator for people to use a Mac rather than just to the thing that people tend to worry about is just to get the team.
00:30:35 ◼ ► It's, oh, we want to build, like you can imagine it, but this wouldn't probably happen with a product as successful as Pixelmator, but it does happen is that companies like Apple, including Apple will be like, we want to build a product.
00:30:57 ◼ ► And a way you could get that done potentially would be to acquire a company like Pixelmator and then quietly wind down the product they were working on and then ramp them up with, they would be the sort of team who could execute well on Aperture reborn.
00:31:13 ◼ ► Like maybe rebrand, but like to throw away the code base of this working app that has a loyal fan base, then obviously that would upset so many people.
00:31:20 ◼ ► So it seems like keeping that product rolling and improving has to be the current mindset.
00:31:26 ◼ ► Obviously acquisitions tend to go in all sort of different ways where you have this great vision when there's acquisition and then a huge kind of like embarrassingly high percentage of acquisitions end up failing for various reasons, including we tried to merge our code bases and then we turned out there's a whole bunch of demons inside or the person who championed this ended up leaving the company the month after the acquisition closed.
00:31:50 ◼ ► Yeah, I think so too, especially Pixelmator, just because there aren't that many apps that do that.
00:31:58 ◼ ► The other one, I'll just throw it out there just for anybody who hasn't looked at it and tried is Acorn from Flying Meat Software.
00:32:06 ◼ ► And our mutual friend Gus Mueller, which is one of those, oh man, I in theory, I could do it or someone else could do it, but like a just compare and contrast Acorn versus Pixelmator would be such a fascinating.
00:32:22 ◼ ► You could do like a whole semester long user interface study just based on those two apps and the differences between them and the way that they could both be good or great and both sort of the same basic idea.
00:32:42 ◼ ► Right, and both from the user benefit perspective of, oh, you're going to be you're going to find your way around this app because it uses the idioms of the Mac platform, but also how can an seemingly incredibly small team make an app that does so much?
00:32:58 ◼ ► It's because both apps base so much of what they do on the, I guess, mostly core image, but the frameworks that Apple provides that don't really have a user interface, but give you all sorts of things like file format support and all sorts of stuff.
00:33:15 ◼ ► And then I don't mean to downplay how much custom processing code because I know especially Acorn because I'm friends with Gus, just how much work it's not.
00:33:27 ◼ ► That's not true at all, but I don't think it's so inextricably tied to it that it would be difficult.
00:33:39 ◼ ► I think it's probably almost intractable to, or I don't know, but that's a longstanding strategy in the Mac.
00:33:47 ◼ ► What I think of is the indie Mac world is like you as a team or as a product company, you say we're going to be the Mac asked version of this thing or for iOS, we're going to have an iOS app and it's going to be really going into the platform conventions.
00:34:02 ◼ ► And obviously those companies have a weird relationship with Apple because obviously Apple celebrates them because they're helping Apple make this case that our APIs are good.
00:34:11 ◼ ► They tend to be platform advocates, but then also they have this weird tension where those companies are so dependent on Apple that if Apple has a lacking in one of their frameworks or they maybe deprioritize something that that team depends on, it's like an existential risk to your company.
00:34:33 ◼ ► And so obviously those parties are motivated to work things out and stay on the same page.
00:34:39 ◼ ► But often these teams are so small that sometimes they get buffeted around a little bit.
00:34:44 ◼ ► It's a very different set of trade-offs than the, oh, we'll just do it all ourselves and make it cross-platform, but then it all feels weird and foreign or electronic on every platform.
00:34:56 ◼ ► And obviously every product team has all these certain economic trade-offs that can make that path, like that Electron path, which obviously is problematic for Apple because if there's nothing special about the Mac app, then that's not good for Apple.
00:35:11 ◼ ► But that Electron path can be extremely beneficial for a team where they're actually providing some service where it's okay.
00:35:17 ◼ ► We're an online HR platform where it's like, oh, you can set up your payroll and you can do all these things and help your team and get feedback or whatever.
00:35:26 ◼ ► And this, are they going to actually build a really nice Mac app using all the latest Swift UI or whatever, and also a Windows app and also an Android app and also an iOS app there?
00:35:35 ◼ ► No, the alternative would just not be to have an app at all, which in some cases, we could talk about the Clod app if you want, if you've tried that.
00:35:43 ◼ ► And it's weird because Apple, for all the Pro tools and the other apps that they've made over the years, and like the iWork suite is still a thriving suite that went through its own similar iPhoto to Photos transition around the same time, where they kind of hit the reset button to get the file formats and the user interfaces to in sync between the Mac and iOS version.
00:36:13 ◼ ► Which was painful for a couple years, but I think they've successfully pulled it off and I don't really do a lot of slide presentations, so I don't use Keynote.
00:36:25 ◼ ► One day I'll tell a bit of the, I don't know if the statute of limitations has fully expired yet, but one day I'll write this to some of the stories of, because I was on the team, the iWork team at that time when the transition from the classic iWork, which like iWork 09, which is how long ago it is.
00:36:40 ◼ ► It's like maybe the statute of limitations has gone by by then, but iWork 09, too, it was like, let's bring iWork to the iPad and I won't say anything like dramatic, or there wasn't, it's not like there was dramatic, dirty laundry or whatever, but it was a really interesting challenge where it's like, okay, we have this thing that's always been a Mac app and it's made these assumptions and it's, the code base had been around for a long time by then.
00:36:58 ◼ ► And it's, can we get this to work on this tablet that has a microscopic amount of memory and doesn't have any of the, because it was iOS 3 at the time, or iPhone OS 3, right? And they were going to bring it to the iPad OS.
00:37:11 ◼ ► And so iPad OS 3.2 offered a few new affordances, but it didn't have even like background, like you didn't have, it was iOS 4 that brought the background behavior.
00:37:21 ◼ ► And so it was like when the, if you wanted to build an app at that time that was as complicated and powerful, when you hit the home button, you would get a timer.
00:37:30 ◼ ► It was something like a second and a half or two and a half seconds, and you had to finish writing out to disk everything that you had done.
00:37:40 ◼ ► And obviously being Apple, like you could have maybe tried to be like, oh, well, we're going to give special entitlements to the iWork apps to run longer.
00:37:47 ◼ ► But the thing is, even if you did say, oh, well, the apps have longer to save to disk, then the whole OS is bogged down with you slowly doing whatever it is you're serializing out or something like that.
00:37:57 ◼ ► So that was like a really interesting engineering challenge that the team rose to was like, hey, we need to like get the under layers of this productivity software that's desktop class into this small memory and this small time window basically to persist out, which was pretty interesting.
00:38:26 ◼ ► I think it was one of the first things when we first became friends years ago, I knew that because it was fresher.
00:38:37 ◼ ► But, you know, when iOS first came out and it's instead of porting AppKit over, which I don't think could have worked because of AppKit makes so many assumptions about the overall environment.
00:38:51 ◼ ► But UIKit was meant to be familiar to AppKit programmers, but with some modernizations in very broad strokes, some modernizations and some concessions to the different application runtime environment of the very constrained early phones and iPads.
00:39:15 ◼ ► But there's all sorts of other stuff in AppKit that just didn't make sense to port over.
00:39:21 ◼ ► But if the iWork apps were depending on it, well, then how did they run on iPad, right?
00:39:25 ◼ ► And for me personally, one of them was that Pages supports really rich typography features, like not just picking font, but when you go to the font panel and then open the sub panel for typography.
00:39:41 ◼ ► Yep, alternate glyphs, you can get proper small caps if the open type font you're using supports them.
00:39:57 ◼ ► Like you can build a really rich typographic app in AppKit without writing the code to do all this stuff because it's right there in AppKit.
00:40:09 ◼ ► But it's not in UIKit, so how do you share a document where you might do these fancy typographic things on the Mac and then you open them on the iPad?
00:40:50 ◼ ► And it's not that there was no code shared or anything, but it was a big engineering lift and they pulled it off.
00:40:55 ◼ ► And again, it helped being a first-party app where if you really needed to get some code out of AppKit and just put it in the app, you could do it in a way that a third-party app doesn't even have the source code.
00:41:14 ◼ ► And I think they really do the, "Hey, you can just open these same documents on your phone, your iPad, your Mac, and save changes just show up everywhere."
00:41:26 ◼ ► So I think that the notion that Apple doesn't care about apps like that, I don't think it holds water.
00:41:35 ◼ ► I think there's a sort of knee-jerk fear that Apple only cares about what makes the most amount of money, stuff.
00:41:42 ◼ ► Well, I think that the fear, you can kind of go with one layer of more nuance where Apple does care about these products.
00:42:06 ◼ ► But from the outside, obviously, they purposely project a uniform, spherical, polished image.
00:42:12 ◼ ► But Apple, in abstract, cares about these products, despite the fact that they are not the highest money-making things, from a strategic perspective.
00:42:24 ◼ ► And they use, I don't know if they use Final Cut Pro, they must use Final Cut Pro for their own internal video products.
00:42:33 ◼ ► But the more, I think, bigger way that they care about these products is that it's a systemic risk to their platforms.
00:42:41 ◼ ► If, I mean, it goes all the way back, I know you like to go back on the history of Steve Jobs, like, getting held over a barrel for, there's no,
00:42:48 ◼ ► there were like, Microsoft had complete control over Apple because you had, there was no alternative at that time, way back to Microsoft Office.
00:42:56 ◼ ► So Microsoft Office kind of had complete control of if they wanted to kill Apple, which was bad for them for antitrust reasons.
00:43:01 ◼ ► But if they wanted to kill Apple, they probably could have done that, and as could have Adobe, because you, they, Apple didn't have any backup.
00:43:06 ◼ ► Like, these products were so important to having a viable platform at that time that you had, it was just wise for Apple,
00:43:15 ◼ ► and strategically a good thing for them to do, to say, well, let's not have all our eggs in this basket.
00:43:26 ◼ ► We're talking about iWork, and we're talking about Pixelmator, we're talking about, effectively, things in Microsoft Office and Adobe's domain,
00:43:35 ◼ ► And so, definitely Apple cares about these things, but it's, I think, a little bit different,
00:43:39 ◼ ► and to maybe give a little bit of credit to the people who are, who get a little nervous when these acquisitions happen,
00:43:44 ◼ ► is that sometimes a company caring about a product in order to protect themselves from potentially bad action or undesirable action
00:43:52 ◼ ► from other competitors who neglect their platform, the type of caring that that results in is a little bit different
00:44:05 ◼ ► And so, if they can make that product have 10% more market share, that's like the entire focus of the CEO of that company,
00:44:12 ◼ ► versus the really fundamental, most important thing for Apple is to make sure that their whole platform strategy is working,
00:44:19 ◼ ► and sometimes it can be slightly less critical that maybe podcasting features are really good in Logic,
00:44:25 ◼ ► whereas if Logic was a completely separate company, maybe there would be, I'm just very mis-speccing, I don't know if that drives that,
00:44:35 ◼ ► Right, but they've never had, or I shouldn't say never, because I even called back to it,
00:44:40 ◼ ► I don't think they've had anything since MacPaint that counts as a bitmap image editor.
00:45:04 ◼ ► There was a third-party app before Photoshop, I think the first version came before Photoshop,
00:45:11 ◼ ► before Photoshop just sort of was like, "Ah, throw all these other ones away, Photoshop kills them all."
00:45:41 ◼ ► the things we think of it just like totally just like plain, obvious checkbox features that wouldn't even occur to you,
00:45:50 ◼ ► There was an app that's like, if you buy this package software, you can print vertically as well as default horizontally across the page.
00:46:07 ◼ ► But I forget the name of that app, but I think that sold, it wasn't really a Mac thing.
00:46:14 ◼ ► But it was like a, no, it was like a DOS thing for like Lotus that you could print your spreadsheet sideways.
00:46:33 ◼ ► But because spreadsheets are often so wide and it most many printers of the time were dot matrix printers,
00:46:52 ◼ ► It was like, well, then of course, every spreadsheet added the ability to print in portrait.
00:46:57 ◼ ► And it's like, whoa, well, is that unfair to the company that had an entire business selling a three or four hundred dollar per seat floppy disk software program just to print sideways?
00:47:13 ◼ ► And that's part of why I enjoy looking back on those because they're so obvious to us now.
00:47:22 ◼ ► And some of them, a lot of them, are building the equivalent of the print the spreadsheet sideways.
00:47:31 ◼ ► There's like multiple startups whose main focus was, well, if you're a programmer and you want to use this AI model, then you often might want it to output JSON.
00:47:41 ◼ ► And so we'll, our whole product, make sure that the model always outputs JSON, even though sometimes it won't and it's inconsistent about that.
00:47:48 ◼ ► And like, obviously, when you're a founder, if you're thoughtful, that's not going to be the end of your whole company and that's the only feature you ever remember to do.
00:47:55 ◼ ► But there's a whole bunch of companies that that was like how they initially marketed their thing.
00:48:04 ◼ ► And now it's obviously every model that's just, we assume, of course, you can enable that as a developer.
00:48:31 ◼ ► But you're right, though, that whenever there is a breakthrough, there is this explosion in startup ideas.
00:48:40 ◼ ► And we're seeing that with AI right now where, and most of them aren't going to make it, no doubt.
00:48:46 ◼ ► Many of the ideas that they have, like you said, oh, just if what if we could just guarantee that everything's in JSON all the time?
00:48:53 ◼ ► At the moment, that is actually has use cases where there's somebody out there who hears that and is like, oh, my God, that's what we need.
00:49:00 ◼ ► We're having such a hard time because that's what we really want is everything in JSON.
00:49:13 ◼ ► Well, 18 months ago, almost every app that a developer was making on top of AI would have one or more all caps only respond in valid JSON.
00:49:23 ◼ ► Because they were trained so that they could do it in theory in the understood JSON, kind of.
00:49:29 ◼ ► But there's a difference in between it's statistically likely to usually have valid JSON and it 100% always you can just depend on that it's going to do that.
00:49:40 ◼ ► That was like a table state, like almost everyone who's building on that needed some way when we were building these pieces of software to be able to reason about, okay, well, what's coming out of this and how do I display it in my UI?
00:50:05 ◼ ► I think it needs to be loud enough, though, that it feels like at first you had it off in the distance and it's almost like it's background noise.
00:50:27 ◼ ► If you're building B2B, software as a service, chances are you probably want to sell to enterprises at some point, and the challenge is that selling to enterprises requires you to support complex features like SAML, S-A-M-L authentication, SCIM, S-C-I-M provisioning, audit logs, and fine grained authorization that require months of development time of your team and ongoing maintenance.
00:50:49 ◼ ► Work OS solves this with APIs that are flexible and easy to use. It makes adding enterprise features to your app a quick and painless process.
00:50:59 ◼ ► They also have a user management solution called AuthKit that is free up to one million monthly active users.
00:51:07 ◼ ► That means you don't have to pay a single cent to use features like social auth, MFA, RBAC, all these acronyms.
00:51:14 ◼ ► I don't know what they mean, but they probably make sense if you're building these apps.
00:51:17 ◼ ► Bot detection, you get all of this stuff that Work OS offers for up to one million monthly active users free.
00:51:26 ◼ ► And just this last week, they had what they called Launch Week, where they announced seven new features amongst them.
00:51:39 ◼ ► PassKeys, I've been writing about that at Daring Fireball, but now they've added PassKey support. Radar as your SAS, software as a service app, grows in popularity.
00:51:49 ◼ ► At some point, you're going to have to address things like bot signups, free tier abuse, credential stuffing, and other illegitimate and sometimes dangerous activity.
00:52:01 ◼ ► Fine-grained authorization, Work OS FGA is a full managed fine-grained authorization service based on Google's Zanzibar.
00:52:11 ◼ ► Widgets, easy to use components that supply the front end and back end functionality for common enterprise app workflows.
00:52:19 ◼ ► Actions, that's what they call customizable user registration and authentication logic within AuthKit entitlements.
00:52:27 ◼ ► And Next.js B2B starter kit, fast track your software as a service app from zero to one.
00:52:35 ◼ ► It's a curated stack built with convex, Next.js, radix, stripe, and of course, AuthKit.
00:52:49 ◼ ► That's it, no special slug from the show, but if maybe there's like a poll or something after you sign up, you can tell them it came from the show.
00:52:57 ◼ ► You always joke, Jon, that you don't know what SAML and SCIMR are. Do you want to know what they are?
00:53:19 ◼ ► SaaS? Yeah, I have heard SaaS, but see that I worry about other people like me who may not know what it means.
00:53:26 ◼ ► But there's also SaaS, which is like a CSS framework, but I think the target demographic knows what you mean.
00:53:31 ◼ ► Anyone who's going to buy WorkOS isn't going to be like, wait, SaaS, what do you mean by that?
00:53:55 ◼ ► I've been building software, but I think for this audience, Apple intelligence is kind of like the core motivator for a lot of the conversation around it.
00:54:06 ◼ ► And so I think there's two high level lenses that I think are interesting, or at least examples of lenses that are interesting to think about Apple intelligence.
00:54:15 ◼ ► One is, is it useful yet? And the other is, how does it compare with the state of the art, the other offerings?
00:54:25 ◼ ► And those are two very different questions. When I first started using it, installing the 18.2 beta and playing around with this stuff,
00:54:31 ◼ ► at first I would have instinctual reactions to these because I'm over exposed to the state of the art.
00:54:37 ◼ ► A day ago, something came out on Sunday and I'm still kind of itching. I feel behind because I haven't tried the latest cursor updates, whatever, for new AI features.
00:54:44 ◼ ► But most people are not like that. Most people are not aware of what state of the art is or wouldn't even think in terms of state of the art.
00:54:53 ◼ ► And if it's not useful to them and the state of the art thing only starts mattering when it's competitive.
00:54:58 ◼ ► Like if Android has a feature that's better and then a friend says, "Hey, my Android phone does this better."
00:55:05 ◼ ► That matters more than if some insider AI people happen to know that, oh, if you wire in a different model or OpenAI natively offers this functionality that maybe people aren't familiar with.
00:55:21 ◼ ► Yeah. So where do you think, well, how many models out there should people even care about?
00:55:28 ◼ ► If you were like a listener of this show and you're like, I don't know, and you wanted to do your own evaluation of where Apple intelligence stands versus the competition.
00:55:42 ◼ ► ChatGPT obviously is the one that everybody talks about. It's like the Kleenex brand of the industry at the moment.
00:55:48 ◼ ► So there's a distinction that's worth making. And I think when you're trying to think about this stuff that's useful and it's very easy to miss is the distinction in between a model and the product that's built on top of the model.
00:56:02 ◼ ► So you go in to ChatGPT, it's going to serve you the current free, I mean, if you're a free user or if you're paid, then you're paid.
00:56:09 ◼ ► But the models that OpenAI has developed for use on that service, which currently the frontier models or in the jargon in the AI industry, they'll call the frontier model is like the biggest and best model available.
00:56:22 ◼ ► So their frontier model is 4.0 for general usage, GPT 4.0. And then there's the 0.1 model, which is not GPT 0.1, which is another like one of those jargony things because OpenAI is just as bad at naming stuff as every other AI company.
00:56:38 ◼ ► But I think like a little inside baseball on that, I think OpenAI is actually backing away from GPT as a term because they're struggling to trademark it.
00:56:46 ◼ ► So they don't want a GPT 5, GPT 6 and everything is based on GPT, but then they don't own a trademark for GPT.
00:56:51 ◼ ► And you see other startups trying to call themselves GPT something else. And then if OpenAI can't own that phrase, then that's a problem for them.
00:56:58 ◼ ► So I think that might be a motivator why they're 0.1 is their like future generation model.
00:57:02 ◼ ► Interesting. So it's not even about persnickety precision to the term of art. It's just as simple as trademarking.
00:57:12 ◼ ► I've heard someone push one of their people on like, oh, why is it called 0.1 and not GPT 5?
00:57:18 ◼ ► And they're like, well, you know, 0 stands for OpenAI and we think this is a 0.1 is a new paradigm and so we thought it needed a new.
00:57:27 ◼ ► And so my speculation is that the trademark they're moving and like they recently bought chats.com.
00:57:32 ◼ ► So they don't just have chat GPT.com, but chat.com, which I'm sure cost a ridiculous amount of money, but they have a ridiculous amount of money.
00:57:39 ◼ ► So just for very briefly, 0.1 is I would say for most people, 0.1 is like not going to be the reach for model.
00:57:46 ◼ ► But it's interesting because it is in OpenAI has definitely been hyping it as the next generation of how they want to make these models better.
00:57:54 ◼ ► So GPT 4 and GPT 4.0, which is what most people, the most common model people interacted with via chat GPT.
00:58:10 ◼ ► Whereas what 0.1 and future models of that series are starting to do is conditionally spend some time, quote unquote, thinking about the response before it responds.
00:58:20 ◼ ► It's still kind of janky. It's like a pre, they call it a 0.1 preview. It's not really fully baked yet.
00:58:24 ◼ ► But that's the like really if the state of the art, like if you're trying to solve a physics question and you're like, which AI LLM that exists is most likely to answer that question successfully, then that's probably the OpenAI 0.1 model.
00:58:42 ◼ ► But something that's really easy to miss and you see it when you interact with chat GPT within Apple intelligence is that there's a difference in between in talking to GPT 4.0, which is the model, and using chat GPT.
00:59:08 ◼ ► But on top of all that, they build all this extra functionality that improves over time.
00:59:21 ◼ ► It's application features that they have added to the chat GPT app that helps GPT 4.0 give you better answers.
00:59:29 ◼ ► And they also have the ability for it to generate images and the ability for it to interpret images and the ability for you to upload code and the ability to,
00:59:36 ◼ ► there's a canvas feature in chat GPT where you can like work on something and then text will appear on the side and you can iterate it back and forth.
00:59:42 ◼ ► A lot of those are like extra affordances that are in the product, but that isn't necessarily part of the model, if that makes sense.
00:59:52 ◼ ► But it also, and we'll get to it, I think, it also further confuses the line of who's solving what where when you ask Apple intelligence question and it bubbles up to chat GPT.
01:00:05 ◼ ► Is it coming out of the model or is chat GPT itself then doing the action of going out and searching the web to get the answer?
01:00:13 ◼ ► Have you noticed, in my experience so far of using the beta chat GPT integration, is it seems like, even though it's calling it chat GPT,
01:00:21 ◼ ► it seems like Apple has mostly implemented using the GPT 4.0 APIs and not all of the extra search, web search and citations and image generation.
01:00:32 ◼ ► I don't think if you ask, hey, Dingus, can you have chat GPT make me an image of a sloth? Like, I don't think any of that stuff is wired in,
01:00:41 ◼ ► which is the same experience you would get as a developer if you were like, oh, I'm going to make a chat with GPT app as a web app or an iPhone app.
01:00:49 ◼ ► And then you just took the APIs that OpenAI gives you to as a developer to make something, you wouldn't get any of that extra stuff.
01:00:59 ◼ ► It's all changing so rapidly that all of this is going to be just a snapshot in time from late November 2024.
01:01:07 ◼ ► Yeah, we should say this is November 25, 2024, and we should get this edited out as fast as possible so that when any of these various models,
01:01:15 ◼ ► I might say, oh, O1 is the best, Preview is the best advanced model now, but in three days.
01:01:21 ◼ ► And, you know, iOS 18.2 and the various other, the Mac OS 15.2, God, I wish those major version numbers were in sync,
01:01:30 ◼ ► have only been in beta for like six weeks or something, but they've already gotten better.
01:01:34 ◼ ► And a friend of the show, Paul Kefasis, on his One Foot Tsunami blog had a thing when he first upgraded to 18.1.
01:01:44 ◼ ► So he just got regular Apple intelligence out of beta, was like, couldn't be bothered to try it in beta.
01:01:50 ◼ ► Like most people, and asked who was playing at the TD Bank Center in Boston, the big basketball concert arena.
01:01:57 ◼ ► And Siri was, I forget what happened initially, it was like Siri gave like a horribly wrong answer with Apple intelligence, like not even close.
01:02:06 ◼ ► And ChatGPT would be like, let me look on the web and let me look on the TD Bank website and gave like the correct answer for whether,
01:02:17 ◼ ► But that's already been fixed, you can ask Apple intelligence questions like that and it'll somehow search the web.
01:02:27 ◼ ► Does it though? Because the thing, I don't know for sure, in like a few weeks, but the sense that I get of the approach that they've been taking so far with Siri,
01:02:35 ◼ ► I know I'm supposed to say Dingus, with Dingus, with this technology is that they are not yet giving the engine,
01:02:51 ◼ ► So like the feature that allows you to ask, hey Dingus, what is the score for the NHL game?
01:02:56 ◼ ► It's doing that from a database, it's not going out to the web and then doing a search and returning the search back to the agent,
01:03:05 ◼ ► But so you have this kind of whack-a-mole problem where when Apple hears, oh, well people are trying to do this thing,
01:03:11 ◼ ► they're asking Siri about this thing and they're getting bad results and this is a common use case,
01:03:16 ◼ ► let's improve the recognition of the system so that it knows, oh, I should look at this source for the location of the next NFL game.
01:03:24 ◼ ► You can settle those really common requests, but with a general, which I assume eventually they'll do,
01:03:31 ◼ ► with a general setup where it can just search the web on your behalf and then it can reason about those answers and then give you the answer,
01:03:38 ◼ ► you could ask kind of an unbounded question where you could say, oh, well, what is the latest episode of the talk show?
01:03:51 ◼ ► But there's this kind of infamous thing about Google where they say like something like 20% of Google searches have never been searched before.
01:03:57 ◼ ► And so, and that's true just every day because people are always have more and more niche things that they want to know.
01:04:03 ◼ ► And so the solution of creating these databases and stuff and fixing the individual problem, yeah, okay, you fixed Paul's problem.
01:04:10 ◼ ► But then I asked something, well, there's also an underlying problem with Siri that it's been true forever,
01:04:15 ◼ ► and it continues to very much be true with the 18.2 beta and probably will continue at least until the rumored iOS 19 Transformer AI-based version of the product,
01:04:25 ◼ ► is that it's just inconsistent, which is also so with ChatGPT for some of the same reasons.
01:04:30 ◼ ► But like I was trying to test on my way today to record this, I was trying to test again something that I had test a couple of weeks ago,
01:04:37 ◼ ► which I asked it to contrast, verbally I asked, to contrast the words tacit and implied.
01:04:44 ◼ ► Because on a dithering episode, Ben was like, oh, well, tacitly this tacitly this, I'm like, what's the difference between tacit and implied?
01:04:51 ◼ ► And I tried that a couple weeks ago. And it like sort of halfway went to ChatGPT, and then it stumbled and it wouldn't read me the results.
01:04:59 ◼ ► Today, I did that and it misheard me and it read me ChatGPT results for what it had misheard.
01:05:06 ◼ ► So I tried it again. And then the second time it gave me, here's what I found on the web for can you contrast the words tacit and implied.
01:05:17 ◼ ► I mean, you can chalk some of that up to just the growing pains of they're working on 20 different AI features all simultaneously and trying to bring them to the OS.
01:05:24 ◼ ► But also that's kind of been the behavior of this sort of voice interface on iOS for 10 years.
01:05:32 ◼ ► Those sort of experiences do happen when you push it outside of, no one's ever asked that question before.
01:05:36 ◼ ► I'll never ask that question again. You need a sort of general more ChatGPT-like system in order to handle requests like that.
01:05:43 ◼ ► Right. Basically, the first thing you want people to take away is that there is a difference between the underlying model and things that are built on top of it, which includes the branded first-party chat apps that the makers of the models tend to make.
01:06:00 ◼ ► Yes. And so if we go to the Clod app, which arguably Clod being Anthropic is the company behind it.
01:06:07 ◼ ► Of course, each one has a name of the company, a name of the model, and then versions of the model. So there's like names everywhere.
01:06:13 ◼ ► But Anthropics, Clod 3.5, the latest version of that, which is out in the last few weeks, is like argued.
01:06:20 ◼ ► A lot of people seem to be arguing that are spending a lot of time with these models that for a lot of things, it's maybe the best.
01:06:30 ◼ ► And on top of that model, which we can use as developers to build stuff, if you use like the cursor IDE, and I'm sure we'll get into programming tools maybe a little bit today.
01:06:40 ◼ ► But if you are interacting with the Clod website, you're interacting with that specific model and all the features the Clod team has put on top of in their equivalent of ChatGPT, which has some features that ChatGPT doesn't have.
01:06:53 ◼ ► And it lacks some of the features that ChatGPT does have, and then they're sort of playing cat and mouse on the product level as well as on the model level.
01:07:02 ◼ ► Whereas if you're looking at a company like Apple who is using these models to potentially build products, they are mostly consuming the sort of the AI part, the model part, and not all the stuff on top.
01:07:12 ◼ ► And I get the feeling that there is a parallel to the earlier era, which is still going, but like when you're building an app for iOS, or you're building an app for Windows, there is a difference between being a third party developer and a first party developer because the first party, like you, we were just talking about in the previous segment,
01:07:38 ◼ ► where if Apple really wanted to special case numbers on iPadOS in 2010 to give it more time to serialize the data to disk just because it got interrupted and was put in the background, they could do it and others couldn't.
01:07:54 ◼ ► ChatGPT, the app, can do things with OpenAI's model that you can't necessarily do from the outside through the API, or at the very least, even if it's the same thing, they don't have to worry about paying for the tokens.
01:08:09 ◼ ► Yeah, and that happens all the time, so it's very common that when a new user-facing functionality, like when OpenAI is, oh, okay, a good example is just in the last week or two, Anthropic Cloud released computer use, which is sort of, so maybe if people haven't seen this, this is like fascinating, where you can say, hey, just use a computer and it will click around, it'll type in fields, it'll scroll, and you just, you ask it to do something, oh, please fill out this form for me on a website, and it will attempt to do something.
01:08:38 ◼ ► It's pretty beta, a little janky, and obviously has huge security for all the implications. They strongly encourage you, start up a little sandboxed virtual machine that doesn't have any of your actual information on it and let it play around with that and watch it, what it's doing.
01:08:52 ◼ ► But it's like this glimmer of a future-aware, which, and product, there's entire startups that are very involved in just doing this, and they're getting more advanced results than what Claude's doing.
01:09:03 ◼ ► But Claude, my point with all that is that they launched this computer use thing a few days ago, simultaneous with a model update that was tuned with examples so that it was better at performing that task than the model that had been available to everybody else the day before, basically.
01:09:20 ◼ ► And it is kind of funny where you just sort of, like a modern desktop computer is a very powerful device but that needs a human being to drive it, which limits how fast certain things can go, because you actually have to do the typing.
01:09:36 ◼ ► And if you could have a robot doing it for you, the robot on its own can do more by driving an existing non-AI Windows or Mac PC than it could on its own, right, and can do it faster than a human being can.
01:09:55 ◼ ► Remember, was it Star Trek IV, where Scotty sits and they travel back to the '80s time and Scotty sits at a Macintosh and everybody remembers the joke where he's talking to the mouse.
01:10:09 ◼ ► But then once he's like, oh, okay, and it doesn't really make sense. The implication is that somebody like a smart engineer from the 23rd century, of course, finds a 1987 Macintosh trivial to use as opposed to archaic and cryptic, which is how it would be.
01:10:30 ◼ ► But then all of a sudden he just like starts typing and clicking real fast and boom, he's got like the molecule from…
01:10:37 ◼ ► But that's sort of what the near future vision of like having Claude drive your computer for you is, is like being able to sit Scotty in front of your computer and boom, do something much faster than you could.
01:10:52 ◼ ► Certainly that's the eventual, but the thing that's closer is that you can have it do mundane, like to your point, mundane, boring stuff that you would lose attention span on that you just don't want to do. That's not a super high use for your time. Even like just like a really, really dumb thing is like running a startup, there's all these administrative tasks where it's like in order to do my monthly update for my finances, I need to do a sync of my bank feeds.
01:11:16 ◼ ► But I have to initialize that and then once it's initialized, it takes like 10 minutes and then for that 10 minutes, you're just sitting there because I don't know, legacy systems.
01:11:22 ◼ ► And then eventually the data comes in and then there's a couple simple tasks I want to do. But I can in theory have Claude wait for that and then once and just watch.
01:11:30 ◼ ► And then once it's done, it does the couple simple tasks that I otherwise forget. I switched to another browser tab or I got interrupted by a Slack message or something.
01:11:36 ◼ ► And so the starting point, and this is true for almost all of these tools, is that the things that are the first stuff that becomes easy for them to do is the stuff that you would have an intern.
01:11:46 ◼ ► Like imagine just the most diligent intern that never says no and they sometimes make mistakes, but they're incredibly well read and they will work around the clock.
01:12:02 ◼ ► And so this is the first low-hanging fruit for these things is what those interns can do.
01:12:07 ◼ ► And to some degree, that's not specific to this. The frontier models like Claude, the most advanced models, that goes all the way down to the Apple intelligence features where they have a much smaller, less powerful model than what what Cloud is.
01:12:20 ◼ ► But there's still some of these tasks where in theory you could have an intern transcribing summaries for all of your WhatsApp group chats.
01:12:27 ◼ ► And they would probably do that pretty okay, even though they're maybe not the PhD level or whatever, but it's a relatively straightforward task.
01:12:34 ◼ ► All right, so there's Anthropix, Claude, and then you've listed LAMA 3.2. Now that's META's-
01:12:40 ◼ ► Yeah, so I would say the two other LAMA 3.2 are, I mean, 3.2 is the current version, but like LAMA's large models, so like they have what they call a 405 billion parameter model, which is a way of saying very large, like particularly large model, which is open source.
01:12:57 ◼ ► So that's the really interesting thing about what META has been doing is they're releasing these open source models and they are continuing to push the envelope.
01:13:04 ◼ ► I'm not just happy with what previous generations of open source models had been like 6, 12, 18 months ago was like, well, there's this open source model and it's not as good as the other ones, but it's like way smaller and obviously not as well crafted.
01:13:15 ◼ ► But you know, it's open source and you need to say anyone or there's arguments about what the definition of open source, it's open weights, or that is actual code used to produce the model open?
01:13:24 ◼ ► No, but like you can download it and use it as long as you are not Apple, Google, or Microsoft, I think is the way their license is written, which is a bit of a musing will turn there.
01:13:35 ◼ ► And so that's one of the other sort of frontier models. It's not thought to be quite as good as the Anthropic or CluD, but if Anthropic and OpenAI disappeared tomorrow, there would suddenly just be huge demand to use this model because it is in the same ballpark.
01:13:50 ◼ ► And then the final one is the Google Gemini models, which there's fans of for sure, and there's certain few tasks that they're very good at.
01:13:58 ◼ ► If you're a developer and you need to put a million tokens of just huge amounts of volume into one API call, then the Google models can do that.
01:14:08 ◼ ► I think the kind of vibe among developers is that the Google models are for most tasks not quite up to the same level as those other models, but those are kind of the four heavyweights, I think, in the minds.
01:14:20 ◼ ► And then there's dozens, hundreds of other either completely separate models where they have a really specialized thing that they do well or fine-tuned variants.
01:14:34 ◼ ► But I would say if you're an enthusiast or curious about this stuff and you want to kind of follow the horse race, those would be the four to care about.
01:14:41 ◼ ► And if Apple wanted to like integrate other models in, those would be the ones that they're probably calling.
01:14:47 ◼ ► Right. And that it wouldn't make much sense unless, but it's probably not the case that there is some secretive dark horse out there who's going to decloak and have a competing model that will join.
01:15:02 ◼ ► Well, there are absolutely secretive dark horses out there that are telling VCs that they're going to decloak and dethrone one of these people.
01:15:10 ◼ ► We just need a few more billion dollars and we will just blow everyone's minds in the style of these.
01:15:16 ◼ ► You know, there's always some startups out there that are doing a humane thing where it's hard to tell from the outside is this going to be the next big thing or not.
01:15:24 ◼ ► But I think the smart money is that probably we're not going to have someone come out of nowhere and dethrone all these people.
01:15:32 ◼ ► But one of the things that makes this moment different than previous, I've called back and I have some other ideas that call back to similar explosions or new opportunities in the computing industry.
01:15:46 ◼ ► But one thing that's different here is that we just need billions of dollars, right? And even taking inflation out of the equation, that just was never the case for startups before.
01:16:00 ◼ ► I mean, and hardware is always a little different than software, but I can't really think of a software startup idea that required billions of dollars in previous times.
01:16:19 ◼ ► Well, we have the mythical person month problem where you can have an infinite amount of money, but you'd be hiring more and more and more engineers isn't going to make you go faster past a pretty small team size.
01:16:32 ◼ ► But if you think about, so certainly in software, yes, but then there's some argument that are these AI startup software? Well, I mean, the end artifact they're creating is actually just a kind of a single giant data file that then can be used.
01:16:47 ◼ ► Right? It really the kind of nut of all what these companies are making is software in that way.
01:16:57 ◼ ► And so there is a hardware component. And so Sam Altman, the founder of OpenAI, which you sort of have to take everything he takes with a grain of salt, because obviously he has a huge vested interest in shaping how we think about this stuff.
01:17:08 ◼ ► But he likes to when people say, oh, it's like the internet or it's like this invention or that invention.
01:17:14 ◼ ► He will talk about as being like the transistor, where when the transistor was invented, then they obviously there's all this VC and like Silicon Valley is named after it.
01:17:25 ◼ ► And it's not as much the mobile revolution, right? If you try to map what we did in 2008 to build all our businesses and the curves and the paths, it's quite different because of it's an enabling technology.
01:17:36 ◼ ► I think in his argument, it's enabling technology that we all start to eventually take for granted underneath everything.
01:17:48 ◼ ► And it is also, I guess, one of the main knocks against this whole thing are the environmental costs of training these models.
01:17:57 ◼ ► That it is extremely mind bogglingly computationally expensive to train these large models.
01:18:05 ◼ ► If you're a small startup and you're trying to convince VCs to invest in that you're going to be the next Claude or something like that.
01:18:18 ◼ ► And it's not just expensive, it's rare to it. There's only limited number of server farms that could do it.
01:18:24 ◼ ► Well, obviously there's a compounding problem where there's way more demand for training hardware from Nvidia than their supply.
01:18:31 ◼ ► And therefore, obviously that gives Nvidia pricing power, which means that if you want to raise VC to be yet another company that's competing for that same limited pool, you're driving the price up more.
01:18:40 ◼ ► And eventually, I think that's why there's been a big downtick over the last year in interest from VCs to be like, "Yeah, okay, sure.
01:18:52 ◼ ► And you see instead doubling down from to some degree VCs, but now at the scale that they're at, increasingly Amazon just had a huge investment in the Anthropic.
01:19:00 ◼ ► Because obviously Microsoft and OpenAI have relationship, Anthropic and Amazon, there's sort of like a natural pairing where Amazon has the thing that Anthropic needs, which is a whole bunch of infrastructure.
01:19:12 ◼ ► There isn't a lot of startup founders going into Y Combinator and be like, "Oh, what's your idea?"
01:19:16 ◼ ► It's like, "Oh, we're going to make a new foundation model that's going to beat OpenAI."
01:19:20 ◼ ► We're way down the path now where the vast majority of startups are thinking about, "Well, how do we apply this stuff?
01:19:25 ◼ ► How do we build really great experiences and features for people that are just take it, assume that, yeah, there's going to be some great models for us and they're going to keep getting better?"
01:19:33 ◼ ► The big callback, and again, maybe it's just because I just, for years I had had, if you ever have a TV show like this, you're like, "I think I'll probably like that and I'd like to watch it, but I don't feel like starting it yet."
01:19:52 ◼ ► Yeah, I feel like I would probably like that, but I feel the same way. It's like, it's in my wheelhouse.
01:19:57 ◼ ► It was on my maybe list, and I know Kotke liked it, and I often, you know, I share a lot of overlap with things he likes.
01:20:05 ◼ ► But there was some, I don't even know how to describe it, but I had a misconception about what Halt and Fire would be like that was totally wrong.
01:20:21 ◼ ► It is, and I wish somebody would have told me that, because Mad Men is probably my favorite television show of all time. I've watched it.
01:20:27 ◼ ► It's that and The Sopranos. And Matthew Weiner, who was the showrunner and creator and main writer of Mad Men, came from The Sopranos.
01:20:37 ◼ ► So it's, there's a certain similarity. And Halt and Catch Fire feels like that type of show.
01:20:43 ◼ ► Except it's about a little scrappy electronics company in Texas in 1982 or '83 who's going to try to make the shift into making personal computers.
01:21:03 ◼ ► And the show, I think, from my recollection, does an amazing job. They have great technical advisors who are making the characters' lingo seem true.
01:21:14 ◼ ► Just the way that there would be like, I don't know, you think you're going to use this chip from Intel as the CPU, but did you hear that they're working on the whatever and it's got like one and a half times higher megahertz clock rate?
01:21:27 ◼ ► And it's like, what? Are you serious? How do we get that? And you say, oh, you got to know somebody.
01:21:30 ◼ ► And it's like, yeah, like six weeks later, six weeks later, the chip that you thought was the hot shit is now old news.
01:21:39 ◼ ► Yeah. Like you've, we've already locked in like this huge buy of motherboards from Taiwan based on that chip.
01:21:46 ◼ ► And you're telling me that by the time we get them back, we're going to be selling yesterday's news?
01:21:50 ◼ ► But that's happening again with these models, with the AI moment where if you had just gone away for a six month sabbatical and just tuned off the internet, you'd find yourself way behind right now from where you were six months ago.
01:22:07 ◼ ► Oh yeah. On this stuff, like especially trying to like be a developer that's building on this stuff and what are the approaches and because what you're constantly, you take for granted that you have this mental model of what isn't possible as a developer or someone who's building a business.
01:22:27 ◼ ► Like we went through this pretty calm period where through a lot of the last 10 years since really mobile kind of settled in that you sort of got this orthodoxy about here's the default tools you use to build a product and here's the different approaches.
01:22:40 ◼ ► And one of the famous ones is like this kind of almost like received wisdom in the startup world, which is like consumers don't really pay for SaaS.
01:22:48 ◼ ► Like they don't really want to pay a subscription. Like, yeah, there's some, yeah, they have Netflix, but like you can't just make a software company and then people will subscribe for it and make a business that way.
01:22:56 ◼ ► But now suddenly with the turn of all these LMs and image generation models, there are like some pretty meaningful companies being built.
01:23:03 ◼ ► I mean, obviously OpenAI is an extreme example where they have $2 billion revenue supposedly, and a lot of that is consumer subscriptions.
01:23:10 ◼ ► But also just like image generator apps and things that automate things for people's church sermons and people are suddenly, the tools are so useful that suddenly we have this huge uptick in consumers paying subscriptions and we'll see whether or not that shakes out.
01:23:22 ◼ ► But like it goes all the way up the stack in terms of assumptions being broken and that's the fun of it. But yeah, you have to kind of, you feel like you're always behind.
01:23:32 ◼ ► So one of the things I've picked up in recent weeks is this sense that everybody in the industry thinks that these large models are sort of running up against the wall, and that's why they're switching to other technologies to add reasoning because the leaps that were made going from GPT-3 to 4 aren't going to happen again.
01:23:57 ◼ ► Yeah, there's definitely, I think it was Information or Bloomberg or one of these had a story where they'd had sources from multiple of these four labs that I mentioned, maybe all four of them, talking about how the scaling law seems to be slowing down.
01:24:14 ◼ ► We've been talking in the inside baseball, people were really plugged into this stuff. There's been discussion for almost a year about the data wall, which is this idea of, okay, we're not me, but they are training these giant models.
01:24:27 ◼ ► And one of the ways that they've been getting huge gains from the GPT-2 to 3 to 4 and similar for the other models is putting more and more and more data that they've pulled from the internet or transcribing YouTube videos or all these sources of data and putting in more data is one of the main things that is making them better.
01:24:42 ◼ ► And then eventually you have this huge diminishing return in how much effort does it take to get the next token of data.
01:24:49 ◼ ► And by no means are these models being trained on literally all human data, but eventually when you're being trained on more or less all of the useful data that you can straightforwardly scrape from the internet, then suddenly each bite becomes way more expensive.
01:25:04 ◼ ► You have to have a licensing deal or in a lot of these cases they're having, there's a company called Scale AI and there's a few competitors of them that are like a billion dollar company that are creating training data.
01:25:14 ◼ ► So if you're open AI or anthropic, you can say, hey, I need a thousand examples of a senior chemist analyzing this type of thing and talking through the reasoning about why it wasn't wasn't that way.
01:25:28 ◼ ► And it needs to be a PhD and then here's the thing. And then the company will go off and find those chemists and just multiply that by everything that they find that there just isn't enough examples of.
01:25:37 ◼ ► There's not enough examples of people being uncertain on the internet because everyone on the internet just claims that they know what they're talking about.
01:25:42 ◼ ► And so they're like, okay, give us more examples of people who are expressing, well, it seems like this, but here's how I would consider the right or wrong and to try to train the model.
01:25:51 ◼ ► And so one of the things that's not clear is, okay, well, obviously we were getting these huge gains and we were getting easy gains and it's not obvious where we are in this moment in November 2024.
01:26:02 ◼ ► Is this like the various times where in silicon where we are getting faster processors and then suddenly like for a year it's like, oh, maybe Moore's law stops.
01:26:12 ◼ ► And maybe just the 333 megahertz G3 or whatever it is that we're stuck at for a while is just as fast as computers are ever going to be.
01:26:21 ◼ ► And then of course that silly looking back where we've been at the same number of gigahertz for 10 years, but they're getting better and better using other approaches.
01:26:28 ◼ ► And that, yes, that's more expensive, but because we're at this scale, it's worth that extra expense.
01:26:33 ◼ ► Or is this something that historically we'll look back on like, hey, remember those two years where everything got way better and AI just roll at once.
01:26:43 ◼ ► My instinct, it's more like the former, if not primarily because we've proven out so much economic value from these things getting better that there's now a whole bunch of resources to get to that next notch that wouldn't have been mustered if we hadn't proved out that these things are extremely useful tools in so many different verticals.
01:27:06 ◼ ► But even, I mean, I'm just kind of going off and on all tangent now, but like I find personally this argument about, oh, if they hit a wall and are the models going to get that much better?
01:27:15 ◼ ► Like it's interesting and it makes an impact to how we're building our businesses and our products to some degree.
01:27:20 ◼ ► But even if the current models only get really incrementally better and we somehow this is like a really big wall and that it's only just kind of like going to get five or 10% better every year from here on out.
01:27:36 ◼ ► It's like we're all the baby versions of actually building products on top of this stuff.
01:27:49 ◼ ► I mean, I can't tell you in my years of enthusiasm for computers how many times I read, I mean, going back to when you had to read them on print, like get used to it.
01:28:02 ◼ ► Oh, sorry, by the time I got to the end of writing this column, there's been a breakthrough.
01:28:07 ◼ ► But you get used to its human nature where when you get used to an 18-month-old computer feeling way too slow year after year after year, that all of a sudden you expect when that doesn't happen again, you think, well, that's it because that's the way it's always been.
01:28:31 ◼ ► But you forget compound interest, 10% annually doesn't take long for all of a sudden something is twice as fast or twice as good or whatever.
01:28:41 ◼ ► Yeah, even one of the aspects of this that some people at the time can be easy to forget is that even if the models don't end up getting way better in meaningful ways, which we don't know what the path is going to track.
01:28:52 ◼ ► My sense is like the successors to Claude and 4.0 and these are already meaningfully better in the lab, they're just not as much better as the previous generations, like more than 10% better.
01:29:02 ◼ ► And we as humans are very bad at appreciating that. We're like, "Wah, wah, I wanted more."
01:29:12 ◼ ► It's a big deal when you are a business and you make a bet that it's like, well, GPT-2, 3, and 4 were each this far apart and they were this much better, and so we're going to make a big bet that depends on GPT-5 being this much better on this path.
01:29:26 ◼ ► So to some people it is really a big deal. But my point was that even if the model architectures need to change in order to like this thing about GPT-01 and then everyone else is going to be taking the same pattern where,
01:29:38 ◼ ► yes, the model can get bigger and be trained on more data to get better, but it can also spend more time thinking.
01:29:44 ◼ ► And some people don't like it when I call AI's thinking, but we know what I mean when I say it's thinking. It's not the same way if I type something into my Mac and there's this rainbow spinner, I call it thinking even though it's not actually thinking.
01:29:57 ◼ ► We're using these other, even if these other approaches are what they use in order to continue to make things better, the incredible investment in making the silicon and the hardware better and cheaper and all of the incredible economy of scale of running these,
01:30:13 ◼ ► which is still a very new thing, like Ben Thompson was talking about how Nvidia is just now rolling out the first architecture, this Blackwell architecture, which is based on the first post-ChatGPT architecture for even running this stuff.
01:30:28 ◼ ► And so that is so obvious to me that we're going to continue to have economies of scale of being able to run these things cheaper and faster, even if we're getting roughly the same order of magnitude of intelligence out of them.
01:30:40 ◼ ► And there's so many usages where we don't even think to use it for this or that either as developers or end users, because even though it's amazingly fast compared to what you might think two years ago, it's amazingly slow for other use cases.
01:30:55 ◼ ► Or expensive, just prohibitively expensive to have it maybe go off and write a thousand scripts and then you compare them all and then it comes back to you and here's the best one or whatever.
01:31:03 ◼ ► When things get cheaper, demand for the thing goes up, so I think there's a lot of that to be had also.
01:31:13 ◼ ► Explain to me, because I feel like I have a pretty good layperson's understanding of how text-based stuff works with these, and including the fact that they seem particularly good at writing computer programming code.
01:31:34 ◼ ► I kind of get why. Everybody I think knows that it's like, "Oh, it's like predictive pattern matching based on, 'You give me this input, I can guess what you want to do going forward in prose."
01:31:50 ◼ ► And in computer code, it's helpful because computer code is so much more rigid than just language prose.
01:32:01 ◼ ► But it still absolutely boggles my mind that image generation works as well as it does, including photorealistic image generation.
01:32:12 ◼ ► Yeah, there's a distinction there that often gets lost in these systems, because users, we interact with a whole system, a product, and then underneath there's these models. But there's traditionally been totally unrelated or mostly unrelated architecture in between the way that we would generate language and the way we would generate an image.
01:32:33 ◼ ► So if you look like the year ago state of the art around language generation is this transformer model, a large language model, and it's effectively predicting the next token, and that generates prose or code or whatever.
01:32:46 ◼ ► And then we have these diffusion models, which is a different architecture, a different approach for creating an image.
01:32:52 ◼ ► And so it's very vaguely like it makes a random field of pixels, and then it tweaks them over and over again to make it look more and more like the thing you want it to look like, which is a duck or whatever.
01:33:04 ◼ ► And those two technologies were fairly separate. Now, there's an enabling technology of the transformer, which gives the system's ability to kind of reason about text, that makes those diffusion models more useful, because then I can say something like,
01:33:17 ◼ ► three ducks sitting on the bay with the sunset in the background, and the transformer can help convert that into something that's meaningful in the space of this image model.
01:33:28 ◼ ► But then most of the work that's happening to create that image is happening in this diffusion architecture.
01:33:33 ◼ ► So I think that's one of the things that's like non-obvious. When you ask chat GPT, "Hey, make this image," at least for a long while, it would shell out to what it would call DALI, which was its different architecture system that was making images.
01:33:45 ◼ ► One of the things that is very new and not very super well understood yet with the limits of it are, though, is what they call multimodality, which is a $5 word for a large language model that has a shared architecture across different types of things it can output.
01:34:03 ◼ ► And so in addition to us training it on a bunch of code and other textual things, we also train it on a textualization of an image, of a video, of audio.
01:34:15 ◼ ► When we have it all in one model, then we can get things that you sort of have transformative examples like chat GPT's advanced voice mode, which if there's anybody who's interested in this stuff that hasn't actually tried,
01:34:28 ◼ ► not the chat GPT voice mode that was out like a month or two ago, but the one that if you have a chat GPT, I think maybe you need the plus or maybe I'm not sure actually if it's in the free plan, but the one where you can interrupt it,
01:34:39 ◼ ► where you can just talk to it and you talk and it talks and you talk and you interrupt. Even just like playing with that for a minute on someone else's thing is like really drives home the power of this multimodality because previously,
01:34:50 ◼ ► what would happen is people would talk and then it would transcribe it into text and it would send the text into a text model and then text would come out and then it would transcribe that into audio and then you would play you the audio and there would be this lag.
01:35:06 ◼ ► Whereas now it's all in the same model. So A it's way faster, but then the thing that wasn't obvious to me until they demoed it is because now my voice is actually just going into the same model intermixed with the text and all the knowledge that this thing has all the way down to its ability to code.
01:35:21 ◼ ► It can know if I'm annoyed. It hears me laugh and it has some that has a meaning to it's not just haha. It can tell maybe the difference between different kinds of laughs. I can ask it to respond to me in a Scottish accent.
01:35:31 ◼ ► It can tell that maybe I'm putting on a Scottish accent and all that multiplied by every little nuance of human voice and that's just like scratching the surface of this multimodality thing.
01:35:41 ◼ ► Right, and I don't want to get into the philosophical question of what is intelligence, but at least what we're mostly concerned about is our form of intelligence and human beings are so good at certain things that we just take for granted and you can be a dumb person.
01:36:02 ◼ ► Somebody with a low significantly lower than average IQ and you still can pick up things like, oh, my dad's annoyed at me.
01:36:13 ◼ ► Yeah, and just from the tone of voice, right? It's it's a really complex thing. If you think about how would you program a computer to tell if this man is annoyed at the moment.
01:36:24 ◼ ► And there's a thousand little subtleties like in the advanced dingus voice on iOS, I've noticed there's more inflection going on. And so when I get a notification of that something was from Slack, I've noticed the voice at least the voice I have selected will tend to sort of sound like kind of like pleased and surprised that it's from Slack.
01:36:45 ◼ ► Oh, new notification from Slack. There's just a little lilt in the thing that makes her sound like it's like, oh, hey, cool. It's that's what what the notification came from. And that is like in the model somewhere in our models to that we wouldn't be able to name it. We don't return for that. We don't have to teach it because it's totally implied in in the way we communicate.
01:37:05 ◼ ► Right. And is it something that we literally like I said, like how we by nature, take for granted, fast paces of innovation after we're done being excited by them, then we just assume well, every six months, there's going to be a better blank. We just take it for granted.
01:37:21 ◼ ► We've actually evolved over 10s of thousands of years to just take for granted that we pick up tone from other human beings, and amongst other things, facial expressions, right? I mean, especially when you get to know somebody like your spouse or your children or your parents, or your best friends or your boss or something you can just read so much on somebody's face sometimes.
01:37:44 ◼ ► And yes, some people are better than that or care more about it. That's empathy. But it's like so much information and we just pick it up when you do. It's instantaneous and you just don't you don't think you did anything. You don't think your brain did any computation to pick up. My wife seems annoyed, but
01:38:03 ◼ ► Yes. And there's a whole bunch of information there now that is like accessible or maybe like I'm over focusing on this AI thing, but it makes me think of when you train an AI model from a transcription of YouTube model, there is a lower ceiling of how useful that knowledge is than if you train an AI model that actually understands the nuances of the voice and maybe even the facial expressions of the person when they say something sarcastic, or they say something that they don't really see, feel like they believe.
01:38:32 ◼ ► They're like, well, I guess AI is going to enslave us all, right? I'm obviously kind of making like a Simpsons reference or something. I don't really mean it when I say that or I'm not concerned about it. Whereas you just transcribe that text and then you ask, what is the sentiment of this? It's like extremely negative. This person is expecting to be enslaved and this is horrible. And so there's so much more bandwidth in that information.
01:39:00 ◼ ► Yeah, well, I mean, it's fascinating for me as someone who's been spending all this time with these huge frontier models that have like the multimodality balloons the size of the model, of course. So it's hundreds of billions or maybe even trillions of parameters. Each parameter is maybe a bite. So we're like, maybe it's like a terabyte model somewhere in a cloud. They don't disclose how big the models are really in that order of magnitude. And then it's like, well, but what can I do with a model in my phone? How good can that be?
01:39:27 ◼ ► And obviously Apple's been like hard at work trying to find the best and highest uses of a model like that, which are very different. The little model that's on our phones is not able to seamlessly jump in between our facial expressions and tokens and coding and stuff like that. It's way more specialized.
01:39:43 ◼ ► What do you think about the just the basic two layer, some of its local on your device and more complex things will get formed out to private cloud compute architecture?
01:39:58 ◼ ► It's so intuitively good strategy for Apple. Apple has the best fleet of devices in people's hands, billion-ish devices with the best processors for this type of work. And they're already in people's hands and having them use their devices locally, obviously, is cheaper for Apple and faster for the people and privacy preserving.
01:40:20 ◼ ► So there's this huge strategy alignment of doing it the local inference, but obviously there's very harsh limits on what a model that fits on your phone can accomplish.
01:40:31 ◼ ► And as these frontier models and the clouds and the G4Os just keep getting better, they're potentially probably expected, my expectation, they're going to keep getting better faster than the local models can get better also.
01:40:44 ◼ ► Because we have 16 gigabytes of memory on our phones now and like maybe the next phone will have 24 gigabytes, but Apple does not tend to double its RAM very often.
01:40:55 ◼ ► Meanwhile, these people who that's their entire job is hosting these things in the cloud, they're able to make them better yet faster and faster.
01:41:01 ◼ ► And so the dual setup where Apple has, okay, we're going to process what we can locally, quickly and completely free for Apple and fast for you away, privacy preserving, great, makes total sense to use that for things that they can.
01:41:15 ◼ ► And then this private cloud compute approach is also we maybe sometimes joke about, oh, only Apple could do this, only Apple to do that.
01:41:22 ◼ ► And like, yeah, maybe Google could create like an equivalence of the secure enclave and secure chain of all of the way that Apple is doing.
01:41:31 ◼ ► Like Google in theory could probably build something like private cloud compute, but it's extremely Apple-y thing and they're already ahead by having a lot of the pieces of that.
01:41:39 ◼ ► And so, I mean, as a strategy, it makes perfect sense. The question is, does it play out in the product as we sort of maybe are seeing it come together in front of us and how will it play for them as the models keep getting bigger and user expectations?
01:41:53 ◼ ► Back to those two things I started kind of thinking about, how do we think about Apple intelligence? You can evaluate as, is this useful?
01:41:59 ◼ ► And then you can also separately ask, is this competitive in terms of capabilities? And those are like, I think they're two of the interesting ways to look at it.
01:42:09 ◼ ► Right. And it's, again, I do feel patterns repeat themselves and it's where Apple by nature of the company finds itself, like that slogan from the original Macintosh, the computer for the rest of us, which wasn't turned out not to be true for the Macintosh until a lot longer.
01:42:32 ◼ ► You know, like if I were around back then and I got a job working under the pirate flag, whatever the bandy for.
01:42:41 ◼ ► Yeah. And suggested that instead of the computer for the rest of us, it was, I raised my hand and say, maybe let's tone it down a little and say the computer for more of us. I'd be kicked right out of there.
01:42:53 ◼ ► But it was right. But it was like the computer for everybody didn't happen until the internet was everywhere because what the true killer app to make everybody want to have a computer, let alone a computer that they carry with them everywhere was communication because it's the one that's what we are.
01:43:11 ◼ ► We're communicative animals, but it's funny to me that Apple, not funny, but it's like where Apple is, is when there's these breakthroughs, how do we make it useful to actual people?
01:43:25 ◼ ► You know, and again, and the knock against Apple from people who don't like them is that everything happened somewhere else first.
01:43:32 ◼ ► The graphical user interfaces with a mouse pointer came out of Xerox PARC. Apple didn't invent that.
01:43:42 ◼ ► Right. There were multi-touch screens before the iPhone. The Microsoft Surface table was out before the iPhone, you know, and there were other people who had like academic things.
01:43:52 ◼ ► But it's like, okay, multi-touch could be a thing that we bring to everybody. What would we do? That's what Apple does.
01:43:58 ◼ ► And they're really just culturally really good at it. And they've been good at it now for close to soon 50 years because it's not the same people over 50 years, but because the culture is there that people who are drawn to approaching problems that way go to work at Apple.
01:44:19 ◼ ► And that Apple has done a good job with steady, consistent, low churn leadership over the years of keeping the, you know, so what's their approach to these large language models?
01:44:35 ◼ ► As opposed to Apple being the company at the forefront of creating the new ones on the frontier, it's, they're more focused on how do we bring it to people?
01:44:46 ◼ ► And that's what they're branding as Apple intelligence. And in some ways, you know, I think that the sort of one word summary in the conventional wisdom is, ah, meh, right?
01:44:58 ◼ ► Apple intelligence is, it's all hype. I think the truth is actually that it's pretty, I think they're on the right track. I think it's a pretty good debut feature.
01:45:15 ◼ ► I mean, they're marketing it, yeah. I mean, you have to get a little bit separated out. Are they over-marketing it from, is it useful, from, is it eventually going to be useful?
01:45:24 ◼ ► There's like three pretty, like the, I mean, they need to market something. They obviously are confident in it because they're very typically marketing a thing before it's fully there.
01:45:32 ◼ ► They're obviously a little bit off their comfort zone, absolutely off their comfort zone in doing that, having national ad campaigns. The Siri glowing effect is launched before the product is even any different from the way it was before and stuff like that.
01:45:48 ◼ ► But I think the question of is this useful is hard because when you use it, obviously someone uses 18.1, it has a few features, 18.2 has more, there's going to be more all the way through all of the 18 and then it's going to get, we're going to obviously get yet more features and then also polish on these pretty, I don't want to say hastily done, but like obviously they built these Apple intelligence features rapidly and under an intensity where some of them are a little bit rough and obviously we're using big,
01:46:17 ◼ ► we're using beta versions of even some of them and even some of the ones that it's not beta anymore, like use the writing tools or I don't know, what's your take? You're a writer and so, and I write often and so we have weird take on writing tools.
01:46:32 ◼ ► It's funny because I've, and I've written this and I do not mean to be arrogant about it, but I just don't feel like I need help with my writing at all and it's not that I can't go faster, I could definitely, you know, if there's any problem with my writing ability is that I'm very slow, but AI isn't going to help me go faster because the reason I'm slow is I don't even know what I want to say.
01:46:58 ◼ ► And I've always explained my, I have a very, if you ever watched me touch type, I have some very idiosyncratic, not according to what's the,
01:47:36 ◼ ► And there's this, it's like a website or something, but it's a pretty cool interface for like trying out keyboards and you get like a words per minute and you can do it in different ways.
01:47:45 ◼ ► And I explained to him, I was like, well, I got to 45 or 50 words per minute and that's about how fast I think.
01:47:54 ◼ ► I just keep forgetting to use it. And then when it pops up in front of me, I'm like, wow, why are you popping this up?
01:48:13 ◼ ► But they're not blown away by how they're not wrong, but they're not exactly what I'm saying.
01:48:32 ◼ ► This is one of these big sort of things that Apple has taken on by saying, okay, we're going to try to build the most useful thing for a model that fits on your phone.
01:48:41 ◼ ► And so it's like maybe like roughly the benchmark it compared to these like kind of three gigabyte ish, four gigabyte models maybe.
01:48:48 ◼ ► And if you want to solve that problem, which you described is totally tractable with current models on the server, you say, oh, well, these little suggestions don't sound like John.
01:48:57 ◼ ► The way you would solve that is you would use what we call in the programming jargon, RAG.
01:49:05 ◼ ► And here is just hundreds and hundreds of examples of the way that John writes in various contexts.
01:49:15 ◼ ► And you can just send way more to it and do that and quickly and reasonably within the kind of arbitrary performance budget because you're in the cloud.
01:49:31 ◼ ► Retrieval augmented generation is the buzzword that people in like the AI engineering world use to refer to this technique that was like completely transformative when people kind of discovered how useful it was, like, I don't know, 18 months ago or something like that.
01:49:47 ◼ ► Where what people would try to do is you would try to get some behavior out of this model.
01:49:54 ◼ ► You say like, you know, all the jokes people say like, we're going to abduct your family unless you produce valid JSON.
01:50:01 ◼ ► Or they would maybe give examples of like, okay, here's some examples of valid JSON to make it like that.
01:50:07 ◼ ► But then there's been a huge now like literature and also just practical experience of developers finding that there's a lot of things that we can get the system to do, especially when it comes to like eliciting a certain style of behavior.
01:50:22 ◼ ► Is by pulling relevant examples instead of just having boilerplate like the same examples every time.
01:50:27 ◼ ► And so a great example, like Apple is using this for the feature where you can ask, hey, dingus, I'm having actually I had experience with this said, hey, dingus, how do I turn on double tap to evoke the typing interface rather than having to speak.
01:50:41 ◼ ► And what they do under the hood is it's not that they've trained the model so it like inherently knows the answer to every single Apple support question.
01:50:48 ◼ ► It has a little database and so it has a prompt that says, hey, the users asked a support question and then it pulls in using kind of traditional procedural code.
01:51:01 ◼ ► Please explain to them using these relevant support articles, not just every single article, how to solve this problem.
01:51:07 ◼ ► And then it is able to do it way more reliably and with less like hallucinations as they call it.
01:51:12 ◼ ► And so in theory, they could and I'm sure in the near term probably will tweak this where like the suggested responses and messages could be informed by the entire archive of things I've typed in message before.
01:51:35 ◼ ► If you were just given unbounded, hey, I want really good suggestions, that would be like the go to like absolutely you would want to do that.
01:51:41 ◼ ► The thing that's not clear to me and maybe they have in the lab actually a solution to this that is good, but if not for this problem, there are pretty big large categories of problems that Apple hit with a local model.
01:51:56 ◼ ► It's harder with a certain amount of memory because one of the big problems is without getting super technical is what we call context length, which is like how much text you can send to the model that it can actually think out at once.
01:52:10 ◼ ► So, if you say here is the five words that John has typed so far, guess of six word, that's a very small amount of context.
01:52:27 ◼ ► And at some point, you end up needing more and more and more memory to be able to process all that and you need larger models and to be able to even also reason about all the subtleties of like, okay, well, what's an appropriate tone in response to this thing?
01:52:39 ◼ ► Is this person joking or not is actually relevant, but like humor is a really complicated topic, which is part of the stereotype that we have that maybe people are funny or sometimes thought of as smart, which is like there's a little bit of evidence that like sometimes actually,
01:52:56 ◼ ► And so you find like basic models and even the most advanced models struggle with that.
01:53:00 ◼ ► But if I'm making a text message quip, like it needs to know I'm joking or they're joking.
01:53:05 ◼ ► And the example like that I'm thinking of, like in terms of picking up on one user's idiosyncratic style is imagine somebody who tends to overuse or just use which any use of it is overuse, bro.
01:53:23 ◼ ► And it's just, you know, like you start sentences with bro, blah, blah, blah, or hey, you're supposed to start this meeting five minutes ago, bro.
01:53:36 ◼ ► So like if I type bro all the time, it's not going to suggest in my suggestions, tacking or prefixing bro.
01:53:43 ◼ ► But then you could go even more complicated and maybe me and my pal Alan address each other as bro, just me and you.
01:53:55 ◼ ► In a completely ironic way, knowing that we sound like assholes using bro in the end of our sentences.
01:54:04 ◼ ► I would never in a million years do it with anybody else, but I know me and you do it all the time.
01:54:17 ◼ ► There's no way Apple intelligence as it is today is going to pick up, oh, when John and Alan talk to each other, they're always calling each other bro for some reason.
01:54:26 ◼ ► I mean, I don't want to be so confident and say it couldn't do that because you can imagine, especially if you're constantly saying bro, like it would be pretty straightforward to say, here, you're responding to John and here are the last 20 messages in between you and John.
01:54:55 ◼ ► Well, I mean, now you're getting to, there's a whole philosophy and like Mike Harley and CGP Grey were talking about this on Cortex about how there's something fundamental about the act of writing to your friend, like actually choosing what words to say, that is so maybe sort of reinforcing.
01:55:12 ◼ ► There's something kind of maybe philosophically different in between just selecting the suggested response versus actually going to the act of saying, either making the joke or if I don't type, I love you, it's just the well, yes, statistically, maybe I was likely to say I love you, so I guess I just tap it.
01:55:28 ◼ ► And then if I tell the AI, oh, just send I love you when you feel like I would do that, like, I'm actually probably maybe hurting something in my brain.
01:55:39 ◼ ► It feels, I feel like on the one hand, like I said to you, I'm looking, I've been looking at, especially on my Mac at these suggested replies in messages.
01:55:50 ◼ ► And sometimes it's just thanks or exclamation mark or it's a real specific reference to the thing that person was talking about.
01:56:00 ◼ ► But at the moment, they're not like, yes, that's exactly what I want to say, click on it or down arrow to it and hit return.
01:56:07 ◼ ► But if it gets a little better and it really is exactly what I want to say, that actually I'm going to be amazed for like a moment or two and then I'm going to turn the feature off.
01:56:26 ◼ ► I don't even use Instagram itself very much at all anymore because I find it, I don't hate it.
01:56:32 ◼ ► I just find it, it's transmogrified into something that doesn't resemble what it used to be at all and what it is.
01:56:38 ◼ ► I don't really, it's not really part of my world, but like it might be unique to me because I do have a bizarre number of followers on Instagram compared to the average person.
01:57:07 ◼ ► Right. And that people can send me DMs on Instagram and I don't have to look at them or answer them.
01:57:21 ◼ ► There's an interview in this, there's this podcast called Latent Space where they interview engineers who are building AI functionality and founders are building AI functionality.
01:57:32 ◼ ► But a few months ago they had on someone whose entire startup, and apparently it was flourishing, was what you just described, which is like AI generated responses to fans, but only for only fans creators.
01:57:44 ◼ ► And so they, if they're successful, they have all these people sending them messages and asking for things and whatever, and they want to respond to all of them.
01:57:52 ◼ ► But right now the solution to that, if you're very successful on that platform, which is I'm sure also true for celebrities that are really big on Instagram, is to have an offshore team doing it.
01:58:01 ◼ ► But then it's hard to tell, like, are they doing what they want them to do? And how do you direct them? And are they doing the right things?
01:58:06 ◼ ► And obviously it's also expensive and maybe slow, whereas this is one of those many, many, many AI startup shapes, which is automating these things.
01:58:14 ◼ ► And yeah, obviously for you and I, we're turned off by the idea of an AI pretending to be us, but if that's your business model, then it happens.
01:58:22 ◼ ► It just seems bizarre. It seems totally bananas to me, but simultaneously I'm like, but I can totally see why some people would say, oh yeah, this is amazing.
01:58:30 ◼ ► Well, it makes sense that you would object to that because when you're on Instagram, you are you.
01:58:38 ◼ ► Whereas Chappell-Rone is not actually, like Chappell-Rone is a character that she's created and that if Chappell-Rone could have someone respond to their Instagram DMs, that probably actually would be great because it's overwhelming and there's so many of them.
01:58:52 ◼ ► And so it's not like, but wouldn't want that to your friends, which is where this thing in messages, it's differently for me and I think for CGP Grey.
01:59:01 ◼ ► In a way that like being a modern, well, maybe being a celebrity has always been sort of like this, even before the modern social age, but certainly today's creator world, there is a blurring of line between being a person and being a character that the person plays.
01:59:21 ◼ ► But if like, I was going to say Fox, but I guess Disney owns them now, but if Disney let you interact with Bart and Homer Simpson, there's no, maybe that's actually even available, but nobody is being fooled that they're talking to the real Bart Simpson because they know there is no real Bart Simpson.
01:59:42 ◼ ► And there's a blurring of the line there. It just seems crazy that Instagram is offering it to me with, I don't know, 40,000 followers or whatever it is that I have on Instagram.
01:59:52 ◼ ► I don't even know. But when I'm not even close to the Chappelle Rohn level of 40 million followers or whatever she has.
02:00:01 ◼ ► And that's a whole, like a whole, there's obviously simultaneously what can be done, what should be done and what's going to get done with all of these technologies.
02:00:11 ◼ ► Right. But it doesn't surprise me that Meta is leading the way. And I don't even mean that in the most pejorative sense.
02:00:28 ◼ ► I just feel that they are completely tone deaf in a certain way, culturally inside the company that that stems from Zuckerberg personally, in a way that a founder can infuse the company with a personality where they, I don't think the way that I think this feature sounds absurd.
02:00:47 ◼ ► And I think a lot of people listening to this episode of the podcast think that that feature chat with Jon Gruber, but it's all AI is friggin ridiculous.
02:00:56 ◼ ► Oh, unless you did it just for fun. Right. But as a serious feature on as part of my professional life, it seems insane. Whereas inside Meta, I'm sure that I don't think that that it really gave them second thought. If they can do it, they did it.
02:01:11 ◼ ► You described Apple's mindset about how they approach products of like, how can we take this thing that's now been invented and create a really delightful, useful version of it.
02:01:19 ◼ ► And they have crafted that to the nth degree of gotten really good at building products that way. Meta has this all, like you say, from Zuck on down through some of the world's most like brilliant people figuring out how do we engage people.
02:01:33 ◼ ► How do we make numbers be statistically significantly more engaging. And sometimes that leads Meta to create things that you would never have thought that these incredible inventors like Ben Thompson talks about the newsfeed, which I remember as a user when they introduced that Facebook many years ago now,
02:01:48 ◼ ► I was like, they're ruining Facebook. This was that was like one of the economic inventions of the century is what they that change that they made, because they think that way. But then it leads them sometimes into things like this AI feature you're talking about, or the way if you spend any time on threads, how they went way into a hole where the algorithm would promote literally anything that people clicked on, even if they just clicked on it in confusion, because just someone figured out, oh, this people tap on this tweet, that's our post, it's got cut off. And it's someone just being like, oh, I was in my kitchen.
02:02:17 ◼ ► And then suddenly, I fell into a dot dot dot. And you're like, why is it? You fell into what? What did you fall into? And it's not it wasn't even like people were creating these tweets to be confusing. It's that it just if it just happened to truncate at a mysterious place, the algorithm would notice that, like accidentally and then promote it and then people would tap through and be like, oh, it wasn't actually interesting. But it is satisfying algorithm,
02:02:41 ◼ ► or even just I guess I have to tap it to see where this goes. But it didn't even go anywhere because it was an accident. And yeah, everybody just but everybody tapped it. So the algorithm thinks Oh, everybody wants to see this happen on it.
02:02:57 ◼ ► Yeah. So it's a blessing and of course, in the same way that Apple strategies is a blessing and of course, sometimes it makes it more difficult for them. And sometimes it makes them more able to do but
02:03:04 ◼ ► So the writing tools to answer your question, I you'd think I would have the strongest opinions about them as a professional writer and I have the least opinions about that compared to the other features they have because I just don't
02:03:35 ◼ ► It gives you more advanced underlines of like, hey, this word is supposed to be written as two dashes or whatever. But you're very specific about your style of how you write things. So you probably find it even more annoying.
02:03:45 ◼ ► Right, because I technically again, according to Hoyle break all sorts of grammar rules, or if it's not a rule, at least considered best practice all the time on purpose. And it's it. When I looked at it, it was green underscores everywhere. Yes.
02:04:03 ◼ ► I do remember I when I wrote my iPhone review, I pumped the whole thing through the writing tools in beta. And it did find some typos and stuff that I didn't ask it to rewrite it. And I keep forgetting to try that again. And I thought about that last night when I was thinking about recording the show to you today. I was like, huh, I thought at the moment I was like, you know what, every time I write something of any, you know, like a page or longer, I should at least pump it through the proofreader.
02:04:31 ◼ ► One of these kind of parts of wisdom that's seen as starting to kind of come about in the AI product world, which if you talk to enough people are trying to build AI products, which is that if you can build an AI feature, like our goal isn't to build AI features, right? Like users don't want AI, they want some problem solving, they want some. And so the ideal wonderful and I'm sure Apple thinks this way too, because it or at least they're I'm sure they're developing this mindset as well, because it fits with their ethos generally, you ideally want an AI feature, the users don't even think about it.
02:05:00 ◼ ► It just makes things better automatically. And so the summary feature is that way where it's I get 20 WhatsApp messages and it's someone saying, oh, does anyone have an avocado that's ripe? Oh, actually, no, I don't. Oh, no, I actually, okay, it's fine. It's settled now. And that all gets summarized as to avocado needed and procured.
02:05:18 ◼ ► And that's just, it works. I don't have to decide, oh, there's too many messages. Should I decide to summarize them? If they had built it that way, the feature like we would get 0.1% as much usage. It would just be kind of like a novelty, but you wouldn't think to evoke it, even if it is useful. And so that's the ideal workflow, but the writing tools one is not wired up that way, probably for cost and performance and battery reasons to be constantly analyzing.
02:05:37 ◼ ► Although actually, well, so that goes into the whole thing about local versus server, right? Because the writing tools for long text, at least I think it's going to private cloud compute.
02:06:00 ◼ ► Yeah, and, well, but they're so super cute. You'd, you'd, there, apparently you can go, I haven't done this either, but apparently if you go into settings, Apple intelligence, there's a way to see a log of,
02:06:19 ◼ ► So what I've been trying to in playing around with these tools I've been trying to notice, and I've actually experimented with like turn on airplane mode. That's the way I've been doing it. There's a lot of that would be better, but turn on airplane mode and be like, does this feature work? Does this feature work? And does this feature work?
02:06:32 ◼ ► And almost all of the 8.2 features that I tried were still working. The few things that I noticed didn't work was the cleanup tool in images, which is going to the server. And then the some of the like writing tools for longer pros, like summarize this long article, key points, list table.
02:06:50 ◼ ► But now that actually I say that I recall that the writing tool I was most interested in, which is the proofread one, not because I'm like you where I'm pretty opinionated, I do break some style things, but I do miss some. It caught a typo I had where I miss capitalized a letter in someone's name.
02:07:06 ◼ ► It was a common name. And the thing that I found, two things I'm understanding. One is that it was very slow because it was doing it locally, whereas the amount of text and the amount of work it was doing could have been done faster on private cloud, but or certainly a big model.
02:07:20 ◼ ► And the other thing that I found kind of infuriating was the UX of it was they defaulted to, and maybe this is better for the median person and we just shouldn't even be talking about this because we're two writers talking about something that's not meant for non writers.
02:07:32 ◼ ► But it defaulted when you proofread to apply in all of its suggestions. So all at once, and I was like, horrified. Like, no, no, no, yeah.
02:07:40 ◼ ► Let me pick the ones that I like. Don't just just rainbows in that cool animation. Now I've gone and made a bunch of mutations and you can like through them one by one and undo most of them because I'm like, no, no, no, that's just kind of a joke.
02:07:53 ◼ ► Or I'm saying, like, you didn't like that I had YOLO lowercase instead of a free case. I would like to style it lowercase in this context.
02:08:02 ◼ ► You're right. It's like you and me, people like us, we want to see like a diff interface, like a nice one. You know, it's Apple. So we make it like kaleidoscope, make it really pretty.
02:08:09 ◼ ► But show me my original and yours and let me select which ones I want one by one with 100% clarity and certainty of which ones I'm getting. Don't just give me the magic apply it button.
02:08:23 ◼ ► Yes. And that interface that you mentioned is exactly the interface that like pro, like the state of the art developer tools right now, like something like cursor, which you've probably heard of, which is like the huge buzzy editor that people are using that have AI or like even in Microsoft Co-pilot, but like cursors more advanced than it.
02:08:40 ◼ ► It are like totally bought into that model of like, I want to know exactly what is happening to my code. I can't just trust an AI to change random stuff. But the AI is extremely useful to be like, Hey, how do I change this, whatever. And you can have a little conversation with it and you can click a button and it'll apply a diff and you can highlight exactly what it changed.
02:08:58 ◼ ► And in these tools like cursor, you can save the file and then even check, does it still run? Does it run the way I wanted? And then if you come back and you're like, no, you can undo or you can keep editing until you're really like, it's so tentative about having the AI change your stuff.
02:09:17 ◼ ► I want that, but I also don't blame Apple for not writing its writing tools that way. I do find of the features, I find that I've mentioned this analogy before, and I still stick by it. I find the notification summaries to be excellent. And I really like the feature, even though it is sometimes wrong and sometimes a little comically wrong.
02:09:39 ◼ ► And my analogy is that it is like it's creating the equivalent of the subject line from an email. And in the same way that it's never been the case that you can skip the email just by reading the subject, because it's a really well written subject, which is its own argument of email style and etiquette.
02:10:01 ◼ ► It just means what's in this email? Do I want to read it right now? Right? So if I got an email from you today and the subject was might need to postpone the recording, I would immediately open that email this morning and say, oh, what's up? I want to know what's going on.
02:10:16 ◼ ► And if you wrote a subject that just said podcast, I would be like, knowing I was recording with you today, I'd probably still open it.
02:10:25 ◼ ► But if I wrote an email saying like, funny article, I'll look at that when I look at it.
02:10:31 ◼ ► Right. And then if you were, I showed up to record the show and I texted you and you're like, oh, I emailed you to tell you I couldn't do it at 3.30. I'd be like, what? That was what was in that email that said funny article?
02:10:44 ◼ ► Right. And like, you know, Joanna was on last week and she was talking, you know, a bit like, it's really useful. I don't know, for the one to any specific use case, you didn't need LLM technology.
02:10:56 ◼ ► But like if you have 20 notifications from your garage door, and garage doors open, garage doors closed, garage doors, but if you could just get them all summarized as a bunch of updates to the state of your garage door, but it's currently closed.
02:11:10 ◼ ► that's brilliant. And you think, well, you don't need to fancy LLM for that. But you do unless you want to hard code. Well, if it's garage door, you want to do this. And it's it's the old Siri, right?
02:11:21 ◼ ► Right. And it's like, okay, we will hook up to a database of sports scores. Okay, here's one that has the NBA. Here's one that has the NFL. Here's one that has, oh, what about college football? And then you're like, oh, well, college football, there's like 400 teams in all divisions. And it's like, oh, well, maybe we'll just stick
02:11:43 ◼ ► So this is like something out of the AI research world, where every AI agent, where you try to build an agent that could do chess or whatever it was that people were trying to build 10 or 20 years ago, would go to extreme lengths to do what you just described, which is create a database of all the things you might care about, and teach and like get expert, the world's best chess master. And what do you think about it's like, well, they need to think in terms of this and that and that.
02:12:07 ◼ ► And the bitter lesson is what all the AI, especially the more senior AI researchers who have been doing that and slaving away for decades, felt when actually these fully generalized neural net approaches were at a certain scale with enough data, were just like, yeah, you just kind of like have it just play against itself, and you give it enough data and eventually just figures it out.
02:12:29 ◼ ► And all your hand tweaks were just all just trying to boil the ocean, you were never going to sit for past once there'd be we got enough data and enough compute. And it was all just a waste of time.
02:12:38 ◼ ► I'm pretty sure I know the term I did too, but but I think I actually took an entire course to get my computer science degree called expert systems.
02:12:55 ◼ ► Yeah, I don't think so. But I remember taking expert systems. And that was exactly I'm probably not very certain I took a course called expert systems. But that was basically it is you find these domains. And it was about like, how do you structure the data models for that particular problem? And can you apply it, you know, our sports scores similar to chess? Well, no, but you know, it's so you'd have to do it differently. And you just keep building up a list. And it's like, nope, it's never gonna work.
02:13:24 ◼ ► Yeah, the sports score thing was somewhat useful before, but it's never going to be as useful as the fully generalized thing that is now inbound, where you can ask the sports scores and it says, oh, maybe it was eight to seven. And then you say, oh, what's their record playing against each other? And then you and then the answers and then you're like, how does that compare to this other thing? And you're just like following your curiosity asking questions that no one ever would, you could do 100 interviews, no one would ask that slice of that question. How does that interact with whatever other thing?
02:13:53 ◼ ► So we have this amazing opportunity, but then on the nuts and bolts of building the product experience.
02:14:01 ◼ ► I like I was saying, I love them for the group chats, like obviously, sometimes they're weird or a little bit wrong. Like I got one and like the types of errors that I see are the sort of errors I would sort of expect a smaller model maybe to struggle with where it says something statistically likely, oh, there was a fight and Mike Tyson was in it. So he won.
02:14:24 ◼ ► The thing about Joanna Stern, it's like talking about, oh, there's an argument and then it's like it makes up a husband for her, which you do that does not exist because statistically, especially the smaller models will have these problems.
02:14:34 ◼ ► I see those and certainly we need to take seriously like being trying to build systems that don't have some of these like cultural biases and stuff.
02:14:42 ◼ ► But just in terms of usefulness for me personally, I find that the notifications, especially for noisy group chats are one of the more useful Apple tokens.
02:14:51 ◼ ► Yeah, and I feel like the problem where it incorrectly and I, you know, I don't take it lightly because I'm on the assumptions are always going to be right. My spouse is a wife because I'm a man.
02:15:04 ◼ ► And some of that, it's like just defaulting to calling your spouse or spouse, just using the word spouse, you'll never be wrong, you know.
02:15:13 ◼ ► And it's totally fixable with what we call post training is like you train a model and then your web service behavior and then you add a few bit of different additional training data to be like, try to say spouse, try to, but you'll of course, it's tricky with cultural things like that.
02:15:26 ◼ ► That's like one of the, yeah, easy to technically fix that you and I would not be fussed if it said something about our spouse, but maybe some Apple users would be fussed about in certain cultures and certain ages.
02:15:36 ◼ ► And then indeed, ideally you would try to intuit that from the data, but then you're working with a small model.
02:15:41 ◼ ► And so how tractable is that? And so I don't envy the teams that are needing to actually do that, but it is tractable.
02:15:46 ◼ ► It seems like not a fundamental thing that it'll never get fixed with AI or even like local models.
02:15:55 ◼ ► You know, and again, the stakes are very different, but it's like you never ever want to insult somebody or just you don't want to call a woman's wife or her husband.
02:16:04 ◼ ► I mean, it's just, it's a bad note, but it's the same thing where you can have an amazingly high accuracy rate of using the right word and getting the right thing.
02:16:20 ◼ ► Yeah, this is a huge problem. This is a strategic challenge for Apple coming into a technology like this is fundamentally chaotic and uncertain and hard to keep on rails and hard to keep within.
02:16:34 ◼ ► This is like the image playground that I've been joking around with and like playing with image playground and trying to get it to do things.
02:16:40 ◼ ► And obviously, A, it's struggling a little bit because it's a local model, so it's small.
02:16:43 ◼ ► And so it doesn't compare to like the greatest and best state Leonardo AI or stable diffusion or mid journey or whatever.
02:16:49 ◼ ► But also they obviously Apple is especially concerned about people in even intentionally they call it a screenshot attack.
02:17:00 ◼ ► It's like when you have an AI system that especially like a high profile company like Google or Apple is published and then people purposely try to get it to misbehave so they can take a screenshot and get the likes or engagement online.
02:17:11 ◼ ► Like, oh, look at what Apple intelligence and like even I was sort of stooping to that a little bit.
02:17:22 ◼ ► And then I was like asking it to do things like, oh, can you get me to be thoughtful and kind of be like thinking, hmm.
02:17:29 ◼ ► But when it started giving me bad output, I purposely like kept scrolling and trying to then I was interested in what are the worst outputs?
02:18:01 ◼ ► Imagine a caricaturist at a carnival or an amusement park and you come up and you give the guy 20 bucks and he draws a comical cartoon style picture of you.
02:18:12 ◼ ► But the way that those caricature artists work is, yes, they're going to, like for me, they're going to exaggerate my nose because I have a large nose.
02:18:32 ◼ ► Well, I just texted you some of them and I have a whole bunch I can send you more after.
02:18:49 ◼ ► And this is, it's absolutely a hard problem to get an image, like a diffusion image model to make a likeness of a person that still evokes them.
02:18:58 ◼ ► Like a caricaturist is like a professional human who is at the state, like at the limit of their craft at pulling out details and being like, look at this person is like, ah, you know,
02:19:18 ◼ ► But that's like a complicated, subtle concept for like a small local image model to try to teach it.
02:19:28 ◼ ► And like if they have a big nose, you might want to make it big, but they might be sensitive about that.
02:19:31 ◼ ► And this is part of the risk to Apple where Apple has to be more careful as a extremely high profile coming to a billion users,
02:19:38 ◼ ► whereas a company that their entire thing is like making a new generation playground stuff for fun,
02:19:55 ◼ ► These round frames that he's been wearing the same ones for 30 years, and he doesn't just wear glasses.
02:20:06 ◼ ► And I'm just thinking like with yours, and again, I remember we were in a private group where you first sent it.
02:20:16 ◼ ► And you said that the crazy part is if you tell Apple's image playground app to make you with cross eyes, it won't do it.
02:20:25 ◼ ► But I'm just imagining like the Curb Your Enthusiasm scenario is me and you and a couple of friends go up and we're like,
02:20:32 ◼ ► ah, let's all get caricatures and mine looks very flattering and somebody else's looks flattering.
02:20:53 ◼ ► Well, I feel like Apple image playground. Well, I thought maybe Apple image playground had a vendetta against me, but Mike Hurley sent me his.
02:21:15 ◼ ► The branding issue of not doing inappropriate things is unique to Apple, well not unique to Apple, but certainly they care more about it than anybody else.
02:21:29 ◼ ► They're clearly, but they obviously think highly enough of the entire suite of features that they've made.
02:21:42 ◼ ► So it's fascinating to see Apple shipping something that is not in great state, that they are clearly internally from the highest level down under let's get this better as fast as we can.
02:21:55 ◼ ► Let's, it's a very interesting way of seeing, let's see how fast Apple can move on this right now.
02:22:07 ◼ ► And even in beta, like I said earlier, you can see it over the course of weeks, this feature is getting better.
02:22:17 ◼ ► I feel like the image playground one, there's two things simultaneously where there's like, okay, yes, it's a hard problem and they have a particularly difficult thing about these screenshot attacks and I'm making a worse by making sure I save the one where I'm cross-eyed or whatever.
02:22:39 ◼ ► And in the keynote, they're like, we'll make an image of your friend and send it to them.
02:22:43 ◼ ► And if somebody made this image of me, right, it was Apple's choice to be like, send your, if I'm going to send an image, a rendition of my friend, I want it to be the most flattering.
02:22:54 ◼ ► Like it should, it totally evoke them, but like an idealized version of them, but still pleasant, but enough that it, it wouldn't be, this is not the choice that I would make on that dimension.
02:23:05 ◼ ► But I think to some degree, maybe they were saying in June, like you had to make this decision in June, hey, in November, we're going to have December, we're going to have a thing that you're going to be able to send the image to your friends.
02:23:16 ◼ ► And in June, they had to make a call, which is do we think this thing is going to be good enough by then?
02:23:22 ◼ ► And maybe they were a little optimistic by how far they would be along by now to be like, yeah, people are going to love when you send these characters or maybe they think it's great.
02:23:37 ◼ ► I made one of my son that really was sort of like an idealized, wow, this is like an idealized version of him with no, no buts.
02:23:46 ◼ ► There's no, yeah, it's pretty good, but it's like, oh yeah, they got his curly hair, they got his face, they got his eyes.
02:23:55 ◼ ► And then I've, you know, I've made others where it's like, oh, I don't even know if I can show this to my wife.
02:24:00 ◼ ► I mean, like a picture of her and it's like, I don't even want to show it to her because it's, yeah, I don't want to, I don't want to go there.
02:24:09 ◼ ► Yeah, it's a challenging place because really with the technology, especially with a local model, so it's not going to be like as powerful as the state of the art stuff.
02:24:18 ◼ ► The real way to get good results from this and the results I posted some like good results from another tool.
02:24:24 ◼ ► I spent a while with it and I played with a bunch of dials and I was like, oh, maybe I would have an anime style, how it looked like.
02:24:47 ◼ ► It's obviously the same or I'm assuming it seems like it's the same model as creates the animation style, Pixar style renderings for the image playground.
02:24:56 ◼ ► I think I'm maybe well positioned in the target market of Genmoji because me and my wife have running jokes about emoji and misuse of emoji.
02:25:08 ◼ ► And so the first thing I did is I sent her the picture of a duck's butt because that's like just an ingest side joke.
02:25:22 ◼ ► So I had to be like, oh, like a picture of a duck from behind looking the other way or something eventually it's like, ah, okay, you can do this.
02:25:36 ◼ ► And then I also, I think I had one or two other ones where I was just like, oh, hey, that's fun.
02:25:40 ◼ ► Like Apple doesn't need to make an emoji for this obscure thing, but it's kind of cute.
02:25:47 ◼ ► But the thing I found that's a little in its current state frustrating is like once I had a couple successes with it, sometimes I think, oh, I'd like to thumbs up or respond to this person with a Genmoji.
02:25:57 ◼ ► But then it takes so long to like generate and then like you have to reject a few of them and you have to iterate your prompt a bit.
02:26:10 ◼ ► But on my phone, it's like pretty quickly I'm like, ah, it's struggling a little bit to make a whatever my vision for this thing was.
02:26:18 ◼ ► The other thing I've noticed is that and I think, you know, it just speaks to the skill of icon artists to make something that scales to a relatively small size.
02:26:35 ◼ ► But if you're typing it in the middle of a sentence, you get the small text type version.
02:26:39 ◼ ► And the Genmoji that I've tried so far at text height, in other words, the small size, they're like smudges because it's very hard to draw a distinctive whatever it is you're doing.
02:26:51 ◼ ► And the funniest things I can think of are all they're not if they were super simple, I would just use the regular emoji.
02:26:58 ◼ ► Well, I tried to send you you suggested the recording time and I tried to send you a head exploding double thumbs up emoji.
02:27:05 ◼ ► Which sounded good, but then like even a professional illustrator would be like, all right, this is a real challenge.
02:27:17 ◼ ► Because the small size matters for the tap back feature where they've added at the same the same year that they finally added pick your own emoji.
02:27:24 ◼ ► Now they're letting you generate your own emoji, but it has to fit in that tiny little size.
02:27:33 ◼ ► Yeah. And that's a little bit of like, certainly they can make the model better over time and making them legible.
02:27:41 ◼ ► But also there's a little bit of I'm asking it for an impossible thing, which is like a really crisp, tiny two thumbs up head exploding emoji.
02:28:14 ◼ ► But you over the last year, you've struck out with a new startup called Forest Walk and Forest Walk is.
02:28:34 ◼ ► So this is obviously like my wheelhouse here is talking about what people can build on it.
02:28:43 ◼ ► And in particular, so far, we've been focused on the tools that are required to test this stuff.
02:28:48 ◼ ► Because as you can imagine, it's very easy to get a demo of the product that's built on these things.
02:28:54 ◼ ► But then you start actually interfacing with customers and customers will put in unexpected things like duck butts and with dairy and fireball blog posts and all sorts of stuff that's outside of your guests of what you as a product team put through it under test.
02:29:07 ◼ ► And so in order to kind of get these very capable but extremely unpredictable models consistently doing great things, there needs to be like a regular rigorous testing program on ideally automated high quality testing program.
02:29:23 ◼ ► Because one of the fundamental, ha, that's not how I'm used to computers working thing about all of this is that the output is non deterministic.
02:29:32 ◼ ► Like you, you have the same system with the same prompt, and you will get different answers.
02:29:39 ◼ ► And it's even worse than that because often when we're doing something, we're using language models to generate an answer instead of reaching for some other more deterministic tool is because it's a subjective thing.
02:30:04 ◼ ► And that makes it harder, the whole domain of how do you do testing like we've solved traditional software testing has become basically a solved problem for 15 years where you see write some unit tests, and you make sure it always does the same thing and it just checks that the answer is 42 and that always is 42 and then your automated thing on GitHub make sure that it's always 42.
02:30:21 ◼ ► And that's just 100 times more complicated when we're building these tools, but also even more useful because if you can actually get some confidence that you're making some changes like you notice some problem about how spouses are referred to in your summarization thing.
02:30:36 ◼ ► And so you add more either examples or change the prompt or whatever to try and make a deal with this, there is 999,000 other use cases where you may have made it worse, or maybe you made it better for this one test case that you're testing but you didn't fix all the various other problems.
02:30:56 ◼ ► And there is you have you have a very succinct five minute video right now on the forest walk website, where people can it's a great example of what you're talking about for anybody who's like, Hey, I think I might need that.
02:31:08 ◼ ► And your example goes through a bunch of that stuff five quick minutes very, very easily understood, but you don't have but to make it clear, you don't have anything to sell or ship yet.
02:31:20 ◼ ► Yeah, well, we're three months in, right. So it's the startup journey, which is many years since I founded Steam Clock. So it's fun to be back in it is every week we learn something that is both like amazing and terrible where it's like, Oh, this assumption that we had isn't true, but actually there's this other opportunity. So it's like building the airplane as it's moving time. So the fun and the joy of doing all that.
02:31:41 ◼ ► Well, I wish you good luck at it. And I think you it's a super useful idea. Because there's always it's like the second step is, oh, here's this new technology. And then the second step is, oh, here's a new way to do developer tools on top of the technology.
02:31:59 ◼ ► Yes, right. Yeah, one that's I mentioned a little bit cursor, which is this AI ID, it's moving so fast that if I had to use like the AI development tools, obviously, there's the testing set, we're working on, but even just like the developer, like ID that you're using to write your code and stuff like that, if I had to use a year ago version of that it would be so painful to have to use or like go back to Xcode or whatever. Oh, man.
02:32:25 ◼ ► All right, moving. Well, I thank you. I will put link to that in the show notes. I have a whole bunch of notes I wrote down. So should be a busy show notes. I thank you so much for taking the time in this busy US Thanksgiving week. You didn't you and I guess I don't need to thank you since it's not really