00:00:30 ◼ ► But yes, everything's cool. He just had a conflict. There's no, it's not baby time yet. As far as I know, it's not, it's not, as far as we know, it's not. I talked to him like 10 minutes ago. He didn't say anything. So, uh, he just had a conflict today. You would think, you would think he would tell you. I would hope so. Yeah. Yeah. He's like, oh, I've had a four year old this whole time. A couple of years ago, a couple of years ago, one of my favorite, uh, video game podcasts, uh, triple click, uh, Jason Schreier, uh,
00:00:59 ◼ ► co-host of triple click and also a reporter at Bloomberg, um, surprised, uh, his two co-hosts with his, uh, second, uh, baby that was actually born months before and he didn't tell them. And it was an incredible moment on the show. Uh, that's awesome. Yeah. That was, that was wild. Uh, but yeah, hopefully Mike will not try to copy that approach.
00:01:22 ◼ ► Yeah. It's like the time you told us that you had been using a PC for like, well, I wouldn't say that was as dramatic as revealing a whole baby, but you know, yeah.
00:01:37 ◼ ► Okay. They're slightly different. Yeah. A little bit, a little bit different. Uh, we got some followup.
00:01:42 ◼ ► That's what we do here at the top of the show. Um, and many people have written in about their own iCloud woes. This was, uh, the bulk of the episode last week, talking about my, uh, my whole iCloud situation with my legacy version of myself.
00:02:04 ◼ ► And the new iCloud family. And I couldn't do the additional space. Um, so it seems like a lot of people are in that situation. People are adding their legacy accounts to their iCloud families.
00:02:16 ◼ ► I will say, you know, a week and a half or a week and a half in, however long it's been. Uh, it seems like the dust is all settled except on my Mac and the Mac app store. I can't update any apps. So.
00:03:22 ◼ ► Uh, a couple of things that I didn't mention that some of them I didn't realize at the time
00:03:44 ◼ ► Um, I mean, thankfully, like I'm friends with most of the people whose test flights I run.
00:04:26 ◼ ► So like, oh, I can just add myself back to Widgetsmith, you know, and underscores other apps.
00:04:30 ◼ ► But, well, uh, this is actually quite perfect because, uh, so, uh, the oldest is TV forecast,
00:04:47 ◼ ► But I just think the, uh, these are like, uh, build expired and build removed because the
00:05:18 ◼ ► Um, so anyways, uh, if I was on your test flight and you'd like me back, please let me know.
00:05:33 ◼ ► Um, our friend Zach had mentioned, uh, they said that, okay, maybe some of these are like
00:05:45 ◼ ► But again, like the test flight thing, blessing in disguise, like, let me just rebuild my library.
00:05:56 ◼ ► And I'm just like, as I'm like finding things, it's like, oh yeah, let me just add this back
00:06:06 ◼ ► And my previous library dates back to like when I first started using iTunes in like 2002,
00:06:13 ◼ ► like a lot of stuff in there I hadn't listened to in a long time that I sort of cut loose.
00:06:17 ◼ ► But first thing, music encoded for Apple music in a lot of cases sounds way better than what I had
00:06:27 ◼ ► Like, you know, I had some ripped CDs and I did iTunes match at some point, but you know,
00:06:42 ◼ ► Um, but the thing that hurt me and Federico, I think you in particular will appreciate this.
00:07:05 ◼ ► But I just noticed as I was like going through adding things, I was like, oh, oh, oh goodness.
00:07:20 ◼ ► Uh, so test flights, uh, Apple music, um, Mac app store, Mac app store and anything else?
00:07:39 ◼ ► Um, I watched a couple episodes, a couple of the first couple episodes of Mr. Robot over
00:08:11 ◼ ► Hey, if you miss Mike on, on this week's episode, here's, here's, here's something that you can,
00:08:22 ◼ ► Uh, it can be chat GPT, can be Gemini, can be DeepSeek, which we're going to talk about.
00:08:28 ◼ ► Well, it depends on what country and ask your favorite AI product to put together a short description
00:08:39 ◼ ► You know, you know, what, what, uh, if Mr. Robot could feature Mike Hurley as a recurring character,
00:09:03 ◼ ► Mike could play a podcaster or radio personality who operates a secret channel spreading anti-corporate
00:09:23 ◼ ► Given his real world experience running a media business, Mike Hurley could be a high-ranking
00:09:28 ◼ ► E Corp executive in charge of internal communication who slowly realizes the company's dark secrets
00:09:44 ◼ ► In a meta twist, Mike could play a fictionalized version of himself hosting a tech podcast that
00:09:50 ◼ ► dissects the rise of E Corp and the decline of privacy, inadvertently influencing key players
00:09:58 ◼ ► Uh, and then the final one, a surreal take, uh, Elliot's consciousness voice where Mike
00:10:04 ◼ ► is never physically present, but his voice is, his voice constantly plays in Elliot's mind
00:10:59 ◼ ► And if you are a member of relay, which you can do, there's a link in the show notes to
00:11:07 ◼ ► All memberships come with access to crossover, which is this feed where we, we publish, uh,
00:11:15 ◼ ► And in February, I'm going to be interviewing JD Davis, the designer who redid relay's branding.
00:11:24 ◼ ► I'll ask for questions in February, but, um, looking forward to that conversation with JD.
00:11:39 ◼ ► I really like this change and it looks, it looks very good at small sizes and it's still
00:12:00 ◼ ► I did want to take just a second, um, and kind of talk about the times that we live in.
00:12:08 ◼ ► We were going to do this last week, but the three of us felt like we wanted a few more days
00:12:14 ◼ ► Um, we know that a lot of people in and beyond our community are hurting given the state of
00:12:27 ◼ ► Um, and we want to do what we always have done, uh, provide a place to hang out, talk about
00:12:41 ◼ ► And in that community, there's, uh, a strong shared belief that everyone should be respected
00:12:53 ◼ ► Um, and one thing that it means is that there's no room for hatred of other people in our community.
00:12:59 ◼ ► And unfortunately, feelings like that, uh, towards, uh, individuals has become more mainstream and
00:13:08 ◼ ► And we want to be a place that we can, again, hang out with our friends, talk about the things
00:13:13 ◼ ► we love and be in an environment where we can be, uh, where we can know we can be safe, whoever
00:13:20 ◼ ► And, uh, you know, we're not turning our shows, uh, into anything that they're, they're not
00:13:36 ◼ ► And we want to use our platform where we can to, uh, protect those who are in our community.
00:13:43 ◼ ► If I may just add some, uh, personal, uh, context, um, these are my personal opinions, but I feel
00:13:56 ◼ ► So, uh, I want to start from my personal belief, uh, that trans rights are human rights.
00:14:11 ◼ ► And I cannot even, because I cannot imagine, I, I personally cannot imagine what it must feel
00:14:18 ◼ ► like to, to feel like, uh, other people want to make you and your identity feel unjustified
00:14:33 ◼ ► I cannot imagine that, but I, I, I can relate to that feeling of feeling powerless and feeling
00:14:48 ◼ ► And to an extent, I, you, Mike, we, at this network are, uh, are, are, you know, it's not
00:15:05 ◼ ► Um, but there's a couple of things that I will, that I would say, um, now more than ever,
00:15:14 ◼ ► I feel like, and unfortunately these discussions are trickling down to Italy as well, you know,
00:15:20 ◼ ► because we have a government who very much sympathizes with, with, with the, with the American government.
00:15:32 ◼ ► Uh, but I feel like now more than ever, uh, it's important if you feel like it to, to be, to be organized, to dissent and to, and to seek a safe space in real life and online.
00:15:46 ◼ ► Um, and I think, um, you know, um, I think it's important to find your people and to, and to not accept what is going on.
00:15:57 ◼ ► Um, and I know that, um, and I know that I am saying this from a position of, of privilege myself.
00:16:03 ◼ ► Um, because I, you know, my identity is not at risk and, but I think it's important for people to be together and to feel those things together.
00:16:14 ◼ ► And the second thing is that from my position and I think from our position, um, it's important to continue to provide the, the utility.
00:16:34 ◼ ► If, if every so often we, we share something useful, that's also my hope, but I think it's, it's important.
00:16:47 ◼ ► It's important to at the same time, keep doing what we do because in, in any dark time, you still need to find that light that never goes out to, to quote a famous song.
00:17:02 ◼ ► Um, I think it's important to, to continue to provide that service for people who want it because like, you know, I, I think in any, in any,
00:17:11 ◼ ► difficult period of your life, uh, you need to have something that brings you a little joy.
00:17:33 ◼ ► Um, and this is like, this is one of those moments where I really miss not having a live show and seeing people in person.
00:17:53 ◼ ► It is hard to, it is hard to put myself, uh, in the place where I'm, I'm told that who I am shouldn't exist.
00:18:04 ◼ ► But that doesn't mean that we can't, um, first of all, empathize with it to the degree that we can, but also, um, provide a place where everyone can feel safe and comfortable and welcome.
00:18:21 ◼ ► And look, our, our discords are the best places on the internet and our communities are incredible.
00:18:44 ◼ ► Well, not socks, uh, but we are going to talk about iPods and, and more specifically, uh, Steven,
00:18:50 ◼ ► I wanted to ask you what's your budget for, um, iPods looking like these days, uh, because if you have an opening, uh, you could participate in the, uh, Sotheby auction for the custom iPods from the late car Lagerfeld.
00:19:20 ◼ ► Um, of all kinds, including a, a, the, the, like this diamond encrusted iPod and microphone, there's multiple color variations.
00:19:37 ◼ ► So this entire story, uh, that this collection is now, uh, up for, uh, sale at an auction is new to me.
00:19:50 ◼ ► I mean, I was familiar with the name, you know, uh, seeing Carl's name float around, but had no idea.
00:20:22 ◼ ► There's some custom, custom, like colorways of the first gen nano, which Apple just shipped
00:21:07 ◼ ► I saw some posts and some old like stories saying that he was basically treating iPods as, um,
00:21:17 ◼ ► So each iPod would be loaded with a specific type of music or a specific collection of albums
00:21:23 ◼ ► and so forth, which, I mean, if you are really into music and the storage limitations of the
00:21:30 ◼ ► time where, you know, the storage limitations at the time, I can sort of see that, you know,
00:21:35 ◼ ► if you have an unlimited budget and you want to be fashionable and you are a fashion icon,
00:21:45 ◼ ► going to accumulate hundreds of iPods and each of them, I will treat as a cassette tape.
00:21:52 ◼ ► But what was interesting is that apparently, uh, Lagerfeld had a whole team of people, uh,
00:21:58 ◼ ► dedicated to managing these iPods because obviously you needed to sync them with, with iTunes.
00:22:04 ◼ ► And so the story goes that there was an entire raid of XServ to run this massive iTunes library
00:22:28 ◼ ► Um, but yeah, uh, terabytes of music, which, I mean, think about the timeframe of the iPod,
00:22:53 ◼ ► On one hand, like, I feel like I, I shouldn't be allowed to judge because like I do keep
00:23:20 ◼ ► Um, I don't know though, it's what a power move to have hundreds of iPods servers to manage
00:23:33 ◼ ► Uh, there are also some modified, I just found them some modified iPod minis, including an
00:23:58 ◼ ► You could, you could, you know, you could touch it and be like, this iPod was also touched
00:24:20 ◼ ► And those have like stickers on the back with notes, I guess, with maybe what was on them.
00:24:32 ◼ ► And I've always wanted to get into a bidding war with somebody, but that only ever happens
00:24:40 ◼ ► Like getting into a taxi and be like, follow that car, but it's a random car and you got to pretend
00:24:45 ◼ ► that it's important, you know, that sort of, that sort of thing, but also like attending
00:24:54 ◼ ► Like it doesn't have to be important and you have to be like really and oddly into it, you
00:25:42 ◼ ► In addition to blocking obtrusive ads, OneBlocker can block trackers, annoying pop-ups, EU cookie
00:25:55 ◼ ► It has an updated design with an entirely new interface to make the app even more intuitive.
00:26:02 ◼ ► Plus, you can expect improved blocking and free ad blocking because users can now enable one filter at no cost, including advanced blocking for YouTube.
00:26:12 ◼ ► Then upgrade to premium to unlock all the filters and automatic weekly cloud filter updates.
00:26:22 ◼ ► One of my favorite things about it is you can create custom rules for domains that may not fit in in OneBlocker 6's other options, but their built-in options are extensive and the new redesign makes it easier to use than ever.
00:26:35 ◼ ► OneBlocker was featured in the Mac App Store by Apple under the best Safari extensions, and it's available for the iPhone, iPad, Mac, and Vision Pro as a native app.
00:26:45 ◼ ► You can get premium for just $1.24 per month, billed at $14.99 a year, or go for a lifetime license.
00:26:54 ◼ ► And a single purchase unlocks OneBlocker across all your Apple devices, so you can share premium with up to five family members.
00:27:01 ◼ ► For unlimited time, OneBlocker is offering listeners of Connected one month of premium for free.
00:27:08 ◼ ► Premium unlocks all features across iOS, macOS, and Vision OS, and can be shared with up to five family members.
00:27:20 ◼ ► That's OneBlocker.com slash connected, the link is in the show notes, and use the code CONNECTED for one month free.
00:27:46 ◼ ► And it's going to be a long segment, and Stephen allowed me to do my research and to take my time.
00:27:54 ◼ ► So, hey, I think it's going to be an interesting one and a fun one, but if you're not interested, well, that's your problem, not mine.
00:28:05 ◼ ► It's going to be exciting because, yeah, it's going to be an exciting moment for the show.
00:28:11 ◼ ► So, in case you haven't seen it, or in case you've seen this name repeated to death and you couldn't be bothered to look into it,
00:28:20 ◼ ► DeepSeek is this new large language model, this new chatbot that is supposedly rivaling in performance OpenAI, which had GPT, Gemini from Google, Claude by Anthropic.
00:28:46 ◼ ► It's very hard to pin down exactly, also because of the language barrier, and also because everybody's saying different things.
00:28:58 ◼ ► DeepSeek was put together, more specifically, the underlying large language model is called DeepSeek V3.
00:29:05 ◼ ► This is a word that I learned this week by QUANT, which stands for Quantitative Hedge Fund.
00:29:15 ◼ ► It's basically a hedge fund in China that uses sort of like advanced mathematical operations and whatnot to track market movements and that, you know, whatever hedge funds do.
00:29:42 ◼ ► So, this company, they were using machine learning and large language models to predict, you know, market trends and stocks and all, you know, that fancy money stuff.
00:29:57 ◼ ► And so, the story goes that they started at some point a couple of years ago as a side project training a large language model called DeepSeek that they were going to use in their main business.
00:30:18 ◼ ► Then, last year, they started training with a lower number of NVIDIA GPUs than are typically used in America, in American AI labs to train large language models.
00:30:35 ◼ ► They started training version 3 of their DeepSeek large language model with a reasoning version called DeepSeek R1.
00:30:52 ◼ ► They have published a white paper, and this model has open weights, which sort of means open source, but I don't kind of want to get into that.
00:31:11 ◼ ► There's a white paper that you can read, and it goes into the details of how it was put together.
00:31:23 ◼ ► When you go to the DeepSeek website or you're using the DeepSeek app on your phone, you can enable R1, which shows you, it takes a little bit longer,
00:31:41 ◼ ► It shows the model, quote-unquote, thinking and trying to understand your problems and basically thinking out loud about the query that you just asked.
00:31:50 ◼ ► Now, most people haven't been exposed to a reasoning model because both Google and OpenAI, I believe they only make it available if you pay for either Gemini Advanced or ChatGPT Plus and Pro.
00:32:05 ◼ ► So ChatGPT, which is obviously the most popular in the world, they don't give you a reasoning model because you've got to pay, and most people don't pay for ChatGPT.
00:32:18 ◼ ► So what's impressive here, according to the theoretical story of DeepSeek, is that the company used a much lower number of GPUs.
00:32:46 ◼ ► There's a, you know, there's this new export laws in the United States that were put forth by the Biden administration, I believe,
00:33:05 ◼ ► So through, you know, the cluster of GPUs that DeepSeek already had before the laws were enacted,
00:33:16 ◼ ► apparently through black market channels, they were able to, so some people are saying 2,000 GPUs,
00:33:29 ◼ ► But apparently, it's a much, much, much lower number than what, again, supposedly OpenAI and Anthropic are using.
00:33:37 ◼ ► Like, I saw someone reporting that OpenAI is using half a million NVIDIA GPUs in their data centers.
00:33:44 ◼ ► So the theory would be like, imagine that for one-tenth of the horsepower and one-tenth of the cost,
00:34:16 ◼ ► Well, I was going to say, there's speculation that maybe they didn't do it all on their own, right?
00:34:42 ◼ ► Like, it's not like DeepSeek, they invented the transformer model or the idea of a large language model.
00:34:50 ◼ ► They built on the foundation of something that was invented by American companies, right?
00:34:59 ◼ ► I believe the first paper on the transformer model is, you know, it's credit to Google engineers for that.
00:35:17 ◼ ► This is what's fascinating, that DeepSeek optimized the foundation created by American companies.
00:35:24 ◼ ► And they did that by applying constraints to their engineering teams that were forced to work with less power and less money.
00:35:36 ◼ ► And they did that by basically sort of putting a unique spin on how large language models can be trained.
00:35:46 ◼ ► I'm going to try and simplify here because it gets boring fast and also because, like, I'm not an engineer.
00:35:52 ◼ ► But my understanding is that typically there's a lot of supervised training where humans are actually supervising the model and, like, feeding the model the correct answers and the correct process when they're training.
00:36:07 ◼ ► There's supervised training and then there's reinforcement learning where basically you are applying a reinforcing technique to say, yes, model, good job.
00:36:21 ◼ ► Basically, DeepSeek only used the reinforcement learning to train their larger language model.
00:36:43 ◼ ► Apparently, OpenAI is now speculating that DeepSeek used training data from ChatGPT to train DeepSeek V3.
00:36:56 ◼ ► So, basically, the news of the day is that OpenAI is mad because DeepSeek used data from ChatGPT to train their model, which is kind of ironic where the company that scraped terabytes and terabytes of data from the open web is now upset that a Chinese startup scraped their data to train their model.
00:37:27 ◼ ► Anyway, the result of the debut of DeepSeek last week, DeepSeek, so DeepSeek V3 launched in beta last month, came out officially last week alongside the reasoning version R1.
00:37:42 ◼ ► By that, I mean that it became the most downloaded app on the App Store, that the entire tech and AI industry lost their minds.
00:37:52 ◼ ► Because, like, they were like, how can this Chinese company, that's coming out of nowhere, spoiler, it's not coming out of nowhere, people who are really into AI knew what DeepSeek was up to.
00:38:07 ◼ ► But in any case, as a result of the debut of DeepSeek, they basically took how many hundreds of billions of dollars off the U.S. market?
00:38:23 ◼ ► So, yeah, we're looking at basically, like, if you consider the NVIDIA stock that was down 17% or something.
00:38:56 ◼ ► And if I were into conspiracy theories, I would kind of believe it, which is, like, wouldn't it be fun if you like this entire narrative?
00:39:05 ◼ ► Because you could say, like, what if this entire narrative about DeepSeek doing this for cheap and doing this with, like, one-tenth of the power?
00:39:18 ◼ ► But this little story that they put together was a tactic by the Chinese government to put a little dent into the NASDAQ, into the U.S. market, and wipe off a little, almost a trillion dollars in a single day.
00:39:53 ◼ ► Like, we're going to get to this, but, you know, many, well, some countries in the U.S. Navy are like, hey, don't use this.
00:40:02 ◼ ► This is very much wrapped up in those international politics that are above our pay grade.
00:40:10 ◼ ► But, you know, there is an element to consider that, you know, what they say should be taken with a grain of salt until it's verified by other people.
00:40:19 ◼ ► Just as the same as we should verify what OpenAI and Google and these other companies say, right?
00:40:25 ◼ ► Like, these AI companies make really grand statements and you've got to put them to the test.
00:40:32 ◼ ► But, so, even if we set the potential politics of it all aside, I think it's interesting to look at DeepSeq and specifically the white paper that the company published.
00:40:45 ◼ ► Because it does paint an interesting picture for the future of American companies and the future of large language models and also the future of open source.
00:40:55 ◼ ► So, it is undeniable that the DeepSeq team used a series of really creative approaches to build this large language model.
00:41:08 ◼ ► So, for example, they were able to work with one-fourth of the memory consumption because instead of using 32-bit floating point operations, they used 8-bit floating point operations.
00:41:21 ◼ ► So, that's like four times less RAM when you're running this in your data center, which is fascinating.
00:41:29 ◼ ► Using reinforcement learning and with some, so basically, like, I was listening to a podcast about this yesterday.
00:41:40 ◼ ► They were convinced that they could do just reinforcement learning without supervised learning, but then they realized that the model was not responding correctly and it was continuing to, like, mix and match English and Chinese in the same sentence.
00:41:54 ◼ ► And so, for DeepSeq V3, they started from scratch again and they did some, like, they did all reinforcement learning with some supervision in the final stages of the process.
00:42:05 ◼ ► So, even the story that it was entirely based on reinforcement learning is not correct if you read the white paper.
00:42:11 ◼ ► So, really some interesting approaches that, as a result, like, there is a product that you can go there, you can go to DeepSeq on the web, or you can download an app on your phone, and you can talk to it.
00:42:25 ◼ ► And I can tell you, because I have been testing all of these things, I can tell you that the performance of DeepSeq, it most definitely rivals O1 and the latest Google Gemini version 2 advanced experimental models.
00:42:46 ◼ ► So, the performance is up there, now, does that performance come from scraping the same data sources as OpenAI and Google?
00:43:05 ◼ ► And I think there's a few interesting takeaways, and also some other things to consider, which I'll get to in a minute.
00:43:16 ◼ ► I think it's undeniable that this large-language model and this product was built because of the work that had previously gone into large-language models in America.
00:43:52 ◼ ► And I think it is undeniably eye-opening for American companies that maybe you can achieve.
00:44:02 ◼ ► And, you know, people who have been keeping an eye on the open-source AI scene knew that this was going to happen.
00:44:08 ◼ ► If you follow, you know, if you follow blogs and podcasts and YouTube channels about this stuff, you knew that this was months in the making.
00:44:17 ◼ ► This idea of a free and open-source model coming out and matching the quality of all one.
00:44:22 ◼ ► It's been like, even the people who saw DeepSeq a couple of months ago, the V3 beta said, oh, when this launches, it's going to be a big one.
00:44:36 ◼ ► And I think, realistically speaking, what's going to happen here is that OpenAI will have to move up the timeline for the release of Chagipity O3.
00:44:53 ◼ ► And O3, by the way, is the even more advanced reasoning model that they announced in December.
00:45:05 ◼ ► OpenAI will also, they also said they will make O3 Mini, which is the smaller model based on O3, they will make it available for free for maybe or at least up to 100 free queries per week.
00:45:23 ◼ ► And I think on the Google side, because like these are the two biggest players right now, right?
00:45:36 ◼ ► And I think what Google will need to do, Google will need to make an even bigger deal of their reasoning model, which is Gemini Flash 2.0 thinking.
00:45:54 ◼ ► But I think Google, the whole interaction with Gemini, when you have to pick a model from a drop-down menu, is just as bad as OpenAI.
00:46:08 ◼ ► And right now, I think both Google and OpenAI will need to clean up their list of models and have something simple.
00:46:17 ◼ ► Because DeepSeq is showing that you open, you have one model, you tap a button that says, do you want to have the reasoning one or not?
00:46:26 ◼ ► And so I think both Google and OpenAI will need to simplify and make available to people a thinking model of some kind.
00:46:37 ◼ ► I think a lot of people are under the assumption that Google Gemini is as bad as Google Bard used to be.
00:47:24 ◼ ► There's another conversation to be had about the safety practices of DeepSeq and the idea of censorship by the Chinese government and the CCP sort of looming over DeepSeq.
00:47:42 ◼ ► First of all, it seems pretty much clear that the folks at DeepSeq don't have a safety team, don't have any safety practices in place in terms of like how exactly was DeepSeq trained, especially if DeepSeq trained itself via pure reinforcement learning.
00:48:03 ◼ ► They haven't published anything in terms of like how safe it is, what kind of content was it trained on.
00:48:18 ◼ ► And also, try and ask anything about the Chinese government or Chinese history or Taiwan, Taiwan independence or the Tiananmen Square massacre.
00:48:30 ◼ ► And you will be able to see in real time DeepSeq R1 exposing its thoughts and censoring itself in real time.
00:48:44 ◼ ► You will literally see the chain of thought stop once the model realizes that you're asking about the Chinese government or Chinese history.
00:48:58 ◼ ► What's interesting, though, is that DeepSeq is open source and you can host it somewhere else.
00:49:12 ◼ ► Now, when you run the model yourself or when you're using DeepSeq hosted by somebody else, for example, I believe Perplexity, they are hosting DeepSeq R1 in the United States.
00:49:35 ◼ ► It'll happily tell you about the oppressive Chinese government and the Tiananmen massacre in the past.
00:49:44 ◼ ► So interesting that obviously DeepSeq, when it's running on the DeepSeq service in China, it's censored.
00:49:55 ◼ ► And obviously, like with DeepSeq, like most people are not going to download DeepSeq and run it on their computers.
00:50:05 ◼ ► Which is why, for example, the Italian privacy watchdog, which is an entity of the Italian government, has banned DeepSeq from the app store.
00:50:19 ◼ ► And they have issued a warning to DeepSeq and the parent company in China, asking for details about privacy collection, user data collection, and user data retention, and about their privacy policies.
00:50:44 ◼ ► So this kind of story, I think you will see it happen in more and more countries in the next few days.
00:50:55 ◼ ► Even though people who are tuned into the AI industry knew that this was going to happen.
00:51:08 ◼ ► And so this kind of, like, governments getting into this and be like, hey, hold on a second.
00:51:22 ◼ ► And, I mean, the upside to this sort of thing is, like, if these models can be created and trained on less hardware, that's good.
00:51:37 ◼ ► I can't help but think that some of the freak out over this, at least in the U.S., is that, oh, this has been an American company thing.
00:51:53 ◼ ► But I'll say this, that undeniably, the DeepSeq team, they applied some really clever engineering constraints to come up with DeepSeq V3 and DeepSeq R1.
00:52:15 ◼ ► And they, for sure, have documented some fascinating techniques that, because of open source, I'm sure others will copy and implement.
00:52:37 ◼ ► But, if you're running a product that hundreds of millions of people use, you still need to, you still need to run this somewhere.
00:53:15 ◼ ► Like, you still need to, like, you still need to, if you really want to be the next model, you still need to match the arguably incredible performance of American models.
00:53:28 ◼ ► Because they have a whole impressive, like, you know, set aside the fact that it's bad for the environment.
00:53:35 ◼ ► But when you look at it as an object, it's undeniable that it's, you know, something that is scaling to how many hundreds of millions of people and how many billions of requests per day.
00:53:58 ◼ ► I saw some reports saying that they are using these NVIDIA GPUs that are not the H100s that, like, most companies are using are the H800s, which are smaller and not as powerful.
00:54:25 ◼ ► Once again, I think the most important aspect to understand is that DeepSeq optimized something that already existed.
00:54:36 ◼ ► I think, though, that the angle that OpenAI and Google will use to continue justify their capital and operational expenses will be, well, but if you want to push the bleeding edge of this stuff, like, if you want to come up with the next new thing that then others will copy, we'll need the money and we'll need the horsepower.
00:55:02 ◼ ► So, it'll be interesting to see how this shakes out, like, is something like DeepSeq enough for most people?
00:55:15 ◼ ► Like, that 500 billion project, you know, backed by Microsoft, by Oracle, by the US government and others, like, building a massive data center.
00:55:28 ◼ ► And that's why it is scary to have this little Chinese startup come up and say, hello, we have this and it's free and we spent less than $10 million on it.
00:55:45 ◼ ► Yeah, and the timing really couldn't be worse for some of those people asking for billions of dollars.
00:55:55 ◼ ► What is also fascinating here in this multi-layer AI onion that we just unpeeled is the Apple angle.
00:56:37 ◼ ► Because they want to roll out Apple intelligence in China, but they cannot because the Chinese government doesn't allow models like ChatGPT or Gemini.
00:57:01 ◼ ► The first and more obvious one, I think, is that Deepsik could be the in that Apple was looking for to be able to offer Apple intelligence in China with an extension.
00:57:17 ◼ ► Basically, the equivalent of the chatGPT extension in the U.S. and other markets where Apple intelligence is available.
00:57:27 ◼ ► A Chinese company with a large language model, high performance large language model, could be the next extension for Apple intelligence to be approved in China.
00:57:38 ◼ ► It would also be fascinating to have another approach and to see Apple acquire Deepsik.
00:58:04 ◼ ► But it would be interesting for Apple to say, well, to kickstart our own large language model, maybe we could acquire this.
00:58:14 ◼ ► And it would fit Apple's approach of like, it's high performance, it's more cost effective, it's more energy, like it consumes less energy than an equivalent to chatGPT or Gemini.
00:58:37 ◼ ► Although I have also long thought that eventually Apple will acquire Mistral, which is the France-based AI company.
00:58:48 ◼ ► But I do think that eventually Apple will need to acquire some kind of large language model.
00:58:58 ◼ ► But I think more realistic, I could see a Deepsik integration in China to get Apple intelligence out the door in that market.
00:59:05 ◼ ► I think I pretty much covered it all, except, oh, there's one final thing that I think it'll also be something that we need to keep an eye on in the short term, which is the, oh, there's actually a couple more things I want to say.
00:59:25 ◼ ► The first one is, I saw this article on the information about how a bunch of different businesses and what do they call it, Stephen, SaaS companies?
00:59:37 ◼ ► A bunch of these SaaS companies are like jumping ship to the Deepsik R1 API because it's much cheaper to run than OpenAI or Google or Anthropic.
01:00:09 ◼ ► To be fair, Anthropic is the only company that's actually thinking in a more serious way about safety.
01:00:16 ◼ ► I do think, I saw some people say that, and this is where I disagree with the Apple community consensus.
01:00:26 ◼ ► I saw people say, oh, actually, Apple being late to the game with larger language models is a good thing.
01:00:54 ◼ ► Because let me tell you, if Apple were not, if Apple hadn't been caught flat-footed in this space,
01:01:15 ◼ ► You would think that if Apple could, if they were in a position to do this, they would say, nah, we're good as an aggregator.
01:01:35 ◼ ► And second, the whole idea of aggregation, if you're just a, if you put the value in Apple as being a distributor of apps
01:01:53 ◼ ► you know what, today, the more I use these AI tools, it's never been easier for me than it is today to switch from an iPhone to an Android device if I wanted to.
01:02:08 ◼ ► And that's because all the work that I've done in Chagipity and Cloud and Gemini, it's all based on a web service.
01:02:57 ◼ ► It's not necessarily the fact that they are behind in large language models because I could see Apple coming out late and offering something that is, you know, powerful and much more integrated.
01:03:10 ◼ ► The greatest threat to Apple right now is the web and the fact that all of this is web-based.
01:03:16 ◼ ► And web-based means inherently it's portable and you can take your data and you can take your identity and you can take your projects and it doesn't matter the phone that you're holding.
01:03:30 ◼ ► The web is also the number one thing they should be worried about as a platform maker, right?
01:03:37 ◼ ► I mean, just looking at the things on my dock, a whole lot of them, just web apps and thin wrappers, right?
01:03:44 ◼ ► Like, there is a world that could be coming where being a vendor of an SDK doesn't matter as much as it does now.
01:04:01 ◼ ► I think over the past five years, we were looking somewhere else, you know, in the Apple community and we haven't kept an eye on what exactly was going on in the web industry and in progressive web apps.
01:04:21 ◼ ► And that's, you know, I think in hindsight, 20 years from now, we'll maybe look back at this and be like, that's a problem that Apple ignored.
01:04:29 ◼ ► And it, and it calcified into what became a big issue for them, which is developers and a whole new generation of young developers being fed up with the rules of the app store.
01:04:41 ◼ ► I mean, like, you know what, I'm going to build whatever I want to build and I'm going to do it in Google Chrome and in a web browser and on the web when there are no rules.
01:04:51 ◼ ► And the progress that has gone into the kinds of modern web apps that you can build today.
01:04:58 ◼ ► Look, look, look at all these services that are now taken for granted in any workspace.
01:05:06 ◼ ► I saw yesterday a post by someone who was running DeepSeq in a web browser with JavaScript-based acceleration.
01:05:24 ◼ ► Yeah, so I think the whole idea that, I know, actually, this is great for Apple because it, you know, ties Apple to this identity of the app store as an aggregator of the best stuff.
01:05:43 ◼ ► You go to the Google Play Store, like, but do you, like, I don't think of Google as the Google Play Store.
01:05:58 ◼ ► What part of Apple's business has the most risk of being blown up by regulation around the world?
01:06:23 ◼ ► Not just because of the threat of web-based products that doesn't really matter which device you're using.
01:06:41 ◼ ► Like, what happens if the best models are not close proprietary models, but they are open source models?
01:06:53 ◼ ► There, you know, anyone who's been following this space for the past few months will know that there's a curve of progress to the open source community and to the open source models that's only going up and up and up.
01:07:07 ◼ ► And it's, you know, I can see why, you know, Sam Altman puts up a good face on Twitter.
01:07:13 ◼ ► But, you know, did you see the report that, for example, at Meta, they have assembled multiple war rooms to figure out how exactly DeepSeq built the model?
01:07:25 ◼ ► And I can see that because it's, you know, and Meta is also doing open source, obviously.
01:07:33 ◼ ► Man, it's been an, even if DeepSeq, you know, the story ends up being that actually they had more GPUs than they said.
01:08:03 ◼ ► It'll have consequences in, you know, it'll ripple out to the entire industry and it'll be fascinating to see.
01:08:08 ◼ ► But it'll also be fascinating to see, since we are an Apple show, what Apple does here.
01:08:12 ◼ ► And I am convinced that I think the most realistic option is that they will partner with DeepSeq in China.
01:08:50 ◼ ► And our friend Rob has made a new tool that lets you design your own outro for the podcast.