0:00:00[Music]
0:00:08From Relay FM This is Connected episode 444.
0:00:12Today's show is brought to you by our excellent sponsors ExpressVPN and Indeed.
0:00:16I'm one of your co-hosts Federico Vittucci and it's my pleasure to introduce Mr. Stephen Hackett. Hello Stephen.
0:00:22Hello Federico, how are you?
0:00:25I am doing fantastic.
0:00:27Good.
0:00:28It's good to be back, episode 444.
0:00:30- It's got a nice ring to it, done it.
0:00:32- It's nice.
0:00:33- We're also, two things,
0:00:34we're coming up on 500 really quickly,
0:00:36and also our 10th anniversary coming up really quickly.
0:00:39It's wild.
0:00:40- Wait, what?
0:00:41Oh my God, okay.
0:00:43- And I am joined by Mr. Mike Hurley.
0:00:46- Hello.
0:00:46I think that like 56 weeks is not that quick.
0:00:51- Oh, is that how many we actually have?
0:00:53Yeah.
0:00:54- Well, you said we're coming up on 500 quickly,
0:00:56and it's 444.
0:00:57I mean, it's not that far.
0:00:59- 450 is sooner.
0:01:01- That's true.
0:01:01- That's coming up pretty quick.
0:01:03- 450 will be near,
0:01:04now it'll be before-- - Closely I checked.
0:01:06- WBC.
0:01:07When's 500?
0:01:08Is that near WBC next year?
0:01:10- I mean--
0:01:11- What's 56 weeks?
0:01:12- I don't know.
0:01:14- Yeah, well that's given that we do an episode a week,
0:01:17which sometimes we take off during Christmas,
0:01:19which is how Upgrade passed us.
0:01:21- Yeah.
0:01:22- Now I cannot count in weeks.
0:01:25Like when you say something in weeks,
0:01:27- This is one of the things that Siri is actually good at
0:01:30is like how many, like what's the day,
0:01:33X amount of days from today or weeks from today or whatever.
0:01:36I do that kind of stuff a lot and it's good.
0:01:38- It's like, you know when you have those parents
0:01:40and they have a kid and you're like,
0:01:42for example, like how old is she?
0:01:46And some parents go like, oh, she's 72 months.
0:01:49Like what does that mean?
0:01:50Give me like a year.
0:01:53- Yeah, I think after 12 months, stop doing that.
0:01:56But no, but there's people who do that.
0:01:58Like it actually happened to me like a few months ago.
0:02:02I heard someone say, "Oh, she's 44 months old."
0:02:06Like, wait, what?
0:02:07- I don't know what that means.
0:02:08- Episode 500 will be around May 1st next year.
0:02:12- Okay.
0:02:13- Ish.
0:02:15May the first be with you.
0:02:17- So we have to take like five weeks off or something?
0:02:20- Can't do that.
0:02:21- Okay.
0:02:23- I got bills to pay.
0:02:24That was the beautiful thing where me and Brad hit episode 500 on the exact day of our
0:02:3110th anniversary for the Pan-Edict.
0:02:33It was incredible.
0:02:34Like, it just lined up.
0:02:36Basically, we'd spotted it.
0:02:37We usually take an episode off over Christmas, right, like we do on this show.
0:02:40And it worked out.
0:02:41If we didn't do that, if we just did an episode every single week and noticed it like months
0:02:46before we were going to hit it perfect.
0:02:48And it was, that was awesome.
0:02:49It's like the exact day.
0:02:52I did realize while recording today that next week I've been podcasting for 13 years.
0:02:58Wow.
0:02:59Now, horrible thought.
0:03:01That's such a long time.
0:03:03A bunch of people wrote in, Stepp and Ben being two of them, that I'm basically doing
0:03:07my AirPod shuffle backwards, right?
0:03:10So I talked about I take one out, put it in the case, get the other one, put it in, and
0:03:14the audio doesn't restart?
0:03:15Mm-hmm.
0:03:16Wait.
0:03:17No, that's what I should do.
0:03:18Oh my god.
0:03:19What I'm doing is putting them in both ears and pulling one out.
0:03:21So anyways, it's very confusing.
0:03:24So what are you actually saying here?
0:03:25Okay, so what I've been doing, I take, I'm wearing one AirPod, right, because I don't
0:03:31want to get hit by a car.
0:03:32But I want to switch ears for some reason.
0:03:34What I've been doing is putting the second one in, and then taking the first one out,
0:03:39and then it pauses the audio, and it won't restart.
0:03:42Yeah, right.
0:03:43Because it's the ear detection thing.
0:03:44What I should do, which everybody wrote in, I think y'all said this maybe, but maybe we
0:03:47We didn't, I didn't go back and listen, but I should take the one out and then put the
0:03:52other one in and then it should pick back up.
0:03:55So anyways, I'm going to try that next time.
0:03:57I'm not sure that you know what you're trying.
0:04:00Maybe not.
0:04:01I gotta say, I don't follow here.
0:04:06Like I'm so confused.
0:04:07Yeah.
0:04:08Here's the thing.
0:04:09I don't understand.
0:04:10I don't think Stephen understands.
0:04:12So how about this?
0:04:14You report back next week.
0:04:17I feel like I'm staring, I'm staring at this text.
0:04:20And the more I try to read it, the more I keep drifting off.
0:04:25First take the AirPod out of your ear and place it in the case.
0:04:28The next one plus is the audio.
0:04:29You don't put the second...
0:04:31Oh, okay.
0:04:34I think I follow.
0:04:35If you take the AirPod out and then put the AirPod in,
0:04:39I don't think it does restart.
0:04:41Hmm. So maybe there is no fix.
0:04:43What AirPods do you have?
0:04:44The AirPods Pro 2, the newest ones.
0:04:46Right, well then do what Step says. You just squeeze the AirPod and it will restart again.
0:04:52I think I have all that turned off, I should look.
0:04:54Why? It's the best feature!
0:04:56I don't like squeezing my ears!
0:04:58No you're not squeezing the ears! You're squeezing the AirPod!
0:05:01Merzy you've been getting it wrong the whole time.
0:05:04What you need to do is...
0:05:06Why is squeezing my earlobe not doing anything?
0:05:08Yeah, why doesn't it do anything?
0:05:09Apple's doomed!
0:05:10Just squeeze the stem and it restarts the audio again.
0:05:14Okay, okay. I will do some practicing and I'll report back, please
0:05:18Okay, one of you want to take this next one?
0:05:21So I made reference to the delayed game is never good bad game is always
0:05:27Rubbish quote or whatever the quote is from yes
0:05:30Not Shigeru Miyamoto
0:05:32Max wrote in to say I recently did an interview with a person that sparked the hunt for the origin of the Miyamoto quote about
0:05:38delaying games
0:05:39We chatted about the quest to find the source and the twist and turns that it took plenty of links in the original threads and sources
0:05:45If you'd like I am going to listen to this. This is a part the max frequency podcast
0:05:49Because I want to know this I got I was familiar recently that someone was trying to find it out
0:05:55Which is how it ended up becoming a thing that oh it turns out. He didn't actually say it
0:05:59So this sounds like a good history of that. So I'm gonna
0:06:02Okay recommend that connected listeners do too we also spoke about voice isolation
0:06:09And we got an email from Anthony who said that they were a FedEx driver
0:06:14And this so they drive this like loud diesel rattly truck and they have used voice isolation since the beta and it's like
0:06:20amazing on phone calls
0:06:23Which is really cool. I'm glad that
0:06:26I'm glad they're doing more processing because for years the iPhones have had multiple microphones
0:06:31Right and Apple's had some limit of noise reduction
0:06:34But this is on top of that and in fact, you can play with this in GarageBand or Logic. There's a new
0:06:40Apple audio unit called AU sound isolation that you can put on a track and
0:06:47It gets rid of basically all background stuff. So I did I played with this this morning and
0:06:53just with some audio that I had laying around and
0:06:56It does do a good job of getting rid of background stuff
0:06:59I'm not sure it sounds as good as something like isotope, which is like a
0:07:03a very expensive suite of audio editing tools
0:07:08that Jason and I use for some things,
0:07:11but it's pretty good.
0:07:13And if you're looking to clean something up,
0:07:15I think in particular, using AU Sound Isolation
0:07:18and something like GarageBand
0:07:20would get you really far down the road
0:07:22without having to go out and buy something expensive.
0:07:25So I was unaware that it was an audio unit,
0:07:28but I am super glad that it is.
0:07:30And I think it's cool when Apple uses
0:07:33bits of technology across different products and things.
0:07:36And so, good work.
0:07:38- That's super cool.
0:07:39I had no idea that that was the thing.
0:07:40I wanna try that out.
0:07:41Those things to me though, I never really liked them.
0:07:43I tend to like, I'll use some kind of, you know,
0:07:46like removal of background sound processing.
0:07:50And I tend to end up going back, like for me,
0:07:53I think sometimes you can suffer through the, you know,
0:07:56sound of a fan or whatever,
0:07:58then like the kind of roboticness that I find
0:08:02that a lot of those sound isolation things do.
0:08:04For a phone call is fine, but for podcasts, I don't know.
0:08:08- My mom told me I sounded metallic
0:08:11when I tested voice isolation with her.
0:08:13- There you go.
0:08:14- It's like, hey, how do I sound?
0:08:15She's like, oh, you sound metallic.
0:08:17Okay.
0:08:18- But I, do you know what?
0:08:20I understand what she means by that.
0:08:22It's hard to describe, but like, it's just something sounds.
0:08:26So in the past few weeks,
0:08:27we've been talking about the potential capacitive buttons
0:08:30coming to the next iPhone.
0:08:32And I made an offhand quote of like,
0:08:34I'll have reference to like,
0:08:36how is it gonna work with cases and gloves
0:08:38and are you gonna need cut outs on the cases
0:08:39and all that kind of stuff.
0:08:41MacRumors is reporting from a source that they heard from
0:08:45and from more information in the MacRumors forums
0:08:47from this person, that Apple will building the ability
0:08:51to adjust pressure sensitivity in iOS for these buttons.
0:08:54Quote, "The tipster has revealed that iOS 17
0:08:57"will include a new toggle in settings
0:08:59"that will enable users to customize the sensitivity
0:09:01of the buttons to accommodate these different usage scenarios.
0:09:05It will detect presses, holds, and respond to various levels of pressure via the use
0:09:09of a new force touch style mechanism and Taptic Engine feedback.
0:09:14So if all of this is true, you're going to be able to customize everything about your
0:09:19iPhone 15 Pro.
0:09:21You'll be able to customize the action button, you'll be able to customize the sensitivity
0:09:26of the capacitive buttons.
0:09:28Everything becomes customizable.
0:09:31sounds fun but also weird though right like let's that's if we like you know
0:09:36we're talking about the fact of like you know is is this button are they like an
0:09:40action button and not a mute switch or like just is that better for regular
0:09:44people right we were talking about that like how good is the toggle or whatever
0:09:47but if it's like oh you buy a case for your iPhone oh now you have to go into
0:09:51settings and play around with the pressure sensitivity to turn your volume
0:09:56up and down yeah you know it's like it's like a I like that they if they have to
0:10:01do it, I'm happy that they would do it and not just be like, "We don't support cases
0:10:04other than our own," right?
0:10:07There's still like a... that sense... it's weird, right? Like that will be a weird reality
0:10:13to be in.
0:10:14I wonder if we're reading maybe too much into this or if this is kind of overstated in the
0:10:19reporting. What it makes me think of is the iPhone 7 and 8 where you can adjust the level
0:10:27of force you need to click the button, the haptic solid state home button.
0:10:34And that was customizable and it could detect different, or I guess it would register a
0:10:39different pressure.
0:10:41But that's, in that scenario, that's just something you can do, right?
0:10:47This would be something you might have to do, which is different, right?
0:10:52But like you buy a case and your case buttons will not work.
0:10:57And you have to troubleshoot to the point where you find this area of settings and tweak
0:11:02it.
0:11:03Like the home button thing, you just do that if you want to.
0:11:06It doesn't change your experience with the device so you don't need to do it because
0:11:09nothing covers the home button in that way, right?
0:11:13So like, yes, may have been a thing in the past of other stuff.
0:11:16Like I can customize my, the feedback from my trackpad.
0:11:20I don't have to do those things.
0:11:22They're just because you can, they offer that.
0:11:25And if, you know, so my, that's what,
0:11:27if this is that, right, it's just like,
0:11:29hey, we can offer it, so here you go.
0:11:31It's like a fun thing, like play around with it.
0:11:33Great.
0:11:34But if it's like, oh no, to get like this case
0:11:38from this company to work, you need to set it at this level.
0:11:41It's like, it's weird.
0:11:43But like, you know, for us, as Federica said,
0:11:45like that's great, right?
0:11:46Like if I have the ability to like,
0:11:48If I hold down on one button, it does this, hold down,
0:11:51you know, that's awesome if it does that.
0:11:53- Here's an idea, here's an idea,
0:11:55I'm just throwing it out there.
0:11:57What if, what if, imagine a new sort of made for iPhone
0:12:02program where compatible cases, you know,
0:12:06they have MagSafe, like the official MagSafe.
0:12:09Imagine if like, if the case came with built-in information
0:12:15on how to set the settings for the buttons for you.
0:12:19So that when you put it on, it automatically--
0:12:21- Similar to that, is if Apple specified thicknesses,
0:12:26which they do stuff like this.
0:12:27So like you can go to Apple's website,
0:12:30it's in the development center,
0:12:32and you can get all of the placement
0:12:34of the magnets for MagSafe.
0:12:36Like that's just a thing that they offer.
0:12:38It's like a thing.
0:12:39And I don't even know if it's part
0:12:41of the made for iPhone program.
0:12:42They just offer that stuff.
0:12:44So like, yeah, like through that program,
0:12:46they could maybe offer it.
0:12:47But the made for iPhone program though,
0:12:49is like you as a case maker now have to pay a tax to Apple,
0:12:53if you're in that program.
0:12:55So my hope would be that like similar to that,
0:12:57they might say like, oh,
0:12:59a thickness of this amount of millimeters
0:13:01will make sure that you won't need to adjust it even.
0:13:05You know?
0:13:06- Discord pointed out the SC2 and 3
0:13:09also have that button, just correct.
0:13:11And that on those phones and you set up,
0:13:13it puts that option in the setup,
0:13:16but that's I think still different
0:13:17than what you're talking about, so.
0:13:19- Again, it's not necessary.
0:13:21It's just like, here's the thing you can do,
0:13:23so we're letting you do it.
0:13:23It's not the same as like, you might have to do it.
0:13:27- Maybe they're gonna do something like,
0:13:28cases have NFC in them, or who knows?
0:13:30There could be a lot of cool things they could do here.
0:13:33I hope that this is not though,
0:13:35an attempt to lock down the case market.
0:13:38I think that would be a mistake.
0:13:40Now you know what that means, just a short sound, it's administration.
0:13:44Unless you hear the, you have to hear the whole sound for the quizzes to begin.
0:13:48That is just an administration sound for the quizzes.
0:13:51I've quizzes administration.
0:13:53So people have asked when I did the do the passionate ones know you quiz a few weeks
0:14:01ago for the two of you, people wanted to know like what about me?
0:14:05like what do the passionate ones think my favorite movie is or my favorite band.
0:14:10So there is a form in the show notes where you can go and fill out "do the passionate ones know
0:14:17Mike" and you can give your answers for what you think are my favorite xyz. This is going to form
0:14:24our member special for this year. So in what I've mentioned this before, quizzes can be in members
0:14:32content and so the member special that we're going to do later on this year will be the
0:14:37quizzes for this and we will follow up later on with what the point scoring was and how
0:14:42that affected things. So you can go and fill that out and later on in the year, I think
0:14:47maybe in the next couple of months, we'll be recording our member special and seeing
0:14:51how well the passionate ones know me and how Steven and Federico think you're going to
0:14:58answer to these questions.
0:15:01This episode of Connected is brought to you by ExpressVPN.
0:15:05Watching Netflix without using ExpressVPN is like playing your favorite game and not
0:15:09using all the power-ups.
0:15:10Why limit yourself when there's so much fun to be had?
0:15:13Netflix has thousands of shows across different countries, but with a VPN you only get access
0:15:17to a limited selection based on your location.
0:15:20With ExpressVPN you can unblock those shows by amending where it thinks you're located,
0:15:24and it works on a bunch of other streaming services too.
0:15:27This means that you can do things like watch The Office, the US version, on Canadian Netflix,
0:15:33or Lord of the Rings on Netflix in Turkey.
0:15:37Lots and lots of options.
0:15:38Back to the Future, Mike and I's I think joint favorite movie on Canadian Netflix.
0:15:43They got it going on in Canada is what I'm learning here.
0:15:46With ExpressVPN it's just one click.
0:15:48You open the app, choose the country, and you're off to the races.
0:15:53There's a bunch of other reasons to choose ExpressVPN.
0:15:55of my favorites is the blazing fast speed. You can stream video, do your calls,
0:16:00whatever you need over ExpressVPN. It is super fast and it's compatible with all
0:16:06your devices, phones, laptops, media consoles, smart TVs, and more. They have
0:16:11servers in over 94 different countries, so you can gain access to thousands of
0:16:15new shows and it works with other streaming services like BBC iPlayer,
0:16:19YouTube, and more. Stop paying full price for streaming services and only getting
0:16:23access to a fraction of their content. Get your money's worth at ExpressVPN.com/connected.
0:16:30That's ExpressVPN.com/connected to get an extra three months of ExpressVPN for free.
0:16:38Our thanks to ExpressVPN for their support of the show and Relay FM.
0:16:42So automation April has begun, which is, we spoke about this before, it is a, did you say a contest?
0:16:49Well, it's an event that also includes a contest, yes.
0:16:55Automation April spans across many things in the Max Stories Extended Universe, but
0:17:00also has the Automation April contest where Federico and a panel of judges will judge
0:17:05your shortcuts for their amusement.
0:17:11And what they're looking for is the best possible shortcuts.
0:17:15So Federy Cover Teacher comes along and flexes on everyone by creating a shortcut that is
0:17:22ChatGPT inside of a shortcut. Like what is this? Day one of Automation April? You're
0:17:28like "thou shalt not pass!"
0:17:30Day three.
0:17:31Day three. Oh, you gave him two days. That's good of you.
0:17:34Day three. Yes. Well, obviously this is not done to flex on people, as you say. It's
0:17:43- It's just, it's part of the automation April,
0:17:45in addition to the contest, in addition to the podcast,
0:17:48has an editorial component where we write articles
0:17:52and make shortcuts.
0:17:54And this just happens to be a really special shortcut
0:17:57that I've been working on for kind of the past month or so.
0:18:02And I really wanted to, because it's special,
0:18:05I wanted to save it for a special occasion,
0:18:07like a fancy suit, you know, that's,
0:18:11So I put on the suit today for Automation April.
0:18:15Yes.
0:18:16Okay, so what do you wanna know?
0:18:17- From a content planning perspective,
0:18:18this is a pretty epic shortcut.
0:18:21You should use it now, not earlier, right?
0:18:23Or later.
0:18:24Like I get, you just like, there's gonna be,
0:18:26you'll get to say like, hey, it's Automation April,
0:18:28while people are reading about a shortcut
0:18:30that I assume is probably gonna get linked
0:18:32all over the place, I would assume, right?
0:18:35- Yeah, it's going really well so far.
0:18:37So yes.
0:18:38So what does SGPT do?
0:18:43Like what is it doing in a nutshell?
0:18:45- So in a nutshell, it's a shortcut
0:18:47that lets you have conversations
0:18:49with OpenAI's chat GPT assistant.
0:18:53It lets you have back and forth conversations,
0:18:56but there's a bunch of shortcuts like that.
0:18:58We've seen, you probably have seen on Reddit
0:19:01and other places, other shortcuts
0:19:03to have a chat GPT conversation
0:19:06in the shortcuts UI or inside Siri.
0:19:09My shortcut does that, but it also does other things.
0:19:14Meaning in addition to conversations,
0:19:17it lets you integrate chat GPT with iOS,
0:19:21iPadOS and macOS directly to,
0:19:26in different places of the OS.
0:19:30So it's in addition to the conversations,
0:19:32it comes with support for native functionalities of your computer. There's a small list of
0:19:40integrations right now. There's more on the way, which we can talk about in a couple of
0:19:45minutes. My idea was, I like ChatGPT. Before the Italian government banned it, I was a
0:19:53ChatGPT+ subscriber. And then I got my refund.
0:19:57I'm only tangentially aware of this.
0:19:59- Yeah.
0:20:00- Why is the Italian government banned chat GPT?
0:20:04- It can curse on this show, right?
0:20:05- No.
0:20:06- Allegedly due to privacy concerns
0:20:09because of the data breach that they had a few days ago.
0:20:15And so the Italian privacy authority wants to know more
0:20:20about how the data of Italian users and customers
0:20:23is being used by open AI.
0:20:25Something like that.
0:20:26I had no idea OpenAI had a data breach.
0:20:28I didn't even know that was a thing.
0:20:30- Anyway, I'm a fan of chat GPT, what you can do with it.
0:20:34But I think chat GPT, its true potential lies
0:20:39in how you can give it your data
0:20:43and basically get another super smart brain
0:20:49to assist you with your stuff.
0:20:52Like, yeah, it's fine.
0:20:53You can ask chat GPT trivia questions
0:20:56or you can ask for recipes, or you can ask for, I don't know,
0:21:00travel plans.
0:21:01I mean, we've all tried that.
0:21:04But in many ways, I think that the true power
0:21:07lies in getting to know--
0:21:11making it-- putting it to work on data
0:21:15that you provide to chat GPT.
0:21:19And also, the power lies-- and I'm
0:21:21going to get to this in a minute, to teach ChatGPT how to return data in a specific format.
0:21:31So what I did was I started thinking about this and I thought, well, I have the shortcut
0:21:35that lets you have conversations with ChatGPT and you can ask it for dinner ideas or you
0:21:42can ask it for, you know, "Oh, tell me more about the country of, I don't know, Argentina"
0:21:49something like, yeah, you can ask all these questions, but what if I figured out a way
0:21:53to make stuff happen on your computer based on GPT's answers and data?
0:22:00So, in this first version of SGPT, which stands for Shortcuts GPT, or Siri GPT I guess it also
0:22:09works because it's a shortcut so you can use it manually in the Shortcuts app or you can
0:22:15invoke it via Siri. Totally works. In this first version you can do a few things that are
0:22:20native integrations, right? You can, for example, access the contents of your clipboard.
0:22:29So by using the trigger word clipboard you can do whatever you want. As long as you use the word
0:22:36clipboard in your query, SGPT will take whatever text is in your clipboard and give it to chatgpt,
0:22:45the cloud service, which will respond with an answer. So for example, if you fire up the
0:22:51shortcut and you say "summarize my clipboard", the first time you do it you'll get a permission prompt
0:22:59to give it access to your clipboard and then you get a summary of whatever is in your clipboard.
0:23:04Or for example, something that I like to do is, maybe you just wrote a blog post in Obsidian
0:23:10or in whatever text editor you like.
0:23:14You copy the text and you can ask something like, "Check my clipboard for grammar mistakes
0:23:20and tell me what they are and give me suggestions."
0:23:24You run that command, you wait a couple of seconds and you get back a detailed response
0:23:29by SGPT saying, "Okay, I took a look at the text in your clipboard.
0:23:33Here's the grammar mistakes that I found.
0:23:36So there's clipboard integration.
0:23:38There's an integration with Safari.
0:23:41So you can summarize web pages from Safari, which I will improve.
0:23:46This feature I want to improve in the next update, which should come up pretty soon.
0:23:51You can also open URLs.
0:23:54So for example, in chat GPT, sometimes you ask a question and then you can follow up
0:23:57and say, "Hey, can you actually give me a URL source for this?" And if the shortcut
0:24:04sees that ChatGPT responded with a URL, it opens that link in Safari. You can... what
0:24:12else do I have? You can... oh, there's a sort of an assistant for your reminders and your
0:24:20calendar events. So if you say to SGPT, "Can you help me with my schedule?" SGPT will take a look
0:24:31at your upcoming reminders, your upcoming calendar events. Again, there's permission prompts for all
0:24:37of this, and ChatGPT will tell you something like, "Okay, I took a look in your calendar.
0:24:42I see that you're a little overbooked for Wednesday because you have five reminders and
0:24:47five calendar events. Maybe you should consider moving some to Thursday when you seem to be
0:24:52more free. Something along those lines. So you can take a look at your agenda and basically
0:24:56give you suggestions. But the real big integration in this first version is the Apple Music integration.
0:25:06So one of the things that you can, one of the many things that you can ask ChatGPT is
0:25:12Can you give me a list of songs based on whatever criteria you want to ask?
0:25:20So you can go on the ChatGPT website and say, "Can you give me the most popular songs by
0:25:26Arctic Monkeys?"
0:25:28Or you can do, "Can you give me a list of 20 emo songs released between 2005 and 2010?"
0:25:36Or you can say, "Give me the top 20 songs by Oasis, Blur, and My Chemical Romance."
0:25:42Or you can get real fancy and be like, "I'm in the mood for something nostalgic, and I
0:25:48want to listen to some indie folk songs from the past 10 years.
0:25:55Can you give me a list of 15 and sort them in alphabetical order?"
0:25:59ChatGPT will understand all of this, and it'll give you a list of songs.
0:26:04So I started thinking about this and I was like, "Hmm, what if I figured out a way to
0:26:10get that list of songs and actually make a playlist in the real music app, like in the
0:26:19native music app on your device?"
0:26:23So this feature works in this first version of SGPT, and there's a couple of fascinating
0:26:33technical aspects behind it. So to make this happen, the shortcut, SGPT, teaches
0:26:42chatgpt, so the main service, how to return this list of songs in a specific format. Like it tells
0:26:55it, this is my query, I want you to return the following list of songs, such as the most
0:27:03popular Oasis songs, but when you return this data, it should be in this specific text format.
0:27:13You need to give me the name of the artists first, there has to be a separator between
0:27:18them and there has to be the name of the song. And there's literally a text prompt, which,
0:27:25if you want, I can read it back to you, which is basically like a set of instructions for
0:27:31ChatGPT to say, "Okay, so I see that the user wants to have data in this specific format."
0:27:39The instruction is, let me find it, so there's your query and then it says, "These songs
0:27:47must be real and not a product of your imagination.
0:27:50They must be available on either Spotify or Apple Music.
0:27:53YouTube alone is not enough.
0:27:55The playlist should be a list of text
0:27:58with each line in this format.
0:28:00And there's the format that I'm using.
0:28:02And then I explain, that's the name of the artist
0:28:05for the song, followed by a space,
0:28:06followed by a separator, followed by a space,
0:28:09and so and so.
0:28:10And then it says, your response should only be
0:28:13the list of text with no additional sentences.
0:28:16Like I am literally asking, I am literally telling chatGPT,
0:28:21do this in this very specific format in natural language.
0:28:25- Why? - Yeah.
0:28:27- Why can't you ask it that, what you just said?
0:28:30Like, you know what I mean? - I asked it that.
0:28:33I asked it that, but the reason is you wouldn't want
0:28:36to ask this yourself every single time, right?
0:28:40So the shortcut--
0:28:41- No, what I mean is, so sorry,
0:28:43like I didn't ask it very clearly, right?
0:28:45The way that you're asking it is like, give me this list and give it in this way.
0:28:53Why do you have to be so specific with it?
0:28:56Why can't you just say what you just said to me then of like, just give me this natural
0:29:00language?
0:29:01You know what I mean?
0:29:02Because my shortcut needs to parse the response in a specific way.
0:29:07Otherwise, there's going to be extra characters or something that I cannot anticipate.
0:29:15And so this, I guess, the real trick here is in the fact that I have built a hybrid
0:29:20shortcut. It's a bridge between chatgpt and your device, whether it's an iPhone or iPad
0:29:27or a Mac. Chatgpt is the brain in the cloud that returns information. Sgpt is the local
0:29:34shortcut that takes that information and based on what it is, makes stuff happen on your
0:29:40device. In this case...
0:29:41I want you to help me with something here, right?
0:29:43Because this bridging part is the part that is very confusing to me.
0:29:48Yes, yes.
0:29:49How can you give it an image and say use live text?
0:29:54Like, I don't understand.
0:29:56Yes.
0:29:56And like, because I assume you can then say like,
0:29:58use live text and like, summarize this.
0:30:01How is that?
0:30:02I don't understand how this is all doing this.
0:30:06So, shortcut has a lot of system integrations, right?
0:30:09as a lot of system actions. They are local to your device. ChatGPT, the cloud service
0:30:17by OpenAI, doesn't have any hooks into your device, right? It's just an API in the cloud
0:30:23that accepts text and returns text. But you can use shortcuts as the glue in the middle.
0:30:30So all of this, what you were wondering about, works with...there's a function. There's a
0:30:37little module, which is the second shortcut that you need to install, right? You installed
0:30:41it. It's called sgpt_encoder. That's a little function, essentially like a little submodule
0:30:50that you never see, in fact. It runs behind the scenes and it checks. Every time you ask
0:30:59something, that little function kicks in and basically checks your question and says, oh,
0:31:06did the user ask the word clipboard? Did the user ask the word live text? Or did they ask for a
0:31:13schedule? Or did they ask for a playlist? It checks your query and depending on what you ask, it runs
0:31:19a different set of actions. So for the live text example, if you say the words live text, it lets
0:31:26you pick a photo, that's a local action of shortcuts. That's the select photos action.
0:31:31You select a photo and then it runs the Apple-made extract text from image action.
0:31:38That's the live text. It's built into shortcuts. As long as you end up with some text
0:31:45that you can eventually provide to the chatgpt API, it doesn't matter.
0:31:53Basically, all these actions happen before making the call to the ChatGPT API.
0:32:00And that's why it's a hybrid approach. It runs some local actions before eventually invoking
0:32:09the real ChatGPT and saying, "Okay, here's the text. Do whatever I'm asking you to do."
0:32:15So for the live text, it uses the live text action. For the clipboard, of course, shortcuts can read
0:32:21the contents of your clipboard. For the Apple Music one, it's a bit trickier because it needs to
0:32:28assemble a playlist in between the response from ChatGPT and your next question. And of course,
0:32:38Shortcuts doesn't have real Apple Music actions. So I'm still using, unfortunately, the iTunes
0:32:45store actions as a fallback. Oh my god. How does that even work? Like, I don't even understand how
0:32:53those things still talk to each other. The catalog, I have no idea why Apple doesn't have a real Apple
0:33:00Music web search action. I really hope that changes in iOS 17. But yeah, so the gist of it all is
0:33:08we've seen a bunch of shortcuts that let you have conversations with Chai GPT. This one has that.
0:33:14It's holding the entire conversation.
0:33:18In fact, this initially was the tricky part
0:33:22of building this.
0:33:24Like this shortcut works because it's got a repeat loop
0:33:29inside of it that repeats a series of actions
0:33:35a thousand times.
0:33:37Like literally it says repeat 1000 times.
0:33:41Because you gotta be able to ask follow-up questions, right?
0:33:44That's the whole point of ChatGPT.
0:33:47And so what was tricky was,
0:33:49how do you store a conversation in shortcuts?
0:33:53And that conversation grows longer and longer
0:33:55and longer over time.
0:33:56- Yeah, 'cause just for clarification,
0:33:58the way that these models work
0:34:01is like every time a question is asked,
0:34:03for it to continue,
0:34:05like it feels like it's having a conversation,
0:34:07you need to send the question an answer
0:34:09for every question that has already been asked.
0:34:13every time you contact the ChatGPT API, you give it the updated transcript of the entire conversation.
0:34:24So every time it looks at the entire thing and it looks at the latest item, it's like "Okay,
0:34:32I will now give you a new answer." So that's the way that it works. And this is the new ChatGPT
0:34:38API that came out like a month ago. It's not the old text completion API that was available before,
0:34:47this is the new one which is super cheap and very easy to program.
0:34:51Where does it store the information during the conversation?
0:34:55Another difference from other shortcuts that I've seen, I've seen shortcuts that every time you run
0:35:00them they store the conversation in a text file or somewhere, like they actually save your
0:35:07a conversation somewhere, like in storage.
0:35:11My shortcut doesn't do that.
0:35:12It's all dynamic and it's all stored in variables
0:35:17that exist when the shortcut is running and disappear
0:35:20when the shortcut is no longer running.
0:35:22It's all--
0:35:23- I thought you were gonna say base64.
0:35:25I thought that was where we were leading.
0:35:27- No, no, no, no, absolutely not.
0:35:29It's all in variables.
0:35:32I only save the transcript
0:35:35if you ask the shortcut to do that.
0:35:37Right. So that thousand loop that you're saying, it's just a number you've put in.
0:35:42It's just a large number.
0:35:44But if somebody did a thousand and one...
0:35:49No, the shortcut would break.
0:35:51Right. That's good.
0:35:52The shortcut would break before.
0:35:53Which is obviously like, that's not going to happen.
0:35:54It would run out of memory before.
0:35:55Yeah. It's not going to happen.
0:35:56It would run out of memory before.
0:35:57Yeah.
0:35:58Like the whole iPhone just like catches on fire.
0:36:02Yeah. So, okay. So I don't know.
0:36:05What else do you want to know?
0:36:06- So like this is, it is wild to me that you're,
0:36:10I understand what you have said
0:36:14about how you tie it into the system actions.
0:36:19It still feels a little unbelievable.
0:36:22Like I feel like it's so, it is so weird.
0:36:25Like it's weird.
0:36:27And so from what you've explained to me,
0:36:29all of that stuff is happening in the other shortcut.
0:36:32like that's passing every query
0:36:35to check for some trigger phrases
0:36:38that then indicate that something needs to happen.
0:36:41Is it before the text is then sent to the API?
0:36:45- Yeah.
0:36:46- Okay, so it's, for example, if you say,
0:36:49I want my schedule,
0:36:50this is running the native shortcut action
0:36:54for get my calendar events.
0:36:55- Yes.
0:36:56- And is including that information in the,
0:37:00do you have to then format the text in a certain way
0:37:03to pass it to the API?
0:37:05Okay, so say like, this person has asked summarize,
0:37:09da da da da, here are my,
0:37:12here are the events that you are summarizing.
0:37:14- That's exactly what's going on.
0:37:16- Yes. - That's wild.
0:37:18- And the other thing like on that,
0:37:20there is a thing that you reference in the blog post
0:37:24where like you have to do this like
0:37:26weird opening kind of gambit.
0:37:29incantation. With the API? Yeah, to say like, here's who I am, here's what this is, here's
0:37:34what you are. Does that happen at the beginning of every shortcut interaction? Like the beginning
0:37:42of it, right? That's called a system message in the ChatGPT API. The system message is
0:37:49something you can use to program a certain behavior for the model. And in this case,
0:37:55I wanted to program ChatGPT to be informative, helpful, to refer to itself as SGPT.
0:38:06The system message says, "You are SGPT and you were created by Federico Vittucci as a
0:38:13fork of ChatGPT that runs inside shortcuts."
0:38:17This is who you are, and then it says, "This is what you're supposed to do."
0:38:21What you're supposed to do is provide short but informative answers that users must be
0:38:28able to listen to in 15 seconds or less.
0:38:31Do not go longer than that unless I ask you to be more detailed.
0:38:36That is literally how you program the behavior of the model.
0:38:41And my reasoning for this was, if people are going to be using this in Siri, you want to
0:38:45be able to have concise information that you can listen to from Siri in just a few seconds.
0:38:52But if you want to have the option to get, like, to be more detailed about a topic, or
0:38:57like, okay, I need you to be more detailed about this specific thing, like, give me all
0:39:02the steps to a recipe, for example, it needs to be able to do that.
0:39:07And if you don't program your, the behavior of the model in a certain way, you will reach
0:39:14some walls, like the model will just refuse to do something for you.
0:39:19It's weird when you see it in action.
0:39:22One of the things that I will release in Mac Stories Weekly this Friday is a set of personalities
0:39:27that I've created, like alternative personalities that I've created for SGPG.
0:39:32One of them is an absolute...
0:39:34You guys have seen screenshots of this.
0:39:36One of them is an absolutely evil personality that is actually much, much worse than care
0:39:43weather and actually insults you, which I think is quite funny because it's like extremely
0:39:49unkind. And there's another that is the sort of a digital personification of Steve Jobs.
0:39:57So chat talks like Steve Jobs and drops Apple references everywhere, even if it's not necessary.
0:40:05Oh, this is like I'm going to install this one. This sounds great.
0:40:10For example, today there's a screenshot, let me show you this, there's a screenshot where
0:40:16I asked the Steve Jobs personality to describe ancient Rome or something like that.
0:40:24And the Steve Jobs personality said, let me read this to you because I thought it was
0:40:30funny.
0:40:31Rome was founded in 753 BC by its first king, Romulus.
0:40:36grow into a rich and powerful city throughout the next few centuries, similar to how Apple
0:40:42has grown over the years. The Roman Empire left an unparalleled legacy in terms of art,
0:40:49architecture and law. The Colosseum, one of its most astonishing creations, could hold
0:40:55up to 50,000 spectators, almost like a modern Apple event. It's like...
0:41:02That is incredible. I love that.
0:41:04So anyway, in weekly I will release the multiple personalities, just if you want to have some
0:41:09fun with it.
0:41:12And then I will do a proper making of, like from a technical perspective on the automation
0:41:17academy, I will do a making of because it's very technically involved, even though it
0:41:23looks like, oh, it's just a bunch of unnatural language prompts.
0:41:27But the difficulty was in sort of making all of these happening shortcuts with the limited
0:41:33tools that you have in shortcuts.
0:41:36And lastly, I have a question for you.
0:41:37Yes, go for it.
0:41:39I feel like I know the answer to this, but I need to ask it anyway.
0:41:42Does the shortcut have feelings?
0:41:46Is it possible?
0:41:47Well, that, but like, could it make changes to things without me wanting?
0:41:54There's no way it could do that, right?
0:41:56Because the shortcut, unless you give it the actions, it can't do it, right?
0:42:02Not in this version. So right now, no, it cannot make any changes on your device.
0:42:08But in theory, you could have an encoder that could, right? Like in the same way that you have the encoder that looks for words,
0:42:19you could similarly have one that takes what the AI passes back. And like when it says to me, you should move calendar events, like,
0:42:30In theory, there could be some actions at least where you could pass that and put it through as a variable and have it do things.
0:42:37So you have touched upon the very thing that I was going to say.
0:42:43For version 1.1, one of the things that I will enable, I've already started working on it.
0:42:49If you run this on macOS, if you run sgpt 1.1 on macOS,
0:42:55I should be able to make it run code on your computer.
0:43:02Mmm.
0:43:03[LAUGHTER]
0:43:05I don't like it.
0:43:06[LAUGHTER]
0:43:09Let me continue. Let me continue. Let me continue.
0:43:13There will be multiple layers of permission prompts.
0:43:17It will not do that without your consent.
0:43:21like I will literally put in a box that says "this is what's about to run, do you want to run it?"
0:43:27And that's also like how shortcuts work, like you need to confirm the step. But one of the
0:43:34really powerful things about chatgpt is that you can ask it for code, you can ask it for
0:43:42terminal commands, you can ask it for apple script, you can ask it... And I know developers,
0:43:48I know developers today, developers we know who make apps we use without naming names,
0:43:55I know developers who do this, who get assistance from chatgpt for Zwift UI code or Zwift code.
0:44:01That's no problem. I mean, this is what that GitHub co-pilot thing has been.
0:44:05Exactly.
0:44:06Very good, right? I don't understand why people get so overwhelmed about that.
0:44:09So, for example, you can already go on the chatgpt website and say, "Hey, I need an
0:44:14an AppleScript that takes a bunch of files in a folder and renames them sequentially.
0:44:21Can you give me that script? And you can go on the chat.gpt website, copy that script,
0:44:26paste it into AppleScript editor on your Mac and run it, and it'll likely work. Why not
0:44:33make it easier? Like, why not make it happen contextually in Shortcuts? Shortcuts has actions
0:44:38for the terminal, has actions for AppleScript. You could be like, "Hey, I need some code
0:44:44this folder, maybe the folder is itself a variable in shortcuts, can you give me an
0:44:51AppleScript or can you give me a terminal command to do this? You will have to verify what you
0:44:56received, you will have to confirm what you received, but then the code will just run
0:45:01without having to do the copy and paste, without having to do the open and different application
0:45:07for it. Another thing I want to do is what you also suggested Mike, what about rescheduling
0:45:13reminders, what about rescheduling things? Like, in addition to like telling me how my
0:45:18agenda is wrong, can you also actually help me make more sense of it? And there's also
0:45:23one of the other things I'm looking into. There's a demo that I already have on my computer
0:45:29of SGPT processing tasks from Todoist in addition to reminders. So I got that working as well.
0:45:37But yeah, the big one I think will be actually run,
0:45:41like I need some code to do something on my Mac.
0:45:46Can you give me that code and run it?
0:45:48So yeah, I am approaching this very carefully, obviously,
0:45:51because like nobody wants chat GPT
0:45:55to just being able to arbitrarily return code
0:45:58on your computer or like imagine a scenario
0:46:01in which it disobeys your orders
0:46:03and instead of sending you back a playlist,
0:46:06sends back the word code and it runs some code you don't want.
0:46:10Like obviously the shortcut will have to be designed with safeguards in place.
0:46:15And that's like what I also I think is ultimately safe about this is that chatGPT will never
0:46:22be able to alter the code of the shortcut on your machine because of how the system
0:46:27is designed with sandboxing on iOS and iOS and Mac OS, like it's got sandboxing all over
0:46:33the place.
0:46:34it needs to be, for example, there used to be a time years ago when you could make a shortcut
0:46:40just out of pure XML and just convert that XML code into a shortcut. These days if you want to
0:46:47do it you got to sign that shortcut with your developer identifier or Mac OS. So like it's not as
0:46:54loose as it used to be years ago, which is why it's ultimately safe, but still
0:47:01when I do that, once I will unlock that integration,
0:47:05like, "Yay, run terminal commands
0:47:08and run AppleScript code on the user's machine,"
0:47:10there will have to be confirmation steps.
0:47:13It's the only way.
0:47:15Otherwise it will be kind of uncomfortable.
0:47:17- Dangerous.
0:47:18- Dangerous, yes, yes.
0:47:20- My favorite thing about this is that ChatGPT,
0:47:23right, this like newfangled technology
0:47:26can write AppleScript, which is ancient
0:47:29and somehow still hanging on.
0:47:31And there's something about that that makes me laugh.
0:47:34- I mean, it's kind of the beauty of it really.
0:47:37It's like, it's, AppleScript is so old,
0:47:39there's so much of it on the web.
0:47:40So like, it has no, it will have no problem with it, right?
0:47:44Like it's just like, it can just do it.
0:47:46- This episode of Connected is brought to you by Indeed.
0:47:49When it comes to hiring, you need to trust your gut.
0:47:52But what if you could give your gut some help?
0:47:54When you want to find quality talent fast, you need Indeed.
0:47:58Indeed is the hiring platform where you can attract,
0:48:00interview and hire all in one place. Don't spend hours on multiple job sites
0:48:05looking for candidates with the right skills,
0:48:08when you can do it all with Indeed. Find top talent fast with Indeed's suite of
0:48:13powerful hiring tools like matching, assessments, and virtual interviews.
0:48:17If you hate waiting, Indeed's US data shows that over 80%
0:48:21of Indeed employers find quality candidates
0:48:24whose resume and Indeed match their job description the moment they sponsor a job.
0:48:29Indeed Match really is incredible. Candidates you invite to apply through instant match are three
0:48:34times more likely to apply to your job than candidates who only see it in search according
0:48:39to US Indeed data. Join over 3 million businesses worldwide who are already using Indeed to hire
0:48:45great talent. Indeed knows that when you're growing your business you have to make every
0:48:48dollar count. That's why with Indeed you only pay for quality applications that match your
0:48:54your must-have job requirements.
0:48:56Visit indeed.com/connected to start hiring now.
0:49:00That's I-N-D-E-E-D, indeed.com/connected.
0:49:05Terms and conditions apply,
0:49:07cost per application pricing is not available for everyone.
0:49:11Need to hire?
0:49:12You need Indeed.
0:49:13Our thanks to Indeed for supporting the show and Relay FM.
0:49:17- So I think just in hearing you talk about this Federico,
0:49:21the elephant in the room is Sherlock Holmes, right?
0:49:25Like you,
0:49:26you like just hearing you talk about this.
0:49:31- Watson, bring the AI.
0:49:33- This should be, this should be what Siri is, right?
0:49:39Like it's not hard to see that.
0:49:41Like this is what we wanted Siri to be, right?
0:49:45You should be able to have a natural conversation with Siri
0:49:49and have it communicate to you things about your schedule
0:49:53in a way that seems natural,
0:49:55and then do the things that you're hoping to do
0:49:57of like have it move things around for me
0:50:00and maybe even take into account the tasks
0:50:03that I have available to me on that.
0:50:05Do you know what I mean?
0:50:06Like all of this stuff should be
0:50:08what a digital assistant should be.
0:50:12And I think there is a ticking clock, right,
0:50:15on is it Google or Amazon that implement
0:50:20a large language model into their assistants first?
0:50:24Like which one is it gonna be, right?
0:50:25'Cause this just feels like the natural evolution
0:50:30of these assistants, right?
0:50:32Was it the Wall Street Journal who wrote an article
0:50:34about like just how stupid like Siri and Google
0:50:39and this is, right?
0:50:40Like when you compare it to these things
0:50:43that like now our home assistants just feel like
0:50:47they have no brain, right?
0:50:50So I guess what I wanted to get from you two
0:50:53is a sense of like, one, do you think Apple should
0:50:57have Siri become a large language model?
0:50:59And two, how long until these features
0:51:03that you're creating in federal could get Sherlock by Apple?
0:51:07- Personally, I think it's not that they should,
0:51:10they have to at this point.
0:51:13They have to respond to this.
0:51:16Because when you compare Syrian Chad GPT,
0:51:19the difference is just embarrassing at this point.
0:51:23And you see, this is no longer a techy thing.
0:51:30Regular people know about and use Chad GPT.
0:51:34You see kids in school using it.
0:51:36It's why so many schools are banning Chad GPT,
0:51:38because it's writing essays, and it's helping students there.
0:51:42And there's actually a very practical example
0:51:45in the article that I put out today
0:51:48when I asked Chad GPT to give me a list of songs
0:51:52from the members of Boy Genius.
0:51:54Boy Genius is a supergroup, so-called,
0:52:00comprised of Phoebe Bridgers, Julian Baker, and Lucy Dacus.
0:52:05And Chad GPT gave me a list of singles
0:52:08by the three artists who are part of the boy genius supergroup. When I asked Siri, Siri
0:52:15had no idea. And music should be one of Apple's strong domains in theory, right? So that's
0:52:23because like, Chai GPT is powered by a large language model that can go multiple layers
0:52:30deep into the meaning of a query, right? It can actually understand what you mean. And
0:52:36it's the neural network that's behind it can make, I don't want to say infinite levels
0:52:42of connections, but it goes, it's like the world's biggest AI onion, if you will. Like
0:52:48it goes layer after layer after layer to understand what you mean. And when you compare that to
0:52:55Siri, the difference is just embarrassing. So my answer is they will have to respond.
0:53:01My concern is how long will it take them to respond?
0:53:06That's exactly my question.
0:53:08When you see a lot of what's coming out now, including the incredible thing that you've
0:53:12built, it's standing atop of open AI's work.
0:53:16I don't see Apple doing that.
0:53:17I see Apple wanting to go it alone.
0:53:21And that's going to take years.
0:53:23Well, and also, you would assume Apple probably want to run on device if they're going to
0:53:28do this?
0:53:29Yes.
0:53:30And how good can the model be?
0:53:34if it has to be small enough to run on device, right?
0:53:38Because, I mean, I know we're in it,
0:53:41really we are in the infancy of this technology
0:53:43at the moment, but like all of the small models
0:53:46are just not as good, right?
0:53:49So like this is one of the issues
0:53:51with Google Bard right now is Google are running it
0:53:55currently on a smaller version of their models
0:53:57and they're getting ready to use one
0:54:01of their more powerful models.
0:54:03But that's why, comparatively,
0:54:06it just doesn't seem as impressive.
0:54:08- So I think what's really concerning if I were Apple,
0:54:13and what I think people are misjudging at the moment,
0:54:18is that you look at chat GPT,
0:54:20or you look at BARD by Google,
0:54:22and you just see a little website
0:54:24where you go and you type in stuff,
0:54:26and chat GPT, of course, also has an API.
0:54:29But what I think the real vision for this is,
0:54:33Then Microsoft, we spoke about this before,
0:54:36actually understood the potential of this.
0:54:39The real potential here, the real vision here, I think,
0:54:42is not like in the little website, in the little chat box
0:54:46where you go and you type in stuff
0:54:48and the thing tells you the recipe for carbonara.
0:54:52The real potential is when you start thinking about this,
0:54:55I was actually discussing this idea with a friend
0:54:58of the show, Steve Transmith today.
0:55:01The potential for this thing is when the language model, in this case, ChatGPT, when the language
0:55:08model becomes the software, right?
0:55:13Like, that's what Microsoft is doing.
0:55:17They are infusing their software with the capabilities of the language model so that
0:55:25it actually looks into your Word documents, it actually looks into your web browser, it
0:55:29into your email client and it's right there, okay? It's not just a chat box, it's not just
0:55:36Bing, it's right there in the apps that you use. What if you project this, you know, five
0:55:43years from now, maybe that's even too long a timeline, maybe even two years from now
0:55:48is enough.
0:55:49I mean, honestly, four weeks could be, you know, like with the way this stuff is moving.
0:55:53I guess what I'm wondering is how long until chat-gpt, or I don't know, the GPT-5 model
0:56:00that should come out in late 2023, like how long until this thing writes its own OS from
0:56:09scratch?
0:56:10Like how long until GPT is able to display little interfaces for you and be like, "Hey,
0:56:20I need a window to compose some markdown text.
0:56:23And it's like, OK, here's the app you're looking for.
0:56:27Like if you're Apple, and if you're looking into this,
0:56:30I mean, this thing knows how to write code.
0:56:33It knows what code means and how to check for errors.
0:56:36How long until it can write its own code and write its own UI,
0:56:41write its own functions, and it's like, hey,
0:56:44I need an audio editor for a podcast.
0:56:47And it's like, here you go, I made an app for you.
0:56:49And it's composed at runtime, and you
0:56:57have an app made just for you.
0:56:59How long until that happens?
0:57:00And if you're Apple, how scary is this for you?
0:57:03As a company who rightfully so until so far,
0:57:07you pride yourself on the idea of--
0:57:11I don't want to say artisanal software, but also kind of
0:57:13that.
0:57:14very designed and custom tailored software.
0:57:22This must sound like a nightmare to Apple,
0:57:25this idea of it's a large language model
0:57:30able to write software.
0:57:32If that's the timeline, five years from now,
0:57:35is an app store even necessary anymore?
0:57:39So that's--
0:57:41I think if in that scenario,
0:57:44I don't know if they would be able to run the App Store
0:57:47because people would be submitting apps.
0:57:49- Yep.
0:57:50- And I am assuming it's already increasing.
0:57:54Like the amount of apps that are being submitted
0:57:56to the App Store has probably increased
0:57:59because of the fact that, as you mentioned earlier,
0:58:02people are able to use these tools today
0:58:05to help them write SwiftUI apps.
0:58:08- Yep.
0:58:08Like that is like year one in many ways.
0:58:12Like, I don't know that GPT-1, GPT-2 was used years ago,
0:58:16but like GPT-3 and 3.5 really were the turning point
0:58:20for the large language model.
0:58:22- Yeah, this all kicked off in like November, December.
0:58:24- Yeah.
0:58:25- Like that's when this really like truly started.
0:58:28Everything else is like-
0:58:29- And the timeline is accelerating.
0:58:30- You've got like BC and AD, right?
0:58:33Like there was a point at the end of the year,
0:58:36probably that day when
0:58:38ChatGPT was launched.
0:58:40That is zero.
0:58:42Everything like I even I'll say
0:58:45Dali and all that kind of stuff.
0:58:47Do you ever hear about the image
0:58:49generation anymore?
0:58:50Like, no. When was the last time
0:58:52you heard someone like talking about
0:58:54stable diffusion, like as a product,
0:58:56not as just like a thing that exists?
0:58:58Right. Like that was and that
0:59:00lasted for like two months.
0:59:02And then we have continued since
0:59:05then. But it was the pop photo.
0:59:07There was the fake, right.
0:59:09But like you just but now it's just
0:59:11like that was just made of AI or
0:59:13like most people like if you ask
0:59:15someone, well, how did that pop
0:59:16photo come around?
0:59:17Some of them someone use chat GPT
0:59:19to make that.
0:59:20But we know they didn't.
0:59:21But like we're at a point now where
0:59:23it's like that is it.
0:59:24That pop photo, by the way, 100
0:59:26percent fooled me.
0:59:27Like, yeah, 100
0:59:29percent. Because this is the issue
0:59:31with that pop photo specifically,
0:59:33is it looks
0:59:35it's not absurd enough, right?
0:59:37I could imagine the pope wearing a coat like that.
0:59:39Because it's cold.
0:59:42But it's just a white puffy coat, like it's not, you know,
0:59:45if it had like diamonds on it, right?
0:59:48Like then I'd be like, oh, I don't believe that.
0:59:50It's a very hype pope.
0:59:52Yeah, I just feel like that,
0:59:54all right, if you take away that one thing that I said about when was the last time,
0:59:58that we've moved on from image generation as like,
1:00:01Yes, by and large, yes.
1:00:04because now it's like Bing just does image generation,
1:00:08like you know, that's going to be what happens to image generation
1:00:11is you'll ask chat GPT to make you an image,
1:00:13which GPT-4 can do some of this stuff, right?
1:00:15So like the image generation now will be sucked into the text-based language,
1:00:19large language models, I feel like,
1:00:21as like just a thing that they can ask another model to do for them.
1:00:26Like this is where it's changed and that's where we are now.
1:00:29Like we are in this wall of large language models,
1:00:31Like that's what this technology is.
1:00:36- I don't know what Apple is.
1:00:38Zach also pointed out in the discord,
1:00:40something that we discussed, I think,
1:00:43I'm sure I posted it on Mastodon a few weeks ago.
1:00:47Did that rumor of Apple wanting everybody
1:00:52to be able to create apps on the Reality Pro headset,
1:00:56the upcoming headset, using Siri.
1:00:59And we also left about,
1:01:00We all made fun of it, but like that's exactly what I was talking to regards to like, what
1:01:06if you used AI to write software for you?
1:01:09And I'll go one level deeper than that.
1:01:14Apple should use shortcuts as the system behind this.
1:01:18I have, I'm sure I tweeted this last year.
1:01:23You can go find a tweet of mine where I said something along the lines of, "Hey, imagine
1:01:30the old feature from Beats Music, the sentence, but applied to making shortcuts with natural
1:01:38language.
1:01:40And if you think about that, like, think about how sort of prehistoric it is that I'm still
1:01:47building shortcuts with a drag and drop visual editor.
1:01:51I should be able to say, "Hey, system, make me a shortcut that takes my five recently
1:02:00edited reminders and renames them sequentially," or something like that.
1:02:05Like I should be able to write shortcuts in natural language.
1:02:08I shouldn't have to build shortcuts with drag and drop.
1:02:12And if you...
1:02:13That is a great point.
1:02:14If you take that one step beyond, then yeah, you should be able to write your own little
1:02:19apps for the headset via voice because you wouldn't want to use a drag and drop editor
1:02:25or you wouldn't want to code on the headset itself, I'm assuming.
1:02:31But yeah, if I were Apple I would be concerned about all of this and I have no idea where
1:02:42they stand at the moment.
1:02:44If they have their own large language model or not, obviously they don't talk about this
1:02:50stuff.
1:02:51What concerns me about Apple is just looking down the street at Google and seeing BARD
1:02:56and how kind of janky it is compared to what OpenAI has been able to do, right?
1:03:03And like, Google's fine doing it on the server.
1:03:05Google's fine scraping a bunch of content to do it.
1:03:08And I just, I think generally in Apple analysis, it's kind of lazy to fall back on, well, Apple
1:03:18won't do this because of their values or their ideals, right?
1:03:21Like, oh, Apple's always been bad at gaming, so they always will be bad at gaming because
1:03:24they don't understand it.
1:03:25Like, that's true, but also like we could go further in our analysis and be a bit more
1:03:29specific.
1:03:33But I do think this is going to be one of those things where they run headlong into
1:03:38what they say is important.
1:03:40And that's not to say that, I mean, yes, they've had a data breach and who hasn't, but it's
1:03:47not to say OpenAI is like doing a bunch of wrong things.
1:03:50I honestly don't know enough to say that definitively one way or the other.
1:03:55But I do imagine that Apple looks at it and thinks, "Oh, that's a little icky.
1:03:59That's a little icky.
1:04:00It's not something we want to do, not the way we'd want to do it.
1:04:03And at some point, do you have to break from that model?
1:04:09I mean, in ways they do break from that model with Siri, right?
1:04:12Like a lot is done on device, but there's still a lot done in the cloud.
1:04:17And all you gotta do is put your iPhone in airplane mode and see what Siri can and can't
1:04:21do, right?
1:04:22You find the limits very easily.
1:04:24But this does feel different in a way.
1:04:28And I think you're right Federico, I think they have to address it because if this is
1:04:32the future, and I'm more and more convinced that it is, I'm not totally convinced but
1:04:37more than I was a month ago, then they do have to deal with it because they are already
1:04:43in the space with Siri.
1:04:45That New York Times article, it wasn't Wall Street Journal, it was New York Times.
1:04:48The New York Times article had a lot of good thoughts in it, but the biggest thing I walked
1:04:51away with was, you know, voice assistants and like LLMs, they're kind of in the same
1:04:57category in that you just give a computer a question or a prompt and it
1:05:03goes off and does things, right? Like at the high level they are the same thing.
1:05:07They're already getting the floor wiped with Siri, right? So I don't know.
1:05:12They have to get into this. Like, why are they moving into AR and VR?
1:05:17This is just that, right? Like, it is an emerging technology,
1:05:24which realistically today large language models feel more like the future than they
1:05:31are in VR but it's too late. Yeah, Apple's headset is due out at any time.
1:05:38I mean it's too late for every company that's getting into it. Now you've kind
1:05:41of just you got to go to see this through and hope that you can connect
1:05:44these two later on like what Federico was just saying right like maybe you can
1:05:48say like oh we actually can but if you today like what wire is technology like
1:05:53this is where it's going we've all forgotten about crypto we've all forgotten about web3
1:05:57those things I feel like we could all look at those and be like that's not gonna work
1:06:02in the way that it's moving this though this does not feel like a bubble this does not feel
1:06:09like a fad like there is a there is there's a thing here like yes it is problematic it is
1:06:15is existentially problematic potentially, right?
1:06:20But this is powerful technology
1:06:24that can be used in really interesting ways.
1:06:27They have to do this, but it's how, right?
1:06:31Like I've heard a bunch of people reference
1:06:33the stable diffusion thing that they did, right?
1:06:35Where they enabled the ability to run stable diffusion
1:06:40locally on Apple Silicon.
1:06:43So like, there is a possibility that Apple have been smart enough that it just so happens
1:06:47that we can finally use that neural engine for something, right?
1:06:51And it's going to be their large language model.
1:06:54But they have to be here soon.
1:06:57They can't walk in four years late and two years behind.
1:07:03Honestly, I think Federico, I think your work, and I cannot tell you how impressed I am with
1:07:09this, it has blown my mind to see you develop this.
1:07:12Thank you.
1:07:13But what you have shown me in this is exactly what Mike just said.
1:07:17Mike said it's not a fad.
1:07:18It's not.
1:07:19You could look at crypto and other things and see how they would fail.
1:07:21The reason this is different, I think, or the lesson I'm learning from this is that
1:07:26this can be a general computing tool.
1:07:28Crypto is never going to be the blockchain.
1:07:30Maybe had other applications, but you're very problematic in other ways.
1:07:34You built this into a shortcut that anyone can run and do stuff.
1:07:39And it's the first time in a while that I think a really new type of technology has
1:07:44come on so quickly that could also be so broad.
1:07:48Right?
1:07:49There are always things on the nerdy end that like, come on fast and nerds get excited.
1:07:55Maybe eventually they trickle down to everybody else.
1:07:57That's how voice assistants were in the beginning.
1:07:59Siri started life as a third party app you could talk to, and then eventually became
1:08:03general computing when it came on the iPhone, it can do everything it can do now.
1:08:08This feels like it's on a similar track where other things that we've seen over the last
1:08:13couple years that have come and gone never had the potential to follow that track.
1:08:18And your shortcut, honestly, is like example number one in my book now of, "Oh, this could
1:08:23be a thing that people just use in their everyday lives."
1:08:27And that's wild to me.
1:08:29I have one last question for the two of you here.
1:08:32All right, so let's all just for the sake of this, and I think we're all pretty much
1:08:36an agreement that Apple will have some kind of large language model thing in the future,
1:08:41right? Like, I feel like it's just inevitable. They're not stupid. That they, I mean, there's
1:08:45been reports that they are already knee deep in this stuff, right? And have been for a while,
1:08:50like all large tech companies. Do they call it Siri? Probably not. You think they cut their
1:08:58losses on Siri? Yeah. They got their money's worth out of it. It's been what, 12 years of Siri?
1:09:06they probably want to start fresh with something that doesn't have the implications of the
1:09:14baggage of the joke, you know, like, ah, it's Siri.
1:09:17What I'll say is Bing. It's still called Bing, although maybe it's fair to say that Microsoft's
1:09:26actual brand for this is Co-Pilot. Like that is that's that's taking time, which is a great
1:09:33brand better than Bard. I mean, geez, Steven, what do you think?
1:09:36Yeah, I think it gets a new brand or some sort of sub brand. Already the Siri brand is suffering from its early days.
1:09:45Because Siri is way better than it used to be. It's not great all the time, but it's miles better than it was at launch. And in the years after. But Siri is one of those things. People are like, Oh, I use it on my iPhone six, and it was terrible. And I haven't touched it since.
1:09:59And it would be a shame if Apple doesn't do any of this, but I just want to
1:10:03reiterate this because what OpenAI doesn't have is they don't make the computers
1:10:10that we use.
1:10:11They don't make the phones that we use.
1:10:13Imagine a large language model made by Apple integrated with the computers that
1:10:19we use.
1:10:19Yeah.
1:10:20Well, we see that now a little bit with a co-pilot and Microsoft.
1:10:24Yes.
1:10:25And you see a little bit with co-pilot.
1:10:27you see a little bit of a little bit of a little bit of a little bit with my shortcut,
1:10:31which is inspired by that idea. But imagine if Apple really went for it, which is why
1:10:39I am excited, but also, I mean, sad that we're likely not going to see any of this this year
1:10:46or probably next year or the year after that. Yeah, we'll see.
1:10:51Knowledge Navigator, man. It's going to happen.
1:10:53Yes.
1:10:54Yes.
1:10:55It's going to happen. Knowledge Navigator, any day now.
1:10:57That's a deep cut.
1:10:581987.
1:10:59Do we get to 2027?
1:11:02I don't know.
1:11:03I don't know.
1:11:04Look, I honestly think your timeline is not too incorrect.
1:11:08Before we go, Federico, what's coming up next in Automation April?
1:11:13Thank you for asking.
1:11:14So the contest to submit your shortcut is live at shortcuts.maxstories.net.
1:11:20You can either log in with your cloud account or make a free Max Stories account.
1:11:24You can submit up to two shortcuts.
1:11:27There's gonna be multiple categories,
1:11:29but there's gonna be the big one,
1:11:31the best overall shortcut
1:11:33where you can win a loop deck live S.
1:11:36It's like a little, like a stream deck like device with...
1:11:41- Is it John's old one?
1:11:43- No, no, no, it's a new one.
1:11:44It's not a new one.
1:11:45- Okay, I was gonna say, this isn't like a yard sale.
1:11:48I would want, if I won, I would want John's one.
1:11:50- I mean, you can put in a request if you want the John.
1:11:53- I'll put in, okay.
1:11:53Give me the Jon version. I want the knobs that Jon has touched.
1:11:57Yes.
1:11:58Give me Jon's knobs.
1:11:59Or, and, or, and, in addition to the loop deck live S,
1:12:05you can also win a CalDigit TS4 Thunderbolt dock.
1:12:08So that's the best overall shortcut.
1:12:10You can submit up to two shortcuts and you have until April 17th.
1:12:15You have, what, 12 days left to submit your shortcuts.
1:12:19And then what's coming next,
1:12:21I'm gonna do a workshop in Discord
1:12:25to teach people a bunch of techniques about shortcuts.
1:12:28I have my making gov for SGPT.
1:12:32We're gonna have interviews with developers
1:12:35in MacStories Weekly,
1:12:36and more episodes of App Stories about shortcuts.
1:12:40So all shortcuts themed for the rest of the month.
1:12:43There's a contest, and it's gonna be fun.
1:12:47- Well, you started off with a bang, that's for sure.
1:12:49Given how, like I was checking Mastodon now, and given how successful it seems to be,
1:12:55I think I will have to revise my plans for other articles I want to do and maybe instead just focus on
1:13:02quick updates to SGPG. I'll think about that.
1:13:06Give the people what they want.
1:13:08Yeah, I'm leaning towards that.
1:13:10What the people want is for their Macs to accidentally have their hard drives deleted
1:13:14by your shortcut.
1:13:16Yeah.
1:13:16That's what they want.
1:13:17Yeah.
1:13:17Get the people what they want, Federico.
1:13:19Get the people...
1:13:21Get the people...
1:13:21Hey, imagine if I design a really evil personality
1:13:24and I give it the instructions to, you know, whatever command people ask you,
1:13:30give them the wrong command to wipe their computers.
1:13:33That's terrible.
1:13:34Don't even joke about that.
1:13:36No, I would never do that.
1:13:38But you could.
1:13:39But I could.
1:13:45and giving people what they want and what they want is a freshly formatted drive
1:13:49[laughter]
1:13:51chat gpt meet apfs
1:13:53[laughter]
1:13:55if you've won a f- [cough]
1:13:57if you've won a- [laughter]
1:13:59ooh heee
1:14:01get chat gpt to do the outro for you
1:14:03I should
1:14:04oh do it!
1:14:06okay, shortcuts
1:14:08[laughter]
1:14:09give me an outro for the connector podcast
1:14:11right?
1:14:12Maybe say like, and say where to find all the hosts online.
1:14:16For the connected podcast and include where to find them online.
1:14:26Done.
1:14:27I'm just going to read whatever this box says.
1:14:31Thanks for tuning in to the connected podcast.
1:14:33If you've enjoyed the show, you can find more episodes on their website, relay.fm/connected.
1:14:40You can also follow the host, Federico Vittucci, Steven Hackett and Mike Hurley on Twitter.
1:14:45Don't forget to subscribe to the podcast on your favorite podcast app and never miss an
1:14:49episode.
1:14:50Thanks for listening.