0:00:00Do people still buy Xboxes? Is this fight even still on?
0:00:04Yeah, all three competitors are viable.
0:00:08You know, Sony destroyed everybody in the last generation, but Xbox is viable.
0:00:12They have enough money to make a second one, you know, and Nintendo as we know
0:00:16is definitely viable. So the market can sustain three.
0:00:20Each generation, maybe the winner will swap and Nintendo won
0:00:24the Wii generation, Xbox won the PS3 generation.
0:00:28See what happens this time. Sega did not win.
0:00:32It hasn't won anything lately. Sorry.
0:00:36Sega won the Genesis generation and that was it. No, it did not win the Genesis generation. Only in your mind.
0:00:40No, I think... It depends on how you measure, but
0:00:44they did very well. Go look up the numbers. It did fine. It was viable.
0:00:48Viable? I think they were very, very
0:00:52successful during that generation. Sega didn't win, but they were very, very, very
0:00:56strong. But I still think Nintendo won. They won Marco's heart.
0:01:00They did win my heart, dammit. Sega was awesome for that one generation.
0:01:04I've said this many times on the show. I peddled so many Dreamcasts when I worked at Babbage's
0:01:08and that just did not work out well. No. The Dreamcast was not viable.
0:01:12It's sad, because the Dreamcast was a good system with good games,
0:01:16but Sega just blew it so hard after the Genesis
0:01:20that nobody trusted them anymore. I'm not going to say it was a good system. Dreamcast had
0:01:24good games. The system? Oh, come on. It had the little
0:01:28Game Boy memory card. What did they call that? The controller was awful.
0:01:32The visual memory thing was dumb. There it is. Yeah, visual memory unit, the VMU,
0:01:36something like that. It did have a fishing controller, but still. No, that fishing game was amazing.
0:01:40Yeah, actually, this is not for this week, but I have
0:01:44a whole retro gaming setup now, and I'm super into this world, and
0:01:48one of these days I will talk about it on the show. It's
0:01:52been quite a ride. Aren't there Sega games on Switch?
0:01:56You can buy them in a virtual console? Yeah, yeah. Sega became just like, you know, software
0:02:00for everybody now. Yeah, I feel like that's probably your best bet to get all your Sega
0:02:04hopes and dreams on Nintendo consoles. Oh, no. I don't need to.
0:02:08Well, I know you can do MAME. Yeah, yeah. No, I have my
0:02:12Genesis running. You'll see.
0:02:16I don't have time to talk about this week, so we can do it next week if you want.
0:02:20Is that less work than just running them in an emulator? I don't know. Oh, it's way more work, but
0:02:24it's way cooler. Alright, it has a warmer
0:02:28feel. I'll get there. It's all about the
0:02:32ritual. I don't have time this week. Save it for the show. Save it for the show.
0:02:36John, you guessed right about the A12Z. I'm
0:02:40very impressed. Yeah, we were talking about that when the new iPad had just
0:02:44come out and we weren't sure what the deal with the system on a chip
0:02:48was except that it was one letter different from the previous one and it turns out that one letter
0:02:52stood for something that I speculated about on the show, which is that
0:02:56all the parts on it are working. The A12X had seven GPU cores
0:03:00as an insurance policy to increase yields because if one core is a dud
0:03:04then you can still use it. Apparently now they've gotten good enough at manufacturing
0:03:08this particular chip or they just saved all the high-bend ones for this particular thing, something like that.
0:03:12But anyway, this is a common practice in the industry. That's what the A12Z is. It's an
0:03:16A12X without any boo-boos.
0:03:20I wanted to very quickly explain what this is because
0:03:24I bet there are some listeners out there who don't know about this process and it kind of affects a lot of things
0:03:28about chip making. If you ever hear people talk about a chip having low
0:03:32yields or things like this or binning, roughly what
0:03:36this is, and please correct me if I'm getting any parts of this grossly wrong and to any experts in the field, I'm
0:03:40sorry because I only have a passing knowledge of it, but I just wanted to explain this. So basically
0:03:44when you make silicon chips, like CPUs, GPUs and everything, they're made on
0:03:48these giant silicon wafers. You've probably seen them. The big circular cool looking shimmery
0:03:52silicon wafer and they're cut out. There's some rate
0:03:56of flaws of either a flaw in the wafer itself or a flaw in the
0:04:00process that is printing the chip on the wafer.
0:04:04There is some rate of flaws and it's like, okay, well you might have a couple of flaws per
0:04:08square inch or whatever the rate is. And so as you're cutting out
0:04:12those chip dies from that giant wafer, some of them are going to have
0:04:16bad flaws and you won't be able to use them. And yield is literally like
0:04:20the percentage of how many that you make can be used. That's one of the
0:04:24reasons why the larger a chip is, the more expensive it tends to
0:04:28be because you can fit fewer of them per wafer and
0:04:32if there's an imperfection, there's a higher chance of each
0:04:36chip having an imperfection because it has more surface area basically.
0:04:40And so if you think about if there's going to be a certain number of flaws on each wafer
0:04:44there's a higher chance that fewer chips on that wafer will contain
0:04:48no flaws. One of the ways you can deal with some flaws is
0:04:52alright, we're going to manufacture a chip that has eight GPU cores on it.
0:04:56But we're only going to actually spec it to have seven. And so we can
0:05:00crank these out more so and for lower prices because
0:05:04if there's a flaw and it happens to land on one of these GPU cores, we just
0:05:08mark that one as the disabled one. And we enable the other seven
0:05:12and it's fine. And we can still use that chip, we don't have to toss it in the garbage.
0:05:16And so what they've done here basically is, it used to be that. It used to be
0:05:20a chip that had eight physical cores on it for the GPU, which takes up a lot of
0:05:24chip real estate. One of them was just disabled. And which one
0:05:28didn't really matter. So if there happened to be a flaw in that area of the chip
0:05:32they could disable that one and still use the chip and still ship it and
0:05:36make their money. And so all that has changed with the A12X versus Z
0:05:40appears to be that they're now shipping all eight cores enabled. So they've gotten better
0:05:44enough at this manufacturing process or it's gotten cheaper or whatever it is
0:05:48the economics are working out better so that now they don't have to
0:05:52leave that reserve where they don't have to increase the yield
0:05:56artificially by having this one disabled area that they can move around as needed. They can just
0:06:00ship only the ones that have all eight that are flawless.
0:06:04So they saved all the ones that had all eight working. I don't know what the economics of that is
0:06:08but these are small items. If this was a premeditated strategy
0:06:12and they really had awful yields they could have said everyone with eight cores is working, just put that over there
0:06:16we're going to use them later. That seems less likely than what you described but it is
0:06:20a possibility. Yeah, definitely. And then the other thing I wanted to touch on while we're talking about this because it's
0:06:24a very similar thing is the idea of binning. A lot of people don't
0:06:28know that when you get a CPU that has the same CPU
0:06:32available in three different clock speeds. That's the same CPU
0:06:36they're not making, like, okay now we're going to make the 2.5
0:06:40GHz ones and then next week we're going to make the 2.2 GHz ones
0:06:44they're making one chip and certain ones
0:06:48will have whatever flaws or whatever better or worse
0:06:52outcomes of their manufacturing, certain ones will be able to run at higher clock speeds
0:06:56without having flaws. And so they basically test them
0:07:00and they bin them, they sort them into like, this one passed the highest test
0:07:04so we're going to sell this one as the highest clock speed. The ones that
0:07:08couldn't quite run at that speed without errors or flaws but could run at a lower clock speed
0:07:12will bin those down here, will set them by whatever method
0:07:16like burning in certain things and whatever, will fix them to this one clock speed
0:07:20that they tested okay at and so on. And as you can imagine
0:07:24there are fewer that can run at the highest clock speeds. So the highest clock speeds are more
0:07:28expensive because they don't have as many of them. And of all the ones they make, they can make a whole
0:07:32bunch more that run at the lower clock speeds than the few that run at the highest clock speeds.
0:07:36So that's, if you ever hear the term binning, that's what that's about. And it's all
0:07:40between binning and this weird partially disabled chip thing
0:07:44it's all about improving yields when you're making these chips and it's directly
0:07:48related to cost and feasibility and everything else.
0:07:52And game consoles is one area where obviously it's very price sensitive, you're trying to
0:07:56sell what is essentially an entire gaming system for less than the cost of a really good PC
0:08:00video card. So in the past several generations it's been
0:08:04common practice to plan ahead and just assume that you're never going to get perfect
0:08:08chips for your game console and just assume that one or more cores are going to be bad
0:08:12and GPU cores, there's usually more of them and they're more sort of like
0:08:16regular and equivalent to each other, like GPUs in general are more, if you look at them like
0:08:20in the wafer view they have a more regular pattern, it's just repeats
0:08:24of the same block over and over and over again. So it's the perfect, A, it's the
0:08:28area that usually takes up most of the square millimeters on the chip and B, it's the
0:08:32perfect place to say, okay, well we're going to allow one or two of these things to be duds and we're
0:08:36still going to ship it because that's the only way we're going to be able to sell people this thing for $400
0:08:40taking a minimal loss on it or whatever.
0:08:44All right, and we also have learned in the last week that
0:08:48all 2020 iPad Pros have 6 gigs of RAM. So on the 2018s
0:08:52which is what I have and I think one or both of you guys have, those
0:08:56only the 1 terabyte models had 6 gigs of RAM for reasons
0:09:00that I don't think were ever clearly explained, but one way or another, apparently in the
0:09:042020 iPad Pros all of them have 6 gigs of RAM, which is a small but
0:09:08potentially significant difference and if you're really scratching at the bottom of the barrel to
0:09:12find a recent upgrade, Marco, then maybe that could be your get a
0:09:16jail free card. I forget the supposed U1 chip that's in there as well.
0:09:20I've heard since that people are saying no, that's not there after all.
0:09:24I am so happy that these iPads are really
0:09:28good and they're the best iPads we've ever made and all that stuff. I have no interest in this
0:09:32update and that's fine. I use my 11
0:09:36inch 2018 model every day around the house and
0:09:40it's wonderful and there's nothing wrong with it and the one thing that we all
0:09:44want to play with, which is the new iPad Magic Keyboard, isn't out yet
0:09:48and the reviewers don't have it yet and so, and it's compatible
0:09:52with mine, with the 2018 models, which is awesome.
0:09:56I don't know yet, I know that Gruber was talking about this in the talk show, I don't know
0:10:00yet if the magnet arrangement might have been improved on the new one
0:10:04like maybe the Magic Keyboard attached us to the old one but it's not as secure
0:10:08or it's not as good or it's more cumbersome in some way. I don't know, but it appears
0:10:12that the 2020 iPad is almost identical
0:10:16to the 2018 iPad and iPad Pro specifically
0:10:20and that's wonderful because on one level it's
0:10:24a little concerning in the sense that wow, they had almost two years
0:10:28and this is all they really changed and I'm
0:10:32minimizing this. In the world of AR, I think the LIDAR sensor is actually a really big deal
0:10:36because it makes it so much faster to pair or to
0:10:40establish the AR room arrangement and it makes it more stable and everything else.
0:10:44That's wonderful. That's a feature I've never once used my iPad to do
0:10:48so it's not going to be a thing that affects my life at all
0:10:52and so the fact that there's literally like nothing else that's different except a RAM
0:10:56upgrade which I also won't use because I don't multitask almost ever on my iPad.
0:11:00If you had an earlier iPad model and you want what was great about
0:11:04the 2018, great, this is a good time to upgrade. There's a slight improvement. Otherwise
0:11:08though, I see no reason and that's totally fine. Man, what's wrong with me?
0:11:12I don't have the new iPad and don't want it. I don't have a Mac Pro and
0:11:16don't want it. Am I okay? I'm not so sure. I'm getting worried.
0:11:20I mean you're going to get that keyboard when it comes out so don't worry. You'll be able to buy something soon.
0:11:24Thinking about this update though,
0:11:28last show I think we characterized it as a speed bump and it is as we've described but
0:11:32it makes you think
0:11:36Apple has been content to not update the iPads for a long time
0:11:40like with big gaps between them and it never seemed like they were in a rush
0:11:44to get out of speed bump. So I'm thinking like why
0:11:48this time? Why be in a hurry to put any change?
0:11:52Why not just make us wait until the A14X sporting
0:11:56the next big jump? Could be that the next big jump
0:12:00is not coming for a while. It could be
0:12:04that they always wanted to have a big jump now and they couldn't hit that date so they wanted to
0:12:08get something out and they had a bunch of these fast A12s hanging around but this is actually the new
0:12:12camera system from the one that will have the A14 in it. I don't know
0:12:16but it is actually a little bit concerning to me. Despite the fact that this is a good
0:12:20iPad and if you want an iPad you should totally get this one. It's really good. But if you've got the previous one
0:12:24it's not that compelling and it does make me wonder what's happening.
0:12:28I forget who was sending around the rumors. The internal part numbers of
0:12:32the supposed next generation chip. I think it was the A14. They have
0:12:36all these internal part numbers and they've gone through the sequence where they've been increasing
0:12:40the number by one with each thing and supposedly
0:12:44again these are all just rumors. Supposedly the next major chip
0:12:48does not increase the number by one of the previous internal
0:12:52codename model part number thing. It's just an entirely different part number. Like it starts over
0:12:56it's like 101 again where they're up to 108 or 9 or something like that.
0:13:00And that's making people think wow this is like a big
0:13:04change. This next chip is not just an iteration of the previous one
0:13:08or maybe it's the big chip that's going to power all the ARM powered Macs and
0:13:12all sorts of stuff like that. If that ends up being in remotely true it makes me
0:13:16think that this model is a stop gap because the next iPad is not going
0:13:20to be out anytime soon because the next iPad is a big leap because it uses
0:13:24again speculating on these rumors. Maybe it uses a totally new
0:13:28chip that's suitable for use in Mac laptops and as well as
0:13:32iPads and it's a new generation and yada yada yada. And
0:13:36that could be coming out according to the ARM rumors now. What are they saying like 2021 or something?
0:13:40And who knows if that will be delayed by all the virus stuff.
0:13:44So if they waited until 2021 to update
0:13:48the iPad people would be getting a little bit cranky.
0:13:52So that makes this A12Z make more sense. They need something
0:13:56they couldn't wait that long because it would be too long. So give them
0:14:00a better iPad now because we know the next one is not coming along anytime soon.
0:14:04Yeah I think that's an important point that in years past
0:14:08when there were multiple years between iPad
0:14:12updates and this was also exacerbated by the fact that I don't think they were really touching software either
0:14:16but nevertheless Apple would go a couple of years between iPad updates of memory
0:14:20serves and everyone would be like "Dudes! What's going on?"
0:14:24Like can we get a little love for the iPad somewhere, someway, somehow? And yeah this
0:14:28is not a very interesting update in terms of the iPad itself although I am so
0:14:32amped for that keyboard I can't even tell you. But I think the alternative
0:14:36is all of us talking heads start raking Apple over the coals for not even
0:14:40doing a speed bump. So yeah the speed bump may not be the most exciting thing in the world but I do think
0:14:44it's important to recognize that they're giving us what we want. They're showing us that
0:14:48yeah even if they don't have something magical that changes our lives to offer
0:14:52they're still doing something. Admittedly this is very little something but
0:14:56I will take something, some sign that something is happening rather than
0:15:00having the iPad somehow befall the Mac mini timeline
0:15:04where it doesn't get updated like you said John for two or three years. So I'm glad that
0:15:08Apple is doing something even if the something they're doing is a little bit of a snooze to me.
0:15:12You want the speed bumps to be like
0:15:16sooner. Speed bumps are good, it's better than nothing but you'd like to see
0:15:20a big release and then the next one would be a speed bump and then a big release. Some kind of iteration
0:15:24like that whereas this was the normal gap that we would get for a big release and then instead
0:15:28we got a speed bump. That said the last time they did a big update it was the change
0:15:32to these flat sided iPads which everybody loves and they're great right so
0:15:36they deliver when the big ones come down the line. By the way someone in the chat
0:15:40found that thread that I was thinking of, Steve Trout and Smith, these are the
0:15:44are they code numbers? Part names or code numbers? Anyway
0:15:48A10 was the T-8010, A11 was
0:15:52the T-8015 so I guess they didn't go by ones. Anyway
0:15:56the A12 was the T-82X and the A13
0:16:00was the T-83X and the next one is
0:16:04T-8101. So it's the T-80, forget about the T-80s, this is the
0:16:08first in the T-81 line. Does that mean anything?
0:16:12Who knows? Like just, you know, it's all just tea leaf reading
0:16:16from rumors and so on and so forth but it is
0:16:20excitement is building in the world of ARM
0:16:24on the Mac and apparently on the iPad too because you would imagine that whatever
0:16:28the work they're doing to make CPUs for laptops it's
0:16:32basically 100% overlap with the iPads which are already faster than their laptops in
0:16:36most regards so fingers crossed for 2021.
0:16:40We are sponsored this week by Squarespace. Start building your website
0:16:44today at squarespace.com/ATP and enter offer code
0:16:48TPA check out to get 10% off. Make your next move with a beautiful website
0:16:52from Squarespace. Squarespace makes it so easy to make
0:16:56a website and your website with Squarespace can look incredibly nice
0:17:00with pretty much no effort from you and you can customize it to your heart's content
0:17:04you can leave things the way they are and you have a great site right there or you can customize
0:17:08it, you can move stuff around, you can add and remove stuff, you can change the fonts and
0:17:12the colors and everything else to make it really look like yours and your site and your
0:17:16brand and whatever else you might need it to look like. And you have amazing functionality
0:17:20built in with Squarespace. Obviously the basics, you can make a website with
0:17:24all these kind of content pages, blogs and everything else, portfolios, you can also do fancier
0:17:28stuff. You can do things like host podcasts there, you can host storefronts
0:17:32you can have advanced galleries and event calendars and all sorts of dynamic functionality
0:17:36all built in all with Squarespace and all of this is done with
0:17:40easy to use visual tools. You don't need to be a developer or
0:17:44a designer, there is no coding required, you can do this no matter what your skill level
0:17:48is and if you sign up for a whole year up front you can get a free domain
0:17:52name with that as well. All of this is backed by incredible support
0:17:56and you don't need to worry about things like upgrades or patches or
0:18:00server uptime, they take care of all of that for you at Squarespace.
0:18:04See for yourself by starting a free trial, whatever you need to build, try it there
0:18:08first, give it like an hour, I'm telling you, you're going to be amazed what you can get done and
0:18:12how great the result will look and work. See for yourself at squarespace.com/ATP
0:18:16to start that free trial and to really test it out for yourself
0:18:20really build something with it. When you decide to sign up after that, and I know you will, make sure
0:18:24to go back there to squarespace.com/ATP and use offer code ATP
0:18:28to get 10% off your first purchase. That's squarespace.com/ATP
0:18:32code ATP for 10% off your first purchase. Thank you
0:18:36so much to Squarespace for sponsoring our show, make your next move with Squarespace.
0:18:40[Music]
0:18:44May I tell you gentlemen a story? Is it a happy story?
0:18:48Is it a story where awesome things happen and you're smiling
0:18:52throughout and it's just joy? No. That's the best
0:18:56kind. This kind of story still hasn't ended
0:19:00and it's been almost a week. So Casey, what computer are you using?
0:19:04[Laughter] Well done. I wish I could
0:19:08tell you that was a real deep cut, but it's not. No, I'm still on the iMac Pro.
0:19:12Oh thank god. The iMac Pro seems
0:19:16to be fine. My phone seems to be fine, although I will say
0:19:20that there is a very frustrating, I'm going to say gouge,
0:19:24but that's way over selling it. Like nick in the screen right where I
0:19:28scroll that's driving me batty. This is by the way the one that I
0:19:32paid $100 to get. You know, the refurb that I got after I shattered the back
0:19:36like a moron. But anyways, that's also not the problem here. So how did you
0:19:40gouge the screen by the way, side mark? I genuinely don't have the famous idea.
0:19:44Obviously I did something. How do you not know when you put a...
0:19:48I'd sound like I put the phone in the same... You put it in a pocket with your keys? I mean what's going on?
0:19:52I was just about to say it's not like I put my phone in the same pocket as my keys. We're missing the point.
0:19:56We're missing the point. It doesn't matter. It doesn't matter. Remind me never to let Casey use my phone.
0:20:00[Laughter] He doesn't remember how he put a gouge in the screen. It's so big that it's
0:20:04annoying his thumb. Yeah, that's very true. Alright, so I gotta tell you
0:20:08a tale of woe about my sonology.
0:20:12Oh no! Things are not good
0:20:16at the Liz household right now. Didn't hear me telling you that you should basically kill it
0:20:20and get upset? Alright, let me set some ground rules
0:20:24for you guys and the listeners. The ground rules are
0:20:28I am not interested in ways of getting rid of the sonology. We've covered
0:20:32that ad nauseam. I'm not interested. Things I should have or
0:20:36could have done differently, don't want to know. It's too late. Doesn't matter.
0:20:40So don't even bother. Don't care.
0:20:44This gets real bad real quick. You're gonna hear about things you could have done differently.
0:20:48I know you're trying to set down rules, but you're gonna... I mean from us.
0:20:52Oh no! You guys all allow it. What do you think this show is? Exactly.
0:20:56Alright, so let me set the stage. In 2013, some very nice
0:21:00individuals at Sonology sent all three of us DS1813+
0:21:04That is the particular model name of our sonologies
0:21:08and it basically means that they were eight
0:21:12bay sonologies in the, I believe, small business line
0:21:16from 2013. And at the time they sent us
0:21:20they were filled with eight
0:21:24three terabyte, what are they, Seagate Red's, I think, hard drives?
0:21:28Western Digital Red. Sorry, yes, what you said. I apologize.
0:21:32In the last seven-ish years, I don't recall exactly when
0:21:36in 2013 we got them, I think I've had to replace
0:21:40one, maybe two drives. The way I have set up
0:21:44my sonology, which I think was Marco's idea, maybe it was, it must have been John's, because I always
0:21:48think it's Marco and it's always John. Somebody told me, "Hey, take two drives, make them time
0:21:52machine, take the other six and make them one humongous volume for everything else." This time this is actually
0:21:56me. Okay, see, there you go, I actually remembered something. Look at me go.
0:22:00So, anyway, so drives, not drives, I'm gonna go
0:22:04one based, so one and two, our time machine, three, four, five, six,
0:22:08seven, eight are all lumped into one big volume using
0:22:12sonology hybrid rate, or I think it's hybrid rate, SHR, which basically
0:22:16means I can lose one of them and everything's fine. And
0:22:20I noticed that all of these drives,
0:22:24four, five, six, and eight were all OGs from 2013.
0:22:28So these are seven-year-old hard drives that have been running non-stop
0:22:32for seven years. I just want to interject here that
0:22:36I am using all of those same drives, those eight
0:22:40three terabyte drives that I got seven years ago, all working perfectly.
0:22:44Well, I hope so, and I hope you have some wood to knock on.
0:22:48I'm just saying, I'm not gonna blame it on the fact that you put it in
0:22:52the same room with you, but that's probably why. Well, yeah, yours is in the basement
0:22:56or something, isn't it, right? Yeah, climate control, dark,
0:23:00nobody disturbs it. Mine is in my un-temperature controlled
0:23:04garage, and it's fine, I've never lost a drive. I cannot believe that.
0:23:08I cannot believe that yours is still working, but anyways, so be that as it may,
0:23:12I have four, five, six, and eight that are all the originals from 2013.
0:23:16Three and seven have been replaced with
0:23:20ten terabyte Western Digital Reds. I'm slowly going
0:23:24to replace all these, put these ten terabyte drives in, and again
0:23:28four, five, six, eight. I would like to replace one of these
0:23:32in order to try to get ahead of all of these eventually failing.
0:23:36This is the first part where we'll talk about what Casey could have done differently. What you're
0:23:40doing now is taking a thing that ain't broke and you're trying to fix it.
0:23:44Yes. Right? Like you did not have any bad drives, but you decided preemptively
0:23:48I'm going to take my working Synology and I'm going
0:23:52to endeavor to upgrade it to stave off
0:23:56what I think is the imminent failure of my seven year old drives.
0:24:00Right. So, why do you think I would have made that choice?
0:24:04Do you get the little emails from Synology that tells you how many bad
0:24:08sectors it's finding on your disks? Do you get those? Mm-hmm. Have those numbers been going up
0:24:12lately? No. Are any of those numbers non-zero? I don't believe they were
0:24:16non-zero now. So, in theory, just to be clear,
0:24:20in theory drives four, five, six, and eight were all fine.
0:24:24In theory. And by the way, I know we all talk about our Synology love, but the thing I just described
0:24:28is a real thing. The Synology has a feature where it will email you weekly
0:24:32or monthly or whatever updates on your drive health and it will tell you if it has found any bad
0:24:36sectors. And if you get those emails and it starts saying that the bad sector count is increasing,
0:24:40your drive's probably going bad and you should do something about it. That happened to me on my
0:24:44other Synology, which, it says bad sector's found or whatever, and it does a consistency check.
0:24:48My other Synology, which does not have Western Digital Reds, one of my drives
0:24:52said I found one bad sector and the next week it found two and the next week it found like
0:24:5637 and I was like, "Okay! Time to replace that sucker." So,
0:25:00another cool feature of this NAS. Yeah, it also tells you
0:25:04how many smart, you know, like the acronym SMART, how many smart errors
0:25:08and disconnects there have been. Generally speaking, when it comes to this kind of thing
0:25:12from a hard drive, it's kind of like, you know, I was going down a hill with my car
0:25:16and I pressed the brakes and once they didn't work. Anything besides
0:25:20zero on these numbers means replace it immediately.
0:25:24Well, that's the Marco approach, but let me give you another story.
0:25:28One of my drives in my Synology has had one bad sector
0:25:32since like the first week I got it. That number is always one. It's been one for seven years.
0:25:36Like, you're right that there is usually a cascade, but drives have bad sectors
0:25:40and usually they're found and mapped out before you get the drive, but it is conceivable
0:25:44that you get one out of the box and it finds a bad sector and it maps it out.
0:25:48And if that number never goes up for seven years, like, I mean it may go up someday, right?
0:25:52But, you know, if you see one and it doesn't change for months and months,
0:25:56fine. But you do have to watch them. Like, if it shows from one to two to five to thirty-seven
0:26:00to a hundred and forty-five, like, replace the drive. I don't think I've ever had a bad
0:26:04sector on a hard drive that didn't then die.
0:26:08I've got seven years with one bad sector. One of my drives did. I just got an email
0:26:12the other day. It's still there. It's not going to go away. It's not going to go back to zero.
0:26:16So why would I try to do this preemptively? Why would I do that?
0:26:20Because you're home all day and you've got nothing to do and you're going to break your working computer.
0:26:24Go ahead. That's part of it. Maybe it's because you were worried that, you know,
0:26:28because it's kind of like, so when you have to replace a drive in a RAID
0:26:32array, and yes, SHR is not exactly RAID. It's kind of like what Drobos do.
0:26:36It's like kind of software expandable kind of thing. But, you know, it's close enough to RAID.
0:26:40So when you have to do a RAID rebuild, one of the common problems is
0:26:44like, a RAID rebuild requires a lot of disk activity
0:26:48because you have to read the entire rest of every other disk
0:26:52to write what is necessary onto the replacement disk. So actually
0:26:56one thing that can happen is kind of like the, you know, dead cat box thing of like
0:27:00you might think everything's fine. Shringer's cat? Is that what you're trying to say?
0:27:04Dead cat box is not what it's called. Go ahead.
0:27:08But like, so you might think everything's fine, but then the second you replace
0:27:12a disk, it has to way more
0:27:16than usual stress the other disks that were in the system
0:27:20to rebuild the array onto the new disk. So you actually kind of risk a bad
0:27:24cascade, which I hope this isn't what happened to you, where like you might think everything's fine
0:27:28replace one disk and then another disk or two
0:27:32die during the rebuild because it's stressing them out so much.
0:27:36So, on Thursday, I say to myself, I've got 4, 5,
0:27:406, and 8 that I'd like to replace. I'm going to take 5
0:27:44because it is in the middle of, physically in the middle of 4 and 6.
0:27:48Sure, let's go with that one. Completely arbitrary seems like a reasonable approach, right? Why not?
0:27:52I replace drive 5, I tell the Synology, okay rebuild yourself, I go to sleep.
0:27:56I wake up on Friday, the 27th of March
0:28:00and the volume has crashed, which is Synology speak for you have lost
0:28:04everything. Drive 8, during the restore
0:28:08had one or more IO errors and died.
0:28:12So, I replaced drive 5 in the process of
0:28:16fixing everything and putting everything back to where it was or where it should be
0:28:20drive 8 sh*ts the bed. Well, now I've got problems.
0:28:24So... No wait, quick question. Can you just put
0:28:28the old drive 5 back in? That's a good question
0:28:32but John, I'd like to hear your thoughts before I answer it.
0:28:36This is not shocking and what Marco suggested is definitely what
0:28:40I would have tried first, but in the list of things you could do
0:28:44differently in general because if you're worried
0:28:48about your drives because, you know, whatever, I forget what the word is, the old raid
0:28:52word used to be called resilvering, I forget what the hell they call it, but anyway, because doing what you're doing, replacing
0:28:56a drive is so taxing on the other drives. If the reason you're doing it is you're worried
0:29:00about all the drives in your thing, in general it's safer to try to
0:29:04do a one time copy of everything off to another volume
0:29:08like, you know, if you insisted on doing this, if you're in a situation where
0:29:12you're like, I feel bad, I feel like these drives are going to go at any minute, what
0:29:16can I do? If you don't already have a complete backup, which obviously you should, but if you don't
0:29:20already have a complete backup, what you'd want to do is get something
0:29:24that can receive all the data and then copy all the data off all the drives and put it onto this
0:29:28new place. Then, after you've successfully done that and you take that safe copy
0:29:32of all your data and you put it somewhere and you unplug it and you just sit in the corner of the room, then you go try
0:29:36to do this whole, you know, swap out drives thing, so then when it blows up like this
0:29:40you're like, oh well, I tried, and now you've got your data in the other place. But anyway,
0:29:44continue. What did you actually do? Did you try Marco's thing of putting back in the five and
0:29:48swapping out the eight? Not exactly, and I have reasons. Let me
0:29:52explain my thought process. You may not agree, but I have a thought process here.
0:29:56So drive eight dies, I walk in and I am on the verge of tears
0:30:00and that is no exaggeration because everything in theory is gone.
0:30:04Now I do have backup, I know I have a couple of backups
0:30:08of my photos, that is not up for grabs, and that is the one thing that I
0:30:12cannot lose. I would be, and in this moment in the story,
0:30:16I am devastated that I've lost other things. But
0:30:20I would be beyond repair as a human being
0:30:24if I lost all my pictures. Did you really lose other things? I thought you had
0:30:28didn't you, did you have online backup? Did you have like your parents
0:30:32backup? Like what was your backup situation? Gentlemen, this is my story, I will tell it in my own time.
0:30:36Oh no. So you were devastated, we're up to the point
0:30:40where you're devastated because you think you've lost stuff. That means at this point in the story, you think
0:30:44you don't have some of that data in another location, right? I don't
0:30:48I am not confident that everything is the way I want it
0:30:52to be in terms of backups. So let me take you through my situation. I know
0:30:56I have my photos on an external hard drive that normally is at mom and dad's
0:31:00but because of the quarantine is actually sitting in the garage. But I know
0:31:04they're there, I have confidence that I'm okay.
0:31:08That doesn't happen to be the drive you put into Bay 5, is it? No, no, no, no, no, no.
0:31:12This is an external, this is an external sitting off to the side.
0:31:16And I don't think I've updated it in a month because I usually do that at the top
0:31:20of the month. But I mean, I don't, maybe I lost a week or two of photos.
0:31:24And even if that's the case, like okay. It's not desirable but fine.
0:31:28I would be okay with that. I still have the last 20 years of photos, whatever it's been.
0:31:32But everything else, I'm not sure.
0:31:36So I've been awake for 30 minutes at this point. It's Friday morning.
0:31:40I walk into my office and I go, "Oh God, oh God, oh God, oh God, oh God, oh God."
0:31:44And I look at Backblaze, which I've been running on my iMac and I know hadn't quite finished
0:31:48uploading everything but I thought it was really close. And it turns out
0:31:52Backblaze is not as close to done as I thought for my own, because of dumb
0:31:56things I've done that I'm not going to get into right now. So okay, next thing.
0:32:00I need to interject again. You're in the middle of running a gigantic Backblaze backup, right?
0:32:04Yep. This is a good time to rebuild my array. You have the itch, you have the itch
0:32:08to do this disk thing. Why didn't you wait for the Backblaze thing to finish? I agree.
0:32:12No argument. That absolutely was a mistake on my part. Absolutely a mistake
0:32:16on my part. No argument. There's nothing I can say to defend myself. This is like one of the situations where, you know, when they have
0:32:20like an analysis of catastrophes, it's never like one thing went wrong. It has to
0:32:24if you do one thing wrong, like usually the system or whatever recovers
0:32:28but it has to be a series of small errors that build on each other. You know what I mean?
0:32:32Mm-hmm. All right. Go on. All right. So crash plan, I
0:32:36haven't run in at least a month, maybe two months. And I
0:32:40have had a reminder for myself to cancel it for at least a week, maybe two weeks.
0:32:44But I didn't. So. Finally,
0:32:48procrastination pays off. Exactly. So in an absolute
0:32:52desperation scenario, I would lose a couple of months worth of other
0:32:56things. Again, photos, I'm confident I'm okay. But everything else, I'd lose a
0:33:00couple months of that, but I'd be okay. Now, restoring from crash plan would be a friggin' nightmare
0:33:04because they don't do that awesome thing that Backblaze does where they'll just send you a physical hard drive
0:33:08and then you can send it back and no harm, no foul. I would have to actually download
0:33:12everything from crash plan, which would stink even on a gigabit connection. But
0:33:16I could get it. That's all that matters. I could get it. So
0:33:20I know my pictures are safe, but at this point I think all of these
0:33:24TV shows that I've acquired by gray means, like
0:33:28using YouTube DL to download stuff off of NBC's website, for example.
0:33:32I'm not signing into anything to get it. It's available for anyone. You can do it on
0:33:36any one of you could do it, but it's work I've put in to download all these shows and sometimes they take
0:33:40them off the internet and I might want to watch them again sometime. And another great example
0:33:44is concerts. I really enjoy watching concerts with Aaron. And
0:33:48I will take the time to use an app called Subler to like go through and add chapters
0:33:52for all the different songs so I can skip around if I want to. You can think that's crazy.
0:33:56That's fine. This is me. This is the way I like to do things. All that stuff.
0:34:00Gone. So I think, "Oh God, oh God, oh God, what do I do?"
0:34:04The thought did occur to me to turn the
0:34:08Synology back off, put Drive 5 back in, and just
0:34:12go from there. Turn it on whistling.
0:34:16Right, exactly. Nothing to see here.
0:34:20So anyway, I didn't do that. And before you jump all over me, let me tell you why.
0:34:24What I thought was, "Okay, the first step is let's restart this thing
0:34:28as is and just see what the state of the world is."
0:34:32So I left Drive 5 with this new 10 terabyte drive, and
0:34:36I reboot it. And it comes up and it says,
0:34:40"Things are in..." I forget the term for it on the Synology. It's degraded.
0:34:44That's it. Things are degraded, but they're there.
0:34:48Drive 8's hiccup is just a hiccup.
0:34:52Everything is still there. Now there is no Drive 5, according to Synology, because it's empty.
0:34:56Like that makes perfect sense. But it's there. And at this point I have
0:35:00to make a choice. I'm not saying I made the right choice, but what I decided was,
0:35:04"Okay, at this moment in time, Drive 8
0:35:08still works." And it has already failed once.
0:35:12I don't know if breathing on this thing
0:35:16is going to cause it to fail again. So the very first order of
0:35:20operations, as far as I'm concerned, is get all this data off this
0:35:24thing as quickly as I possibly can. And that's
0:35:28what I did. So what I did, and I will give you a chance to respond.
0:35:32Hold on, wait, wait, wait. Just hold on. Hold on. Just give me a chance.
0:35:36Just give me a chance. You have to make it through the gauntlet of telling us this story.
0:35:40At various points I haven't told you the different things that I would have done.
0:35:44No, not yet. Just give me 30 seconds. I have the talking stick.
0:35:48So I decide to... Now, mind you, we are in big time
0:35:52lockdown quarantine mode. We are the worst friends. I look at Amazon to see
0:35:56what can I do? Is there a drive big enough that I can prime now
0:36:00on Amazon to get all this stuff off the Synology? The answer is no.
0:36:04But Best Buy is delivering to the
0:36:08parking lot of Best Buy, and they happen to have
0:36:12literally one 12 terabyte external available for purchase.
0:36:16And so that's what I did. And then I connected it to my Mac Mini
0:36:20and started r-syncing the entirety of my Synology to this
0:36:24external drive. And that is where I will pause and allow you to beat the hell out of me.
0:36:28So I understand your thing. You want to get the data off, right?
0:36:32It comes back up. Drive 5 is empty. Drive 8, you're scared, is going to break.
0:36:36Again, why didn't you take out the empty drive 5 and put back the full drive 5?
0:36:40So that way, if 8 breaks, you still have all your data.
0:36:44I 100% agree with you. And in retrospect, maybe I should have, but at this
0:36:48point, I was so nervous that touching anything
0:36:52would screw it up that I knew at this one moment,
0:36:56at this one moment, everything was working. So I have to capitalize on it
0:37:00while I still can. But you didn't know that. Half your data on Drive 8 could be corrupt.
0:37:04You have no idea what's going on with Drive 8. You know it had some kind of I/O error. You're assuming you can get
0:37:08all the data off it. Anyway, that's what I would have done differently at that point. As soon as, if it came back up
0:37:12and if you wanted to get your data off of it, which I suggested earlier would have been the first step,
0:37:16but fine, you're going to do it now, I would have put the 5 back in. So that way the 8
0:37:20could die and I'd still have all my data. Well, one minor complicating factor here, if the array
0:37:24was online at the time, it could have been written to in the meantime.
0:37:28And so we will come to a point where I will answer this question and I think
0:37:32that is exactly what happened. And if that is the case, then your old Drive
0:37:365 is now out of date and is basically corrupt. Yep.
0:37:40Yeah, there was no way to, I mean, what's being
0:37:44written to it? Do you have like jobs running in the background pulling down things from feeds?
0:37:48I honestly don't think so, I don't know, but. Well, and regardless of like
0:37:52no matter what it is, even if it's one byte somewhere, if the RAID controller
0:37:56like keeps track of this, which it should, then it will say, "Sorry,
0:38:00this disk, this old disk 5 is now out of date, we can't use it." And so
0:38:04I will cut to the almost end of the story very quickly and say once
0:38:08I got all 12 terabytes off of that, or well it was like a 10 and a half
0:38:12terabytes off the Synology, onto the external, which seems as far as I can tell
0:38:16to have gone just fine. I did at that point, mind you this is
0:38:20admittedly literally four or five days later, I did plug in Drive 5
0:38:24because at this point I thought, I don't want to lose everything, but if I did
0:38:28I now have a complete backup so it's okay. And I did
0:38:32try plugging in Drive 5 and it absolutely said this drive is useless to me, do you
0:38:36want to, you know, bring it back up as though it's new? Well, that was five days later, I
0:38:40can imagine stuff would have happened in five days. Agreed, agreed, but
0:38:44again, my thought process was, the only thing I care
0:38:48about is getting this data off this box, that's all I care about. And as
0:38:52long as I can do that, anything else is secondary. So
0:38:56I am in this degraded state, I ordered this 12 terabyte from
0:39:00Best Buy, and within a couple hours I have it, within an hour of that it's
0:39:04hooked up to the Mac Mini, the Mac Minis are syncing literally 2.2 million files
0:39:08and something like 10 and a half terabytes to this 12 terabyte drive.
0:39:12So, at the same time, because I'm a moron but I'm really freaking out now,
0:39:16I'm also manually r-syncing a handful of things that I just really don't want to lose.
0:39:20So I'm getting a third copy of my photos, a second copy
0:39:24or third copy, I'm losing count now, of some of the kid shows that
0:39:28the kids adore that I've spent a long time amassing. I got
0:39:32another copy of Top Gear, you know, literally 25 seasons of Top Gear that I have
0:39:36and I do occasionally go back and watch some of all of these concerts and
0:39:40things like that, and just some other essentials that I really don't want to lose. So that was
0:39:44started on Friday, this past Tuesday, yesterday,
0:39:48the 11-ish terabytes and 2.2 million files are complete.
0:39:52So what do I do? And I already told you, I thought, alright, let me just try throwing in that
0:39:56old drive 5, obviously it didn't work, I already told you it didn't work, but
0:40:00let me try it. So now, what do I do?
0:40:04Well, the only real option I have
0:40:08other than just nuking it all from orbit and starting anew, like you were saying, John,
0:40:12is, well, I take that 10 terabyte that's been sitting in drive 5
0:40:16all this time, and just try again, and hope that 8 holds on
0:40:20long enough. So, I thought to myself, what can I do to make
0:40:24this work, or what can I do to improve my odds? And although
0:40:28this room, outside of when I'm recording and the door is closed, this room
0:40:32doesn't get very hot, I thought to myself, well, heat is the enemy
0:40:36of everything, so I got a box fan and sat it on the filing cabinet that's like
0:40:402 or 3 feet away from the Synology, and I have been blasting this box fan up until this recording
0:40:44on the Synology for a day or so
0:40:48while it's rebuilding itself, hoping, hoping, against all odds, that drive 8
0:40:52will hold on long enough for it to repair itself. And then, thankfully
0:40:56I actually have another 10 terabyte that's brand new waiting
0:41:00just in case one of these dies, and I would put that in drive 8
0:41:04and then hopefully, what did I say, the other ones were 4, 4 and 6
0:41:08don't die in the same way as I'm replacing drive 8. So
0:41:12it takes about a day for my particular Synology to put everything back together
0:41:16and I'm watching this like I'm watching the school clock
0:41:20at the end of the day on Friday, I'm watching it like a hawk. And just
0:41:24a couple hours ago as we record this, I see it tick, well, a few hours
0:41:28ago I see it tick past 50%, which is I know roughly where it was when drive 8 died.
0:41:32I see it get past 60, 70, 80, 90, I even took a
0:41:36screenshot at 99.5% thinking, this is when it's going to die
0:41:40and that will make for a hell of a great story, but I'm going to be miserable. So I want
0:41:44to get a screenshot so we can put it in the show notes, so when it dies at 99.5%
0:41:48at least we'll all get a good laugh out of it. During this process
0:41:52however, I realized, oh, today is April 1st.
0:41:56Guess what happens on April 1st, John?
0:42:00It's time for a smart test on all of my drives.
0:42:04The monthly, yeah. Indeed.
0:42:08So while I'm doing this restore, using
0:42:12a drive that is on the edge of death as far as I'm concerned, the Synology says to itself
0:42:16you know what would be a great idea right now? Let's do a smart test. And
0:42:20as we're recording, which is in the evening on Wednesday,
0:42:24let's see, the smart test has only been completed on five of the eight drives.
0:42:28The other three are sitting at 90%. So naturally that's slowing
0:42:32everything down, it's further hammering these drives that are probably on the verge of
0:42:36catching on fire. And so as I'm watching this tick up, finally
0:42:40right around bedtime for the kids I reach 100%, I get my email saying
0:42:44volume two, which is the volume I'm talking about,
0:42:48the consistency check of storage pool two on disk station
0:42:52has ended. No abnormality has been found. And at this point I expect to
0:42:56see the degraded or repairing become
0:43:00normal, and everything should be good, right? Well, no.
0:43:04Apparently, for reasons that I don't understand, it has decided to
0:43:08start its consistency check over at 0%. So
0:43:12as I sit here right now, it has gone from 0 to 100, back
0:43:16to repairing, checking parity consistency 2.01%
0:43:20as I record right now. So maybe in another day it'll
0:43:24work this time? I don't even know. But I am so freaking
0:43:28miserable and annoyed and upset, and this is like the most ridiculous
0:43:32just inconsequential problem in the world, but I'm about to rip all of my hair out
0:43:36and go crazy. Is the box fan off now? The box fan's
0:43:40off now. Well, here, I can turn it on for you. That's alright.
0:43:44How loud would it be? Because you're basically, you could be sacrificing your data to podcast.
0:43:48There it is. Do you hear it? It's on now.
0:43:52Your data's fine now, right? Well, the data is fine in the
0:43:56- I gotta turn this box fan back off before Marco kills me. - Thank you.
0:44:00The data is hypothetically fine, insofar as the
0:44:04volume on the Synology is functional at this moment.
0:44:08Like, it's not happy, but I can pull data off of it. I guess hypothetically
0:44:12I could put data on it, although I'm trying my darndest not to. It is theoretically functional.
0:44:16Additionally, that 12 terabyte I got from Best Buy does,
0:44:20as far as I can tell, have a complete duplicate of everything on the Synology.
0:44:24So, what file system did you use? APFS.
0:44:28No, I'm not on that. On the Synology. Oh, on the Synology? I don't know.
0:44:32Does it offer ZFS that would actually make this useful?
0:44:36As BTRFS and EXT4?
0:44:40I think it's EXT4. Here's the thing about you wondering about the state
0:44:44of all your data. Like, oh, I got that copy off under the 12 gigabyte driver. You know
0:44:48you have drives that are having some kind of problem. If you're using a file system like most file systems
0:44:52that doesn't do any kind of consistency check of the data,
0:44:56it's like, I asked the drive to read it, these are the bits that came off the disk, here you go.
0:45:00And you successfully copied those bits to another place, and another place dutifully stored them.
0:45:04Are those the right bits? That's why file systems
0:45:08with data integrity checks like ZFS are handy, because it can tell when
0:45:12the data right off the disk is not the data that was originally written there, and there are other applications
0:45:16that can store checksums off to the side, so on and so forth. EXT4, which I think
0:45:20was the default when we all got our Synologies, does not have any features like that.
0:45:24So, if your data went bad, I mean, the good thing is with media files,
0:45:28so you got a few bad blocks here and there, you'll see a weird glitch, or maybe you won't even
0:45:32see a weird glitch in the video, because it's a lossy compressed thing, and it can tolerate
0:45:36errors, and it's not like it's an executable program where it might
0:45:40not even launch if the wrong part of it is corrupt. So for media files, it's probably not
0:45:44actually that bad, but not knowing, you know, and propagating
0:45:48bitrod is always a problem, which is why it's better to do these things
0:45:52before things go wrong, or to go whole hog and have some
0:45:56kind of checksumming system. Checksumming is more reasonable when you have a small
0:46:00number of very large files, but it sounds like you have a large number
0:46:04of large files, like 2.2 million files is a lot. It's nothing compared, like
0:46:08I have way more than that on my boot drive on my Mac, but they're all tiny.
0:46:12Media files tend to be big, which is usually pretty good for dealing with files, because big files
0:46:16means long sequential reads, means less metadata to shuffle around,
0:46:20and if you're going to store a bunch of checksums offline using some other third-party program,
0:46:24the fewer checksums you have to store, the better. But it sounds like you didn't have
0:46:28any of that stuff there, so you're just kind of crossing
0:46:32your fingers, hoping that nothing was corrupt, because otherwise
0:46:36you just successfully copied the corrupt data to your Best Buy drive.
0:46:40Indeed, but I don't feel like I have a whole lot of other choices. So,
0:46:44the funny thing about all this is, in the end, after going around
0:46:48and around for like two or three episodes with you two about what I should do to back up the Synology,
0:46:52the future, once I get the Synology itself squared away, the future
0:46:56backup approach is going to be hook up that Best Buy hard
0:47:00drive to the Mac Mini, and have Backblaze back that up. You just did that, you just
0:47:04did the backup, right? Well, in theory. Well, yeah, but not to Backblaze. The Backblaze one
0:47:08is still in some sort of limbo for uninteresting reasons that I'm not going to go into
0:47:12right now. But as soon as I get the Synology to a good place,
0:47:16even if that means nuking it all from orbit, I will then be permanently
0:47:20hooking this external up to the Mac Mini and having that go to Backblaze
0:47:24and just calling it a day. But, gentlemen, it has been an adventure.
0:47:28Can I suggest some things? If it involves getting rid of the Synology, no.
0:47:32Well, okay. First of all, we'll get there.
0:47:36Step one, go get yourself
0:47:40a second 12 terabyte drive. I don't care if you want to return it
0:47:44in a few weeks, just get yourself one. First thing you need to do is copy everything
0:47:48from this 12 terabyte drive to a second 12 terabyte drive.
0:47:52After you do that, connect it to a computer that will
0:47:56finish the online backup and back it up. During that process,
0:48:00so, okay, so copy it onto the second drive, so you have two new
0:48:04hard drives that have this. The second one,
0:48:08remove it from the computer and power it down. The Synology,
0:48:12turn it off and keep it off until you have an online
0:48:16backup of that entire dataset. Just because
0:48:20if things are, I know certain times, sometimes powering things down can actually make them die
0:48:24and actually turn them on, but for the most part that's rare. Usually things wear with usage
0:48:28and so, for the love of God, freeze this data in
0:48:32place, like until it is securely backed up somewhere useful.
0:48:36After that, I want you to consider, you mentioned
0:48:40recently that you don't use Dropbox or iCloud Photo
0:48:44Library anymore. The lack of iCloud Photo Library in your
0:48:48setup has just been made apparent by some of your descriptions of your photo status here.
0:48:52Why? I know Apple's
0:48:56not wonderful at certain services. I know that the plans are not
0:49:00free to get enough storage. I know that
0:49:04iCloud Photo Library doesn't have all the features of things like Google or whatever, who cares.
0:49:08But, in this case, this data is so important to you
0:49:12that I think whatever cost it would be, which, I mean you could probably
0:49:16get away with what, the $10 a month one probably, right? Whatever cost it would be
0:49:20for you to have enough storage to have your entire photo library in iCloud Photo Library
0:49:24is probably worth it, just for considering it as an
0:49:28automatic off-site backup alone. Not to mention all the other
0:49:32convenience features of it and the integrations and everything else. But just to have
0:49:36that be somewhere else that you know that your photos
0:49:40have one additional layer of backup safety,
0:49:44whether it's iCloud or Google, I don't care which one you use, but one of those you should
0:49:48be using. I personally would recommend the iCloud one because I think Google Photos has a bunch of weird
0:49:52issues with their local uploader apps, but that's up to you.
0:49:56You know all this too. None of this is news to you, but I think this should
0:50:00inform that decision that you should probably have
0:50:04a photo, an online photo backup service in place
0:50:08because they exist, they're pretty good, even among Apple's reputation
0:50:12for weird service problems, most of those problems haven't hit
0:50:16iCloud Photo Library. It's been pretty good.
0:50:20For me, it's been perfect, honestly, as far as I know. But I haven't even heard
0:50:24of major issues from other people with iCloud Photo Library. It seems to work very well.
0:50:28So that is, I think, whatever you
0:50:32choose to do, your strategy I think should include one of these online photo
0:50:36services because they're well integrated, they're automatic, and
0:50:40for something as important as your family photos, having that
0:50:44additional backup plus one layer
0:50:48I think is worth it. So moving on from that,
0:50:52you are running a whole bunch of seven year old drives inside of a seven year old
0:50:56RAID enclosure. This entire setup is dead to you.
0:51:00Now, whether you want to replace it with
0:51:04a better, with like a new Synology, that's
0:51:08up to you. I wouldn't, as we've talked about, and I don't think you necessarily
0:51:12need it, but I buy stuff I don't need all the time because I like it.
0:51:16You know, you only live once, so sometimes you just buy stuff
0:51:20because you like it. So I won't follow you if you want to
0:51:24replace the Synology with another Synology. But
0:51:28one thing people don't always consider when they're thinking about RAID
0:51:32or RAID-like things is the RAID controller itself as a
0:51:36potential point of failure. But this happens. RAID controllers die, or flake out,
0:51:40or have problems. I'm not sure I would trust any
0:51:44part of my storage infrastructure that was like
0:51:48critical primary storage that was seven years old. I think at that
0:51:52point, you're rolling the dice more than necessary. And if
0:51:56you're going to really count on something that is that old as something that is not
0:52:00well backed up, you know, I think that is taking
0:52:04too much risk. So, A, for God's sake, get
0:52:08iCloud Photo Library. I know it's not perfect. Suck it up. Whatever problems you have with it, suck up
0:52:12those problems. If I can live in the same world as Dave Matthews, you can get
0:52:16out of Photo Library. It'll be fine. And so,
0:52:20A, do that. B, get yourself a second
0:52:24hard drive and copy all these files onto it and turn off the Synology
0:52:28because, C, you need to preserve that until the online
0:52:32backup is done. And D, after that, you should really retire
0:52:36the Synology. And what you do to replace it, that's up to you.
0:52:40But you're playing with fire here. There's no evidence that
0:52:44the Synology hardware other than the hard drives is bad, right?
0:52:48Agreed. So, I know you're suspicious of it because it's old, but
0:52:52as someone who just got through using a 10+ year old computer that worked
0:52:56fine the whole time, hard drives, they're moving parts.
0:53:00They go bad, for sure. He's got some bad hard drives in there, sounds like, right?
0:53:04But for a low-powered CPU that's doing the same task
0:53:08all the time with no GPU to speak up,
0:53:12in the absence of any evidence that there's anything wrong with this Synology hardware
0:53:16other than that it's got a bunch of old, crappy drives in it, I don't think he needs to go on and buy a new Synology.
0:53:20Now, that said, new Synologies are cool, and I've thought about getting one just because they're cool, exactly
0:53:24for the reasons you said. There's nothing wrong with my Synology. In fact, all my drives are still working.
0:53:28There's no reason I should get a new one, but I look at the new ones every once in a while because they're cool.
0:53:32They have better CPUs and fancier file systems and other neat
0:53:36features. But, at this point, I'm still just waiting for mine to die. But it sounds like
0:53:40Casey may be in, may have had enough
0:53:44pain and anguish that as a reward for himself, as a treat,
0:53:48he could get himself a new Synology. Sounds like he's already getting a bunch of new drives, right?
0:53:52So once you're doing that, you can get a new Synology too. And then you can use the old Synology
0:53:56which I believe will be resurrected, and use it to back up the new Synology.
0:54:00That actually is not a bad idea. In the money-no-object
0:54:04perfect world, that is what I would do, is I would get a new one
0:54:08and I keep saying, "Oh, I need a new 8 bay, I need a new 8 bay." But
0:54:12now that I'm thinking about it more, and listeners had said this to me, I could
0:54:16just put bigger drives in like a 2 or 4 bay, probably like a
0:54:204 bay. I think 4 is the minimum that you should get
0:54:24and probably the right number. Yeah, we're saying the same thing though. That I could get like a
0:54:284 bay Synology, and that would probably be enough. And then I'll put
0:54:32more smaller drives in the existing ones, stick that at mom and dad's house,
0:54:36and then I have a live, or nearly live, duplicate
0:54:40of the one in the house. And I might do that, I don't know, it depends
0:54:44on how grumpy I am with all this whenever I come out the other end of it, be that if I have to nuke it from
0:54:48orbit, be that if it does finally repair itself, I don't know.
0:54:52Yeah, and speaking of nuking it, by the way, that will be faster than letting it validate
0:54:56itself again. If you just wipe it clean and re-copy the data onto it from one of your
0:55:00two 12-derafight backup drives, I think that will take less time than allowing it to
0:55:04re-verify, based on my experience. Oh, almost certainly. Trying to do re-verification.
0:55:08Yeah, because if you figure like verification is going to be a very like
0:55:12you know, a small block read-write-read-write-read-write kind of cycle, whereas
0:55:16like, you know, re-copying onto it is going to be these like giant block transfers
0:55:20and it only has to do like one pass with the writing, whereas
0:55:24like if you're rebuilding, it has to like, you know, to write the
0:55:28parity back on one new drive, it has to read everything off of all the drives, right? Then
0:55:32you're going to have to put in another new drive, it's going to do the exact same process again.
0:55:36And re-read everything again for that next drive, and like, you're going to have to go through the whole process what, four
0:55:40times? Something like that, yeah. And I
0:55:44hear you, but at the same time, it took four
0:55:48days to rsync all of these files from the
0:55:52Macbenny sitting literally underneath the thing. That's because you had multiple rsync
0:55:56jobs running at once. Lesson number one for copying large volumes of data off
0:56:00of a spinning disk, just do one copy at a time. You think you're going faster by having five
0:56:04copies in parallel, you're not, you're just angering the disk, you're just making the disk controller angry.
0:56:08I agree, I agree, no no no, I didn't think I was making it go faster by any stretch of the imagination,
0:56:12what I was doing was getting redundant copies of the things that I really and truly
0:56:16couldn't miss. Right, but if you wanted to do that, do the small copies first, like do things
0:56:20and, you know, do the small emergency copies first, one at a time, serially.
0:56:24Because there's only, there's only one, I'm not going to say there's only one disk head, because there's multiple disk heads,
0:56:28but there's only one set of disk heads on a single arm, and that arm can only
0:56:32follow one instruction at once, go left, go right, go up, you know, just, let it
0:56:36do its, you know. No, I hear you, I hear you, you're absolutely right. And so,
0:56:40sitting here now, we have moved to 2.14, no no,
0:56:442.15% in the time we've been talking, and so I figure in about a day
0:56:48it will either finish or one of the drives will die, or
0:56:52it'll finish and start over again, and if it starts over for a third time, then I think that's
0:56:56probably a sign. But yeah, I don't know what the final
0:57:00answer is. I don't see my life not having
0:57:04a Synology in it, but in contrast to the last we spoke about this,
0:57:08I do see a world where maybe what I end up doing is I get a
0:57:12smaller Synology, like I said a moment ago, and putting larger drives
0:57:16in that, having that local, and then sticking this one in, like
0:57:20Mom and Dad's garage, and syncing to that and calling it a day.
0:57:24And then additionally, for now anyway, I'm going to
0:57:28plan on doing some sort of replication onto that 12 terabyte
0:57:32once everything is squared away. Once everything's squared away, and I know I'm good
0:57:36again, then I'll do some sort of replication onto the 12 terabyte, that will be backed up to Backblaze
0:57:40the way it's supposed to be backed up, because it'll be physically connected to the Mac Mini, Backblaze
0:57:44has no issue with that. Everything will be right as rain once it spends all
0:57:48the time uploading to Backblaze, which is taking quite a long time in and of itself, but be that as it
0:57:52may, then in theory I will be in a much better place. And if
0:57:56Backblaze had already had everything, which is not Backblaze's fault, it's my
0:58:00fault, if Backblaze already had everything, I could have asked them for like,
0:58:04I don't remember how big the drives are they offer, but I could have asked them for one or two physical hard drives
0:58:08to be next day aired to me, and then I could nuke everything, put these hard drives,
0:58:12attach them to some things, maybe the Synology itself, copied all that data
0:58:16back, like John was saying a moment ago, and then sent these hard drives back to Backblaze
0:58:20and I have literally spent no money other than the normal Backblaze fees.
0:58:24So, I'm feeling better about it now because of that
0:58:2812 terabyte drive, which by the way, I don't think I said, I did disconnect and put it
0:58:32in the garage, so it is literally across the house on a different floor, so
0:58:36God forbid the house goes up in flames in the next 48 hours, it is as far away
0:58:40from the Synology, so hopefully one of them will survive if
0:58:44the house literally goes up in flames, and it is not connected to any sort of power source
0:58:48or anything like that, but it has been, it has been a rollercoaster.
0:58:52It has been a rollercoaster, mostly of my own doing, and I appreciate you letting me
0:58:56get this off my chest, and I hope that this time tomorrow everything will
0:59:00be restored and everything will be in a good place again, and then
0:59:04I'll have to weigh whether or not I really want to replace drive 8 preemptively, or just
0:59:08let it sit until it explodes. You keep forgetting about or not mentioning the
0:59:12thing that you should do that it seems like you're not going to do because you've already forgotten about it, which is what Margot said,
0:59:16make a second copy of the 12 terabytes. Yeah, well, I should. And you can do it by buying
0:59:20a drive, like you can do it in such a way that later you can reuse that drive into one of your Synologies
0:59:24or whatever, and speaking of new Synologies, you should look at, I was looking at
0:59:28in my many, in my fantasy Synology shopping, there's more
0:59:32variants in the CPUs these days, and it's not linear with the
0:59:36sort of price of the device, like you can get a big one with a wimpy
0:59:40CPU, and you can get a small one with a powerful CPU, so shop around and see
0:59:44if you can get one that might actually be able to do all your transcoding for you, especially
0:59:48most of your stuff is H.264, not H.265, it's possible you could get a
0:59:524 or 5 bay Synology that can do all your transcoding without breaking a sweat, so check that out
0:59:56if you're shopping. Yeah, just looking, like the price differences for the bay count
1:00:00actually aren't that big, like, if you, like, a 5 bay
1:00:04is $650, and an 8 bay is $931
1:00:08right now on Amazon, like, of the same line with all the premium processor, the plus line
1:00:12and everything, so like, you actually, given, if you're gonna, you know
1:00:16spend another 7 years with this, whatever you might get next, it might
1:00:20be worth getting something a little bit bigger, but, uh, but you probably don't need
1:00:24to. Also, the new Synologies let you connect expansion
1:00:28bays to them, so you can buy one good Synology with a good CPU and 4
1:00:32bays or something, and if you fill those 4 bays, you can buy another dumb
1:00:36Synology bay thing, and use it to expand
1:00:40your existing Synology, which I don't think that feature existed in 2013. It did,
1:00:44our support it. Really? I didn't know that. I'm almost sure they do, yeah. I think, I'm
1:00:48pretty darn sure that Marco's right. I love how on their site, uh,
1:00:52they have, you know, if you go over, go to products, and then, you know, they have all these, like, sections
1:00:56'cause they're all enterprise-y, and you have to choose whether you are a personal and home user,
1:01:00or an IT enthusiast, or, you know, small,
1:01:04small and mid-sized business enterprise, but is Casey an IT enthusiast?
1:01:08I think so. I sure hope so. The funny thing is, the models it shows you seem to be
1:01:12identical between whether you pick IT enthusiast... Well, that's the thing. If you
1:01:16pick the business one, they're not gonna give you the ones with the good
1:01:20CPUs. Like, it could be that the enterprise-y ones are the wimpier CPUs, that they just expect
1:01:24to be doling out files to a large number of people, so look at the actual CPU options. They vary
1:01:28a lot. That's fair. I don't know. So that's my,
1:01:32that's my tale of woe, and what I, what I really want to do is
1:01:36throw a whole pile of money at this problem, but I don't want to throw a whole pile of money at this
1:01:40problem, so I'm trying to do this on the cheap, and that's half the issue right there. But
1:01:44I will update you next week as to where all this lands, but it's been...
1:01:48After you, after you copy the 12 terabyte to another 12 terabyte, right? Yes.
1:01:52Please do that. He keeps not mentioning it. I know. I don't know why.
1:01:56Like, yeah, 'cause even if you can get back to Best Buy and just get yourself
1:02:00another, you know, 12 terabyte external from the parking lot, you can use that
1:02:04even if you don't want an external, just take, take the drive out of it. Like, I've done that before, where like you buy
1:02:08a drive versus an external. Later on, you want it to be internal. In fact, I think two
1:02:12of the drives in my Synology, I think, are this exactly, where like, I just
1:02:16took, I just took a screwdriver, took apart the enclosure that they came with from Seagate or whoever,
1:02:20and inside of those enclosures is a regular drive with a little USB controller board. You just unplug it
1:02:24from that. It's a regular SATA drive, and you plug it into whatever you need. Yeah.
1:02:28Yeah, I probably should. That's why I always buy my, my mechanisms
1:02:32specifically, and I buy all these crappy external cases, just so I get to pick the mechanism.
1:02:36Because if you buy some, I mean, obviously what you bought from Best Buy is like, you gotta do what you gotta do. You need to
1:02:40get drive ASAP, you know, situation, I get it, right? But with more
1:02:44time to spare, it's always better to pick the mechanism yourself. Even if you pick wrong,
1:02:48at least you can, you know, skew towards drives
1:02:52that you think are, have a better reliability reputation.
1:02:56You can look at the back play stats. I do that all the time. They, they have the hard drive stats of like
1:03:00their failure rates. The problem is, there are two problems with the back play stats. One, most
1:03:04people's use case is not like back plays' use case. Yours might be
1:03:08similar, kind of, but not really. Like, I don't, back plays drives
1:03:12are running in a different environment than, you know, they're running in a data center, they're running in these
1:03:16big racks and these machines right next to each other. They don't have a giant box fan pointing
1:03:20at them. Yeah, they might have better cooling than you, they might have
1:03:24worse, they might have more activity or less, but it's, it's not a typical
1:03:28use case of like, oh, occasionally I watch Plex off of my thing. Like, that's,
1:03:32that's calmer than back plays. And the second thing is, back plays gives you the exact
1:03:36model numbers. By the time they do their readouts of the exact model, sometimes it's hard
1:03:40to find those exact models. So it's like, wow, or you can find them, but they
1:03:44only have like, you know, three months worth of data. And if you look
1:03:48at the past reports, you can say three months is not a long enough time to know if this thing is going to die
1:03:52in the first year. When you see one, it's like, we've had these drives for two years and their reliability
1:03:56has been excellent. You can no longer buy those drives. So it's not a slam
1:04:00dunk, but at least it can give you a feel for
1:04:04manufacturers, model lines. It's so hard, because like, it could be, you know, wow,
1:04:08it looks like, you know, Western Digital Reds are doing really well. But it turns out the 14 terabyte
1:04:12Western Digital Red is a reliability disaster and you get it based on the reputation
1:04:16of the smaller ones and you make a terrible mistake. So it's, it's not an
1:04:20exact science, but I don't know, I always feel better being able to pick the magnets, especially since
1:04:24if you buy one off the shelf, it could have something in there that isn't even a drive,
1:04:28you know, a quote unquote consumer drive. Like, it's a drive that's not
1:04:32even meant for NAS type situations where it's expected to be
1:04:36on all the time and, you know, maybe it has a little bit, a little bit more
1:04:40redundancy. Like, that's in theory when you buy a spinning
1:04:44disk that's intended for use in a data center or intended for use in
1:04:48a NAS. A, you're paying some stupid premium for their profit margins.
1:04:52Just live with that. But B, in theory there is something physically different about the drive
1:04:56that makes it slightly more suitable to this purpose.
1:05:00Yeah, I gotta rethink my whole world, unfortunately. But one step at a time.
1:05:04We'll see how this restore goes.
1:05:08We are sponsored this week by Eero, the WiFi your home
1:05:12deserves. Now, we've all had bad WiFi experiences.
1:05:16In our houses, in our relatives and parents and friends' houses, whatever it might be.
1:05:20So I told you last time about how I had a vacation rental with terrible WiFi and I
1:05:24got an Eero and I fixed their WiFi and took it with me when I left. It was wonderful.
1:05:28And where that Eero went after that was to my in-laws house.
1:05:32They had, like, you know, areas where their WiFi wouldn't reach. They had bad rooms
1:05:36including the room that we would stay in, which hit me personally.
1:05:40When we stayed with them, they would just have slow WiFi there. And I thought, I gotta fix this problem.
1:05:44At least for us, but also for them. So we
1:05:48moved that Eero there. And their problem was fixed
1:05:52immediately. They have amazing WiFi now, thanks to
1:05:56Eero. There's also an all new Eero now, so not only
1:06:00can you fix your WiFi problems. As soon as tomorrow, which I'll get to in a moment.
1:06:04But now there's an all new Eero starting at just $99. So you
1:06:08too can have your entire home blanketed in fast, reliable WiFi.
1:06:12Get rid of your slow rooms, your dead spots. You get a strong signal
1:06:16wherever you need it. It's super easy to set up an Eero system. It sets up in just minutes.
1:06:20You plug it right into your modem or your router, and you manage it from there.
1:06:24Super easy to use app. This is the easiest router setup I've ever seen.
1:06:28Single router or multi router setup. Easiest I've ever seen by far with their
1:06:32awesome app. So there's no more WiFi problems. No more, you know, video buffering
1:06:36in certain rooms or can't load Instagram, whatever it is. It's just fixed.
1:06:40Let Eero fix your WiFi problems too. And you can get it
1:06:44fixed as soon as tomorrow. Go to Eero.com/ATP
1:06:48That's Eero.com/ATP and use offer code
1:06:52ATP at checkout to get free overnight shipping with your order.
1:06:56That's Eero.com/ATP. Code ATP to get your Eero delivered with
1:07:00free overnight shipping. You must use this URL to receive this offer.
1:07:04Eero.com/ATP. Code ATP. Thank you so much to Eero
1:07:08for fixing our WiFi and sponsoring our show.
1:07:12So there was some news today as we
1:07:16record, and it surprised me a lot.
1:07:20Apple has bought Dark Sky. And if you're not familiar, Dark Sky
1:07:24I don't know, it started something like five to ten years ago
1:07:28and at the time it was its own weather app
1:07:32and then they shortly thereafter provided an API. And
1:07:36especially early on, it was eerie how accurate it was.
1:07:40It's still very accurate, but I don't personally feel like it's quite as accurate as it used to be.
1:07:44But it would eerily predict when it was going to start to rain, where you were
1:07:48sitting. So it would say something like rain in seven minutes. And
1:07:52five, six years ago whenever it came out, I would look at my watch and it was, I don't know,
1:07:56ten past ten, and sure enough, seven minutes later, it started raining.
1:08:00It was bananas. It was magic as far as I'm concerned.
1:08:04And over time, I stopped using the Dark Sky app, I personally am a
1:08:08Carrot Weather user, but Carrot Weather, for Americans anyway, still uses
1:08:12Dark Sky for most if not all of its weather information.
1:08:16And pretty much all of my favorite weather apps over the years, including
1:08:20Check the Weather by Underscore, and I forget what else I've used in the past, but almost all
1:08:24of them use Dark Sky in part if not in whole, for Americans
1:08:28anyway. And Apple has bought them. And they, I believe they
1:08:32either killed the Android app or have said it's not going to get updates. Do you guys remember
1:08:36the 20 dollars? I think it's removed from the store or something like that. But I think it still works.
1:08:40Okay, and then on top of that, they have announced that the
1:08:44iOS app is still available, no, it's still available and it's still purchased,
1:08:48which is a little bit weird, but the API is going away next year.
1:08:52And I think that's basically the summary of what's happened. Marco, any thoughts
1:08:56on this as the person who I think is most close to
1:09:00developing an app that would use, obviously you don't use Dark Sky, but
1:09:04you're the kind of person that would develop this kind of app, so how do you feel
1:09:08about this? Well, I do use Dark Sky. That's the thing. I have their app installed,
1:09:12I also use, my usual weather app is Weatherline, and
1:09:16Weatherline uses Dark Sky data. It used to use it, I think, for everything.
1:09:20When they redid the app, they now use a hybrid that
1:09:24includes Dark Sky for certain things, but uses other data providers for other things.
1:09:28This, I really don't like this news, honestly. And we'll see what happens.
1:09:32Maybe over time, I will end up liking whatever Apple has in mind
1:09:36to do with this here. But I don't love this the way it is now,
1:09:40because the Dark Sky API has been so important. It really is,
1:09:44it has powered so many
1:09:48indie weather apps that I have really liked it. As you mentioned, Carrot,
1:09:52Weatherline, and including Dark Sky's own app.
1:09:56And it's been really great. There have been times in the last,
1:10:00whatever it's been, ten years, there have been times where it is less accurate.
1:10:04But, you know, it's an algorithmic based thing. It's reliant
1:10:08on certain data, and I'm sure they've tweaked the algorithms over time. So, you know,
1:10:12some people try it and it doesn't work out for them, or it works for a while, then it's bad for a while,
1:10:16then it's good again. It's always been good for me. And I use it all the time, because I want
1:10:20to know, like, every day, I'm going to go outside, I'm going to walk my dog for the next half hour, is it going to rain?
1:10:24The whole concept of the app is a great, it's a great app, and
1:10:28it has really been important to me, you know, both as a user
1:10:32of Dark Sky directly, and as a user and fan of third party,
1:10:36especially indie, weather apps. It's been great. I even,
1:10:40I forget why, but I even had a couple of email exchanges with the
1:10:44founders forever ago, and I remember them being very nice. Like, it's just, it's a
1:10:48wonderful app, run by wonderful people. It's also been very
1:10:52successful. It has been the top paid app, or within the top
1:10:56few paid apps on the app store for years. So,
1:11:00you know, they've, whatever they're selling to Apple for, it's probably not
1:11:04that their sales sucked, like, it's not, it's probably not that they needed the money,
1:11:08and they also charge for the API, and they make good money from that, I assume.
1:11:12So, the reason for them going to Apple, I'm sure they're going to do great things,
1:11:16I'm sure, you know, Apple wants them to, you know, work on the weather team or whatever,
1:11:20that's great, the Apple weather app could use some love, but
1:11:24I would feel a lot better about it if they were still going to run the API for other
1:11:28apps to use as well. And maybe there's a way that they're going to do this. Maybe Apple
1:11:32will offer, like, a weather kit API for iOS apps, you know,
1:11:36screw Android, but for iOS apps to use this, maybe.
1:11:40That would be nice, but I don't know that they will. So,
1:11:44you know, if the idea is just for Apple to, you know, have their own weather
1:11:48source, and I guess not have, maybe not have to pay, like, whoever it is, like,
1:11:52the Weather Channel or Yahoo or wherever they're going to source now, if, whatever Apple's
1:11:56value is getting out of this, if it's just to, like, save on a license
1:12:00fee to some other data provider and to have, like, local rain forecasts
1:12:04in the iOS weather app and screw everybody else and, you know,
1:12:08no one else is going to have this data anymore, that's no good. But if there's a way
1:12:12that third-party apps can continue to be good and have access to
1:12:16this kind of data, I hope they can manage to do that.
1:12:20But for now, I don't feel good about this, because that's an uncertain future,
1:12:24and I really do like all of the
1:12:28benefits that the Dark Sky API has provided. It also, like, they
1:12:32also have a website. For a while it was forecast.io, but now I think
1:12:36it's just dark-sky.something. I don't know, it's always auto-completing for me. And what's
1:12:40great about the website, not only can you see weather anywhere, of course, which is actually surprisingly nice
1:12:44on something like a Mac where there really aren't a lot of good weather apps on the Mac, but
1:12:48also you can look up historical weather data, which is really cool.
1:12:52All that for free on their website. Like, it's just been a wonderful service and a wonderful
1:12:56app and a wonderful API that has been such a key part of so many people's
1:13:00daily usage. So to have Apple take it over and
1:13:04give it a shutdown date is concerning. I always wonder how Apple feels about
1:13:08these situations where
1:13:12there's, like, an API or some other thing
1:13:16that third-party apps build on. From
1:13:20our perspective as users and even as developers, these all seem like
1:13:24clear winds. You just mentioned, like, oh, this could go into some kind of weather kit thing or whatever, but
1:13:28think about cases where Apple has done this either by accident or on purpose.
1:13:32There's tons of calendar apps for the Mac because Apple provides a bunch
1:13:36of APIs that allow those apps to access the Apple calendar.
1:13:40Also, those calendar apps access other calendars, like Google Calendar
1:13:44and so on and so forth. Apple, you know, was just a super plan for Apple
1:13:48to say, we want to have tons of awesome calendar apps on iOS and the Mac
1:13:52for that matter, so let's do this open calendar API. Clearly,
1:13:56they made whatever the calendar APIs are called with that intention, same thing with the
1:14:00address book, contacts, all that stuff. They made those APIs with that intention, but
1:14:04at various points they could have made different decisions. They could have said, oh, great, well, you can
1:14:08make an iOS app that accesses the shared calendar, but your app
1:14:12can't access Google Calendar or something like that. That totally sounds like a thing that Apple would entertain.
1:14:16They do all sorts of stuff like that related to an app purchase and other scenarios where they're trying
1:14:20to sort of steer the market through restrictions because they control the whole
1:14:24store. API stuff, CloudKit and all that stuff.
1:14:28They make a lot of essentially faceless services. The
1:14:32only reason they exist, aside from Apple using them themselves, is, hey, if you want
1:14:36to make an app for iOS that has an online component but you're not a
1:14:40server-side developer, we have a bunch of libraries that talk to our
1:14:44servers with a fairly attractive business model of
1:14:48you don't have to worry about the service being up or anything like that. We will store your data.
1:14:52They have this tiered thing where if you have a small number of users, it's not a big deal.
1:14:56The fees go up as you get more. Those services
1:15:00exist so that people can write better, cooler apps
1:15:04with less, not with less effort, but with less people power, with
1:15:08less expertise. It makes their platform better
1:15:12for users because they get better apps. It makes it more attractive for developers because smaller
1:15:16developer teams can do better things. So you look at this scenario.
1:15:20There's tons of cool weather apps for iOS and
1:15:24less so for the Mac, as usual. But anyway, why are there all these weather apps?
1:15:28Is every one of these weather apps backed by some company that owns weather
1:15:32stations and weather data? No. Sometimes they use commercial APIs that exist out in the world
1:15:36and the Dark Sky API, I'm assuming, is just a wrapper around a bunch of commercial APIs because I
1:15:40assume the Dark Sky people don't run a bunch of weather stations. So it's just wrappers on top of wrappers
1:15:44are just fine. But if you want to make a weather app and your idea is it's going to
1:15:48be named after a vegetable, it's going to have snarky messages about the weather,
1:15:52you can implement that without worrying so much about
1:15:56"Yeah, but how would I get the weather? Do I have to contract with AccuWeather?
1:16:00How do I do all this stuff?" And if the Dark Sky API is
1:16:04a path of least resistance to do that, then that makes
1:16:08more weather apps available. So here Apple is faced with a choice. We want Dark Sky
1:16:12because basically, I'm assuming, they want to make their weather app better because their weather app has been falling
1:16:16behind. Great, fine, buy them. It's great for the developers because they presumably get a big pay
1:16:20day. As has been discussed any time anyone gets acquired, a lot of the motivation
1:16:24for people going to Apple is often that their work can
1:16:28be put in front of more people because Apple has a huge audience and as
1:16:32successful as Dark Sky is, if you are the default weather app on the iPhone,
1:16:36that is a bigger impact. So if you're looking for what that next step is, boy, I'd like to be
1:16:40in front of hundreds of millions of people instead of just single digit millions, right?
1:16:44But anyway, it's all good for them and it's good for Apple to get a better weather
1:16:48app, but I really hope that Apple looks at what
1:16:52they've acquired and the fact that it is an API and says, "Regardless of whether
1:16:56we shut down this API or not, we should do the same thing that we have done, again, either
1:17:00accidentally or on purpose, with things like CloudKit
1:17:04and like the calendar and contact APIs."
1:17:08It doesn't always have to be server-side, but enabling
1:17:12there to be more competitors for their own apps.
1:17:16And that's why I started this by saying I wonder what Apple thinks about this because
1:17:20if I was inside Apple, I'd be like, "Don't you see, in all the places
1:17:24where we have the most richness in the App Store, it's because we did things
1:17:28like this?" On the other hand, I can imagine someone saying, "How did our weather
1:17:32app fall behind?" Well, it fell behind because we allow all these other apps to just have
1:17:36this ecosystem of sharing this API and we need to nip that in the bud and now we'll have the best
1:17:40weather app again, which is short-sighted and stupid. And I don't think they think that,
1:17:44but I also don't see, I never see them come out and sort of
1:17:48other than in obscure technical WWC sessions, tout the idea
1:17:52that by making services and APIs
1:17:56available, we are making a better software ecosystem
1:18:00on our platform. I know they have a big
1:18:04services picture. When we say services with the capital S as in these earnings
1:18:08that we've been talking about for the past several years, it's like, "Oh, I'll pay Apple
1:18:12so I can watch TV shows. I'll pay Apple so I can store my photos for you. I'll pay Apple
1:18:16for more disk space. I'll pay Apple for a music service." It's all about just paying Apple,
1:18:20consumers paying Apple. But this scenario where
1:18:24there are services but the people who pay for them and use them are developers,
1:18:28like one degree separated from users, doesn't seem like something that Apple
1:18:32it's not that they don't like it, they do it enough times, but
1:18:36it always strikes me as a little bit too accidental, huh?
1:18:40That they just found themselves in a situation like, "Why is it true for Calendar but
1:18:44not true for something else?" And the podcasts are a good example. They have this podcast directory
1:18:48which accidentally or on purpose powered this entire ecosystem of podcast apps,
1:18:52but was that the plan from the beginning or did it just kind of happen and they were smart enough not to screw it
1:18:56up? I think they should embrace this model because I think this model has been proven to work.
1:19:00It gives Apple control. They have control over the CloudKit APIs. They have control
1:19:04over all the other things they offer. They have control over their libraries.
1:19:08They could make a weather kit and they could have essentially Dark Sky
1:19:12API v2 powered by Apple's access through weather kit
1:19:16and that would enable someone else's great idea of how to make a weather app
1:19:20to be done or someone's bad idea for a weather app. Either way, it will
1:19:24attract developers to the platform. It will make it a more
1:19:28attractive place for customers to shop because they have lots of cool weather apps made by
1:19:32small teams with a good idea who didn't want to deal with the weather stuff. So my fingers are
1:19:36crossed for that eventuality, but if they just take the
1:19:40crew, make their weather app better and stop the API after
1:19:44a year, it's not the end of the world, but it's not great.
1:19:48That's where we get into Marco being pessimistic about this. I'm choosing to think
1:19:52that they're going to do the right thing and if not continue that API
1:19:56then have a similar API fronted by a library or something so that
1:20:00we can continue to have tons of weird and ridiculous weather apps on
1:20:04iOS. Yeah, I totally think and hope that that's the case and
1:20:08it vaguely reminds me of when Google Maps was the
1:20:12only mapping software on the iPhone and if you wanted to put a map in your
1:20:16app, you had to jump through some hoops. I don't remember exactly what they were, but you had to jump
1:20:20through some hoops to establish a relationship with Google and
1:20:24I think you might have even had to get a Google SDK in order to do it, which kind of makes sense.
1:20:28And then Apple starts providing its own maps and granted they
1:20:32were garbage for a long time, but now I don't think they are for most people. And
1:20:36adding a map into your app is super simple because the API is right
1:20:40there. It's part of the system and I feel like this feels as
1:20:44though it's an analogous sort of event and this is exactly what you're saying John, that maybe
1:20:48there will be some sort of weather API. Perhaps it could be even part of the MapKit API,
1:20:52who knows. But some sort of weather API that will allow you to
1:20:56get weather information for all sorts of different things. I mean if you think about it, weather
1:21:00information, I'm trying to think off the top of my head, on my phone shows up in Fantastical,
1:21:04it shows up in the Maps app, it shows up in my weather apps,
1:21:08there are more places than you would expect. Oh, and day one it shows up, there's more
1:21:12places than you would expect that have or show weather
1:21:16data and to be able to... Is there already a weather API? I don't even know, maybe there's already a simple
1:21:20weather API. There isn't. Yeah, I don't think there isn't. So I don't know, I don't know where
1:21:24this goes. It certainly could be another
1:21:28form of lock-in, you know, if the only way you get a really good weather port is if you have
1:21:32an iPhone. Like I hope that's not the case, but it would be an interesting
1:21:36and very unusual form of lock-in. Well yeah, I really like that Android phone,
1:21:40but I can't get my weather app that I rely on because, I don't know, I'm a dog walker or something like that,
1:21:44like a professional dog walker, or I don't know, I'm a landscaper, you know, and it's
1:21:48incredibly important, I know exactly. I don't think that's that bad of a situation,
1:21:52that's basically my optimistic scenario, is that it is, you know,
1:21:56that Apple uses it as a platform differentiator. Again, Apple, as far as I'm aware,
1:22:00you never know, doesn't own a million weather stations around the world, like some, the weather
1:22:04data comes from elsewhere. This is all just about mediating access to that and providing
1:22:08a nice interface to it, and if it's a differentiating factor
1:22:12to say, you know, anyone can get like the weather from just some source or whatever, but Dark Sky
1:22:16is a big thing, as Marco said, is like, it was super, you know, whatever they call it, hyper-local,
1:22:20like you can get the weather right where you're standing right now,
1:22:24which was a differentiating feature and the reason it's a popular application. So,
1:22:28having a weather API is something anybody can do. Having a really good weather
1:22:32API is harder, and that's, if Apple wants to use that as a differentiating feature for the iPhone,
1:22:36oh, the iPhone has, you know, they bought Dark Sky, so they have the current best
1:22:40weather API, whether it's because of the way Dark Sky processes the data that it's
1:22:44got or the data sources it has or whatever, but it's not like they're locking anybody out
1:22:48because, you know, the weather is, information is
1:22:52publicly available, as in we can all see what the weather is, and anybody can build weather stations
1:22:56or anyone can pay someone who has weather stations, so like, I don't feel that there's
1:23:00any evil in them, you know, confining the best weather API
1:23:04and the easiest to use weather API to their platform,
1:23:08that's exactly what they should be doing. You know, Android and Google
1:23:12can do exactly the same thing. You know, that's how they should be competing with each other.
1:23:16Yeah, and so the other angle to this is
1:23:20that weather apps are famously
1:23:24frequently bad for privacy. Not the ones we've mentioned, not like the nice indie ones,
1:23:28but there's a lot of weather apps out there. You know, weather apps, by their nature,
1:23:32need your location for most convenient features.
1:23:36You know, you can always not give some of your location and maybe just type in your location or have it just
1:23:40always display that location or have it to change it manually, but for the most part, most people
1:23:44give most weather apps their location access so they can show the weather wherever they happen to be.
1:23:48Because weather apps have location access, they can sell that information.
1:23:52Location data is very valuable in the ad tracking creepiness business.
1:23:56Not only do many weather apps, you know, have a lot of
1:24:00not only do many weather apps just sell this data outright,
1:24:04if you have a weather app that gets location access, you will be approached, people will email you
1:24:08from these scam companies offering you money to
1:24:12integrate their SDK. And this might be more money than you might be making from your
1:24:16app sales if you're not doing that well in the app store, which is very often the
1:24:20case for a lot of apps, right? They will offer you money to build in their tracking
1:24:24SDK because your app has location access, right, by the user.
1:24:28So they'll give you money to put their SDK in that will measure your
1:24:32location that the app already has access to and tie it to ad profiles
1:24:36and then they can sell that and make more money from you. So that's a whole thing that's
1:24:40going on behind the scenes now, and that might be informing this decision by Apple as well.
1:24:44There's this whole seedy underbelly of weather apps that because
1:24:48they by their nature usually get your location permission,
1:24:52they have this valuable data and many of them are being really creepy with it.
1:24:56Again, not the ones we've mentioned because we wouldn't use an app that did that, right?
1:25:00The ones we've mentioned, you know, Carrot, Weatherline, like these are all good apps made by good people. They don't do this kind of crap.
1:25:04But many apps do. So it's a very big deal
1:25:08that weather apps are a notorious privacy hole
1:25:12for your location data. And Apple's very sensitive to that. So maybe
1:25:16this is part of some kind of play to crack down on that.
1:25:20I haven't quite thought about how they could do that effectively.
1:25:24One option would be they no longer allow weather apps to
1:25:28have your location and they just provide weather data for them.
1:25:32But that's a really aggressive move. And that would have a lot of other
1:25:36problems too. Like, for instance, no one
1:25:40weather data source is good for the entire world. There's different
1:25:44data sources in different parts of the world and certain ones are better than others for different
1:25:48regions and everything. So I don't think they would or should do something
1:25:52as drastic as prohibit any other weather API from having location
1:25:56access. That would be weird. So I'm just trying to think through, like, is there
1:26:00some kind of privacy angle on this that would lead to an actual
1:26:04possible and likely and enforceable outcome
1:26:08that they could maybe crack down on this incredibly seedy
1:26:12underbelly of privacy selling weather apps by somehow
1:26:16buying Dark Sky. Again, I haven't quite figured out how that can work yet, but maybe there's something there.
1:26:20I was thinking about all the other platforms that Apple has where weather data is
1:26:24useful. Apple has weather data like this weather. There's a weather widget on your watch,
1:26:28there's weather thing in the notifications, whatever the hell that thing's called on
1:26:32the Mac on the right side of the screen, the Today View, that has a weather thing in it, I believe.
1:26:36And obviously on our phones. And buying Dark Sky will give them
1:26:40better weather presumably, gives them people who are experienced making
1:26:44more advanced weather apps than Apple's own and plus all the hyper local stuff with the local
1:26:48radar to say it's going to rain in five minutes. But, you know,
1:26:52having your watch nudge you that it's going to be raining on your head in five
1:26:56minutes is pretty cool. Dark Sky app, I already have a watch app that does
1:27:00that. But, you know, AR glasses, you can look up in the sky and see which direction
1:27:04the rain clouds are coming from. There's all sorts of scenarios where even better weather
1:27:08than they currently have is an attractive thing to Apple. So this
1:27:12purchase doesn't surprise me in the slightest. I'm just really hoping
1:27:16that the path they take with this purchase is not simply to give Apple's
1:27:20platforms better weather stuff, but to
1:27:24make an API and a service that everybody can use, even if they
1:27:28discontinue the current one, make a new API, like Margot was saying, perhaps with different privacy
1:27:32trade-offs. Hopefully not with too many different restrictions, like
1:27:36what we don't want to see is like, oh, Apple has a cool new weather kit that's very
1:27:40advanced and has all the features of Dark Sky plus an API and you can use it in the right theory of the reason, and
1:27:44every other weather service is forbidden on the platform. I don't think they would do that, but
1:27:48that's the worst case scenario. That's like the in-app purchase scenario where
1:27:52if you want to sell things through your applications
1:27:56for this huge swath of stuff where you have to go through Apple's thing and give them 30%
1:28:00and stuff. So that's at one extreme. And the other extreme is it's a free-for-all
1:28:04Apple is going to compete in that free-for-all by having a really good weather service, which they can
1:28:08do now that they bought Dark Sky and I feel like they should just be happy to compete with that, like they do with CloudKit.
1:28:12You can use CloudKit or whatever, you know, core data
1:28:16I don't like cloud core data, maybe you don't want to use that, but anyway, Apple has a bunch of services
1:28:20that you can use in your iOS app, or not. By the way, isn't it a shame that they already
1:28:24use the name CloudKit? Like wouldn't that have been perfect for this?
1:28:28Yeah, and they don't know if they wanted to use Dark for Dark Sky, but
1:28:32like the only scenario where Apple has most recently
1:28:36forced its hand is in a situation like Marco was alluding to, which is where there's a privacy
1:28:40thing. So Apple has a thing that you can use for your sign-in, sign-in with Apple,
1:28:44and there they do mandate that if you have an app that allows you to sign in with Facebook
1:28:48with Google, with whatever, you have to offer sign-in with Apple. And the reason they're doing
1:28:52that is privacy related. Part of it is yes, okay we want everyone
1:28:56to use Apple's things, but honestly Apple has you already. The whole point is, you're on your iPhone
1:29:00you have an iCloud, you know, Apple's got your Apple ID. Like they're all set on that front.
1:29:04Sign-in with Apple is, I guess, you know, partially
1:29:08so they can get their hooks into you one more way, but the reason I think they're
1:29:12mandating it is not just for adoption, but also because
1:29:16they have a privacy angle, as evidenced by the way the API looks, where you can use it without
1:29:20even giving people your email, which is unheard of on the other services.
1:29:24They want everything from you, right? So if there's
1:29:28a privacy angle on this, I can see Apple being a little bit more forceful
1:29:32perhaps not mandating the use of their API, but further restricting
1:29:36the other APIs to compete on a level privacy playing field with Apple's.
1:29:40They haven't done that with the sign-in stuff. You sign in with Facebook, Facebook gets all
1:29:44your everything. You sign in with Google, Google gets all your everything. You sign in with Apple, Apple
1:29:48is the one voluntarily saying, "Hey, when you use our API, you
1:29:52have to actually ask people for the email address. You don't get it by default.
1:29:56We'll make up a random email for you." You know, all that stuff that Apple is doing
1:30:00is tying their own hands behind their back in terms of the amount of data they get.
1:30:04Apple doesn't want the data. They want to have a more private
1:30:08API, but they didn't require everybody else to do the same thing, at least yet anyway.
1:30:12The weather API, we'll see. This is an interesting thing to watch, not so much because we're
1:30:16interested in weather APIs and the ecosystem weather apps on the iPhone, but because it shows where Apple's head
1:30:20is at currently in terms of the range of things that they can do
1:30:24both policy-wise and tech-wise for their platforms.
1:30:28We are sponsored this week by Postmates.
1:30:32From an early morning breakfast to a bottle of wine for movie night, sometimes you just need
1:30:36what you need, and you need it delivered fast. That is where Postmates
1:30:40comes in. Postmates delivers food from every restaurant you can think of
1:30:44right to your door. And they don't just deliver takeout. You can also get deliveries from
1:30:48grocery stores, convenience stores, clothing stores even, you name it.
1:30:52This is especially nice to have right now. So, no more trips to the
1:30:56store, no more late night fast food runs. Just download Postmates
1:31:00on iOS or Android, find your favorites, and you can get anything you want
1:31:04delivered within the hour. For a limited time, Postmates is giving
1:31:08our listeners $100 of free delivery credit for your first
1:31:12seven days. So, that's right. If you are new to Postmates, you can get
1:31:16$100 of free delivery credit with no minimum purchase.
1:31:20So to start your free deliveries, download the app and use code ATP.
1:31:24That's code ATP for $100 of free delivery
1:31:28credit in your first seven days. So, once again, to start your free deliveries,
1:31:32download the Postmates app and use code ATP for
1:31:36$100 of free delivery credit with no minimum purchase for
1:31:40your first seven days when you download the Postmates app using code ATP.
1:31:44Anything you need, anytime you need it, Postmate it.
1:31:48Thanks to Postmates for sponsoring our show.
1:31:52Just today,
1:31:56apparently it's now possible, and I'm very unclear on where
1:32:00the dividing lines are, but it is possible to rent
1:32:04or buy or do something with video through
1:32:08Amazon Prime Video's app, but not using in-app
1:32:12purchase? Did one of you pay closer attention to this? Because I am deeply confused as to what the situation is.
1:32:16Well, why don't you read the very, very clear Apple quote that's in the show notes.
1:32:20Well, I would be happy to, John. Apple has established
1:32:24program for premium subscription video entertainment providers to offer a variety of customer benefits,
1:32:28including integration with the Apple TV app, AirPlay 2 support, TVOS apps, Universal Search,
1:32:32Siri support, and, where applicable, single or zero sign-in.
1:32:36On qualifying premium video entertainment apps such as Prime Video, Altice One, and
1:32:40Canal Plus, customers have the option to buy or rent movies and TV shows using the payment method
1:32:44tied to their existing video subscription.
1:32:48Wow, look what they snuck in there. That's a lot of words. What does that actually mean?
1:32:52So this is incredible. So forever, Amazon
1:32:56and Apple have been battling over in-app purchase rules.
1:33:00Apple has always, since the introduction of in-app purchase on iOS,
1:33:04they've always required that apps are not allowed
1:33:08to collect payment information and have their own payment processing
1:33:12to buy digital goods in their apps
1:33:16without going through in-app purchase. Like, Amazon and Apple
1:33:20has not been allowed to have something like the Kindle app,
1:33:24where I believe we first saw this, and later on when they launched Prime Video
1:33:28that Amazon will sell you videos to rent or buy
1:33:32or whatever, and Apple has not allowed them to do
1:33:36Amazon's own purchase system inside the app in their iOS app.
1:33:40They would require them to use Apple's in-app purchase system, but Apple's in-app purchase system
1:33:44charges 30% for most things, and they would require
1:33:48Amazon charges 30% for most things, and had other various
1:33:52limitations that Amazon didn't want to comply with.
1:33:56There's always been this battle of heads, you know, budding, and they've been going back and forth
1:34:00kind of quietly in the background for years, and Amazon and Apple
1:34:04had this, you know, pretty tense relationship for a while.
1:34:08And last year, I think, or whenever it was, they basically
1:34:12announced that they had come to some kind of agreement. For the first time in a while,
1:34:16Amazon would start selling Apple products again, which like,
1:34:20things had gotten so bad that it was actually tricky to buy
1:34:24Apple products on Amazon. Like, that's how bad their relationship had gotten.
1:34:28And there was this whole thing. But the critical,
1:34:32like the main thing that Amazon didn't like about Apple was
1:34:36this in-app purchase rule, that Amazon just wanted to offer their own purchases
1:34:40in their app. And Apple doesn't care if you do that for physical goods,
1:34:44or for services. That's why you can use your own payment methods
1:34:48for things like, you know, Lyft rides, or for, you know, Amazon's
1:34:52shopping website, because that's physical goods. Apple doesn't have
1:34:56a payment processing system for that, but they do have, for any kind of
1:35:00digital goods, like buying or renting videos or songs or ebooks,
1:35:04Apple always has required that that goes through their system.
1:35:08And Amazon has always said, basically, no. And that's why, and
1:35:12they have all these other crazy rules, like you aren't even allowed to link out to a browser to do it,
1:35:16and that's why you have all these crazy things where you have situations like the Amazon,
1:35:20like the Kindle app, who won't tell you that you can go to
1:35:24Amazon's site to sign up or buy these things. It just has to kind of assume you'll figure that on your own
1:35:28somehow. And there's all these crazy rejections. Spotify's been involved, and
1:35:32you know, HBO and Netflix have done their own things also. So it's been this whole thing.
1:35:36So what has changed today is that Amazon's
1:35:40app for Prime Video, not for anything else, but so far only
1:35:44for Prime Video, and I think there's a reason for that that we'll get to in a moment, now allows
1:35:48you to buy and rent, you know,
1:35:52or otherwise pay for video using your
1:35:56Amazon billing method that's already entered. So you can't type in
1:36:00a credit card newly into this system. Otherwise, there's some
1:36:04weird exceptions that you might be able to do it some way, but for the most part, you can't do
1:36:08that. But if you, you know, most people with Amazon accounts have some kind of
1:36:12payment method already tied to them, and so if you have that,
1:36:16now you can use the Prime Video app on iOS
1:36:20with Apple's blessing, maybe through gritted teeth, but with Apple's blessing
1:36:24to buy videos that don't go
1:36:28through Apple's payment processing at all and don't pay Apple's 30%.
1:36:32There's a whole bunch of complicating factors here. For instance, 30%
1:36:36has not been 30% for people like Amazon for a while. Large
1:36:40companies like Netflix, HBO, way before Apple
1:36:44announced the whole like 15% for years to and up thing for the rest of us,
1:36:48whenever that was about two years ago, one year ago, long before that, the big companies
1:36:52were already paying something like 15%. Like Apple had already cut deals with some of the big ones
1:36:56because they wouldn't do it otherwise, and they eventually made that public. So there is already
1:37:00a history of Apple having to negotiate with big companies
1:37:04with exceptions to these rules here and there. But this is a
1:37:08really big one, and I think the reason why they were willing to do this
1:37:12for Amazon specifically, and apparently Altus One and Canal Plus,
1:37:16which I don't follow this business, I've never heard of those, I hear Canal Plus is big
1:37:20I think in Europe or France or something, but it's fine, whatever, I'm sure they're big somewhere, but
1:37:24the key part of this, of why Apple capitulated on such a massive
1:37:28thing they've stood so firm on for so long, I think is in the
1:37:32previous sentence. Apple has established a program for blah blah blah
1:37:36video providers specifically, not all apps, video
1:37:40providers, including integration with the Apple
1:37:44TV app, and then AirPlay support, TV OS, Universal
1:37:48Search, blah blah blah blah blah. Okay. That's it right there.
1:37:52Apple was willing to do this because
1:37:56they have something else they're trying to get off the ground, the TV app.
1:38:00Which is really important to Apple's new subscription push services
1:38:04push, TV Plus, it's very important to Apple that
1:38:08the TV app become the home of where everyone goes to
1:38:12start their TV watching it. That's like, they want that to be where
1:38:16people's TV lives. And they don't have all the buy-in.
1:38:20The TV app is not new, all the APIs it uses, things like the Universal Search
1:38:24and the Siri support, all this other stuff they mention, and not to mention just the
1:38:28integration period with the TV app. Netflix still doesn't do that.
1:38:32I guess Amazon probably didn't before this, or I'm sure there were holes in the
1:38:36functionality. And so what Apple needs, like, you know, it was
1:38:40easy when Apple had all the power in their relationship. Like Apple
1:38:44has iOS, this amazing platform that tons of people use
1:38:48and where lots of money flows through, and if Amazon doesn't want to play nice
1:38:52with iOS, that's mostly Amazon's problem, not Apple's problem.
1:38:56And so Apple was able to basically dictate the terms of this in-app purchase relationship
1:39:00for a long time, because they had most of the power in that relationship, you know, from
1:39:04certain angles. But in the last couple years, Apple needs something from them.
1:39:08Apple needs these companies, in specifically certain
1:39:12areas like video, to adopt Apple's APIs and to participate in
1:39:16Apple's TV app and that ecosystem and that whole integration
1:39:20and everything. Apple needs them to do this. And understandably, people at Netflix
1:39:24looked at this and were like, "Why should we do this exactly? Why should we
1:39:28not have our own app be the place to be? Why should we integrate with Apple's TV app,
1:39:32which is inferior in a lot of ways to our own app, and
1:39:36then we lose all these metrics and all this data? Like, why? Why?"
1:39:40Like, there's no reason, from Netflix's point of view, there's no reason to integrate with the Apple TV app.
1:39:44People's home for Netflix content is the Netflix app, period, right?
1:39:48Amazon probably, I haven't followed the Amazon situation closely in the video area, but they probably
1:39:52had similar feelings. Like, why? What's in it for them, right? Why should they adopt Apple's
1:39:56thing and go into Apple's playground if they don't need to, if they're powerful enough that they can have their own
1:40:00app and have all the advantages of that? But the Amazon apps always suck. But anyway,
1:40:04that's a different story. So finally, Apple needed something
1:40:08from them. And Amazon had all the power on that relationship and Amazon's like,
1:40:12"Well, no. Probably, like, why should we integrate your TV
1:40:16app?" Right? So because Apple finally needed something from someone
1:40:20else, for something that's very important to Apple, you know, their TV services push
1:40:24and TV app integration and everything else, that's why I think they finally made
1:40:28this deal specifically for this area. You know? That's why I don't think
1:40:32we're going to see the same thing, like, in the Kindle app or Comixology, like, because I don't
1:40:36think Apple's really trying really hard to make, like, you know, a home of all eBooks on iOS.
1:40:40Like, I don't think that's going to happen.
1:40:42I mean, Amazon couldn't negotiate for that because it's always, like, give and take. What does Apple,
1:40:46how does Apple have to offer, "Hey, we'll let you do sales and rentals through the Prime Video app."
1:40:50Amazon could say, "And also, please let us do it through the Kindle app." Like, it's another thing that you can
1:40:54offer Amazon. But, like, that brings up, the Kindle app and the iBook
1:40:58brings up another point. There's strategy taxes all over this because despite what you said
1:41:02all being true, Apple also has a service that competes with Prime Video
1:41:06called Apple TV Plus and that competes with Netflix, right? So Apple
1:41:10is trying to be a platform and have that TV app,
1:41:14just like you said, and they need the services that
1:41:18people want to be in that. Netflix is big enough to still
1:41:22tell Apple to, you know, go walk off a pier.
1:41:26Amazon is apparently, you know, weak enough that they're willing to integrate, but
1:41:30in the end, Apple, you know, Apple may have to decide
1:41:34do you want to be the app that is the gateway for all of these video services
1:41:38except for Netflix because they're too big? Or do you want to be
1:41:42the biggest video service, right? Apple TV Plus
1:41:46is a sibling to Prime Video,
1:41:50Canal Plus, or whatever, all these other sort of things, like people that have video
1:41:54services, Apple TV Plus is competing with them in the same way that the Apple
1:41:58Bookstore, whatever the hell it's called now, no, it's not the iBook store, competes with
1:42:02the Amazon store, with the Kindle store, right? And it's
1:42:06like, I understand this move from a strategic
1:42:10perspective, especially if you're trying to make Netflix feel bad about being left out, if you can
1:42:14get everybody but Netflix into the TV app
1:42:18and maybe give advantages to apps that integrate with the TV app that aren't available
1:42:22to the Netflix app, like, maybe you can use this as a wedge, but at
1:42:26a certain point, whether that strategy works or not, you've invited
1:42:30all the foxes into the henhouse, and how well is your
1:42:34Apple TV Plus service doing that you're paying billions of dollars to make these, you know, TV shows
1:42:38for when you've got all these other competitors that you are helping
1:42:42by, you know, whatever you're trying to make Netflix jealous with,
1:42:46you're also helping competitors to Apple TV Plus in the process
1:42:50of doing that. So it's this balance where I
1:42:54do think that, you know, both efforts could just fizzle and go nowhere and we just get the status quo, right?
1:42:58But if one of the things starts going really well, there may come a
1:43:02point where Apple has to decide what's more important, Apple TV Plus becoming the next
1:43:06Netflix, or Apple becoming the platform gateway
1:43:10for television viewing. Like, what's the
1:43:14we have to sacrifice one or the other, we can't have both of them, because I don't see any scenario where
1:43:18Apple TV Plus dethrones Netflix as the most important streaming service
1:43:22and all video services go through the Apple TV app and they become
1:43:26the gateway and platform for all TV watching on Apple platforms. Like, I don't see both
1:43:30of those happening. I barely see one of them happening, but certainly not both. So
1:43:34best case scenario Apple has to choose. More likely scenario, they never have to
1:43:38choose because neither one of them goes that well. Yeah, maybe, but I think the power dynamic
1:43:42is so different though. Like, you know, you brought up the whole Fox and Hen house
1:43:46argument, but I think it's more like Apple walked into
1:43:50a Fox house. It was already an established Fox house and Apple's walking
1:43:54in as a hen and being like, "Hey, can I hang out here too? You guys, maybe you could do me a favor
1:43:58also?" Like, Apple had so little power
1:44:02trying to get this TV app off the ground and
1:44:06Apple TV Plus is such a small player, really. I mean, I'm sure Apple's going to talk
1:44:10about numbers and how many people they have using it and everything the next time they
1:44:14get the opportunity to, but the reality is it's a very minor player in this game right now and
1:44:18will probably remain one for some time, if not forever. So, like,
1:44:22Apple had to come to these companies on their terms. And so I'm guessing
1:44:26that this has been in the works for a while.
1:44:30Possibly even since before the Apple TV Plus launch.
1:44:34Well, that's part of the strategy tax things. Because you never know.
1:44:38Like, the organization is so big that someone's trying to work this deal.
1:44:42As you said, the TV world is different. You do need to work with not
1:44:46just streaming services, but also cable providers. There's a whole bunch of parties that Apple has
1:44:50to deal with to do anything in this space, right? So there's one team working on that.
1:44:54And elsewhere in this giant organization, there's another team saying, "We should have our own service
1:44:58because we could charge people for it." And, like, you have to reconcile that somehow. It wouldn't surprise
1:45:02me at all if there was some tension between these two efforts because they are at odds with
1:45:06each other in many respects. Anyway, I'm guessing
1:45:10this was in place for a while. This was negotiated a while ago.
1:45:14Probably back when Apple and Amazon announced they reached some kind of settlement with everything
1:45:18two years ago. I'm guessing this has been in the works since then. Because this also
1:45:22-- so, Guy Rambo and Steve Trouton-Smith have been doing some spelunking
1:45:26on the Amazon binary, and it has a private entitlement
1:45:30to get extra data from StoreKit. And so there's
1:45:34clearly some kind of, like, software side of this that they
1:45:38had to do in addition. And given that that software side of things
1:45:42involves the StoreKit API, which is one of
1:45:46Apple's largest, most important, and most creaky and old
1:45:50and infamously horrible things to work on web services,
1:45:54I'm guessing that the software implementation side of this took some
1:45:58time and required certain rollouts of certain
1:46:02API changes and blah, blah, blah. So I'm guessing that
1:46:06this has been in the works for a long time as part of that big agreement
1:46:10that they made a couple years ago. And that's why it's just coming out now.
1:46:14But anyway, a lot of people -- this is rubbing the wrong way because it's like
1:46:18Apple's treating these big companies separately and giving them separate rules
1:46:22than the rules that the rest of us have to follow within
1:46:26AppPurchase. And you know what? That's true.
1:46:30It's wonderful that Apple doesn't do that most of the time.
1:46:34It's wonderful that for the vast majority of
1:46:38aspects of the app store, big companies and small companies
1:46:42play by the same rules. But that's not true for all the rules. That's just
1:46:46business, you know? Facebook and Uber get away with murder
1:46:50with what their apps do and the privacy things that they
1:46:54take. Facebook and Uber, you know, big companies like that, like
1:46:58they're so important that Apple can't ban them from the app
1:47:02store. That's business. That's the reality.
1:47:06Amazon is so important that no matter what Amazon does,
1:47:10Apple can't kick them out of the app store. Spotify probably
1:47:14also has this status, right? These big companies
1:47:18are super important to Apple's platform. Way more important than
1:47:22Overcast or any other independent app that any of us work
1:47:26on and run and anything like that. Apple does make business deals
1:47:30sometimes with big companies behind the scenes that do change the rules for them. Or they
1:47:34do give them, like in the case of Facebook and Uber, they do give them like
1:47:38maybe a private heads up from an executive, "Hey, you need to stop doing this."
1:47:42Rather than just ban them from the app store immediately upon their first offense.
1:47:46Big companies get special treatment because they're really important to Apple.
1:47:50And because Apple doesn't have all the power in those relationships. Yes,
1:47:54Apple made a special deal with a small number of big companies.
1:47:58It's not the first time, it won't be the last time, and that's just business.
1:48:02I don't love that as any developer who, you know, I have to follow
1:48:06like I can't implement my own credit card thing for, you know,
1:48:10my stuff in Overcast. I have to pay Apple's 30%. But
1:48:14that's the reality. That's business. Apple can live without
1:48:18Overcast. Apple can't live without Amazon and Netflix and HBO and
1:48:22stuff like that. So they have to make deals with big companies sometimes.
1:48:26That's just the reality. And so from that angle, you know, again,
1:48:30I don't love this, but I see why they have to do it.
1:48:34I think this is just, I keep fast forwarding like a year and saying like, "How could this possibly
1:48:38shake out?" And especially if this was a thing made
1:48:42before Apple TV+ was announced, before the rumors
1:48:46of it were solid, like, you know, there's been, there's always a possibility
1:48:50that Apple could do what they did with LG TV+, but it took for the
1:48:54recent couple of years for it to really come to a head. So say this deal was negotiated three years ago,
1:48:58Amazon had to judge. If we do this, what if Apple just comes out
1:49:02with their own video service and competes with us? And the judgment could be, "Well, it's going to take them a while to do that, so we
1:49:06should just do this deal anyway, and, you know, we'll get all the get-ins good. We'll
1:49:10sell stuff to our app. We won't have to pay them 30%." And if they come out with the service,
1:49:14it'll probably take them a while to get up to speed, and, you know, like, you can always stop
1:49:18whatever, I don't know what the contracts look like, but I can imagine them doing
1:49:22this while they can, making money while they can, and re-evaluate in a year or two.
1:49:26Because even for Amazon, Amazon is also trying to compete to Netflix. They don't want
1:49:30to be just a little serf in the Apple TV, you know, app kingdom.
1:49:34Everyone wants to be big enough that you can just rebuff
1:49:38Apple and say, "No, we're doing our own thing." Right now, Netflix is big enough to do that,
1:49:42and continues to do that. Netflix took all their purchases out of the
1:49:46Apple ecosystem, and Netflix doesn't want to be in the TV app
1:49:50thing that tries to control everything, right? Amazon would love to be
1:49:54in that power position, but apparently they're not. And also, I think Amazon has more things
1:49:58they want from Apple, which kind of surprises me that they didn't get more concessions, like, you know, the Kindle
1:50:02store or whatever, because Amazon isn't just a video company. They have all sorts of things.
1:50:06They would love to not pay Apple for book purchases, to sell
1:50:10books on phones and not pay Apple 30%. You can give Apple 0%,
1:50:14right? But apparently they couldn't negotiate for that.
1:50:18So, as the balance of power shifts, and interestingly,
1:50:22in this particular scenario, the balance of power among these video
1:50:26services is influenced by
1:50:30creative output, right? If you get good TV
1:50:34shows, that puts you in a more powerful position. It's unlike many
1:50:38other scenarios where, like, it's a technical concern, or there's some other
1:50:42regulatory scenario. The reason Apple
1:50:46TV+ has been successful as it has been is A, the free trial, and B,
1:50:50the fact that they made TV shows that weren't terrible, that people
1:50:54were interested in watching and talked about. That's what made Netflix what it is today.
1:50:58Netflix started by just giving a bunch of content that they didn't make, and they started making their own content.
1:51:02In the beginning, they had one or two good ones, and now they make so much stuff that the
1:51:06percentage of stuff that's good may only be 1%, but that
1:51:101% is a lot. That is a weird scenario for
1:51:14Apple to be in. I mean, we talked about this when they were doing the service
1:51:18to begin with. Apple has a music store,
1:51:22but they don't have a bunch of bands that they started. Well, I guess the breakpoints, but they don't have
1:51:26a bunch of bands that Apple is fielding, but they're making TV
1:51:30shows. It's weird to see Apple suddenly in a
1:51:34scenario where you can really influence your odds of success
1:51:38by doing things that have nothing to do with technology and everything to do with
1:51:42creative output. It's like, how do you become a powerful movie studio? You make good
1:51:46movies that people want to buy or people want to watch. Pixar is
1:51:50Pixar because they made a bunch of good movies. Now, Apple can,
1:51:54in theory, try to get a leg up on its competitors in the
1:51:58studio market. Yes, there's the technology stuff going on over here. Yes, there's the TV app and
1:52:02there's all sorts of angles of control. We control the store and there's all those things that they normally
1:52:06do. But also, A, they have to make good shows
1:52:10and B, if they make really, really good shows and a lot of them, it can give them an
1:52:14advantage, which is a strange situation
1:52:18for a technology company to be in, but it's a fact of life.
1:52:22Netflix became Netflix not because their app is awesome. Hell, they were able to
1:52:26voice stupid auto-playing trailers or whatever on us
1:52:30before they finally gave us a preference for that. The reason they were able to annoy us to that degree
1:52:34is Netflix is not Netflix because of their app. They're Netflix
1:52:38these days because of their original content and that's what Apple is competing with ultimately
1:52:42in the end. For Apple TV Plus to even be a contender,
1:52:46it has to have good enough content and if it ever wants to
1:52:50get bigger and start taking market share, it has to have better content.
1:52:54Thanks to our sponsors this week, Squarespace, Eero, and Postmates
1:52:58and we will talk to you next week.
1:53:02Now the show is over, they didn't even
1:53:06mean to begin, cause it was accidental
1:53:10Oh it was accidental
1:53:14John didn't do any research, Marco and Casey wouldn't let him
1:53:18Cause it was accidental
1:53:22And you can find the show notes
1:53:26at ATP.FM and if you're
1:53:30into Twitter, you can follow them
1:53:34@C-A-S-E-Y-L-I-S-S
1:53:38So that's Casey List M-A-R-C-O-A-R-M
1:53:42Anti Marco Arment S-I-R
1:53:46A-C-U-S-A Syracuse
1:53:50It's accidental, accidental
1:53:54They didn't mean to accidental, accidental
1:53:58Tech podcast so long
1:54:02John do you want to tell me about consoles? Cause I'm really excited to hear about it
1:54:06Yeah this is console follow up and after show
1:54:10Do we have after show follow up? That's right, well it's not really follow up
1:54:14I guess it is, it's stuff that I forgot to mention or didn't know, so last time I talked about
1:54:18the new Xbox and the PlayStation 5 and I hadn't watched the full Sony presentation
1:54:22and so now I have, but I did talk about it last time
1:54:26that Microsoft had decided against the trends in the industry
1:54:30to lock the clock speed on their
1:54:34system on a chip, essentially, instead of having a turbo boost
1:54:38or going faster and slower clock speed depending on load and all the stuff that modern
1:54:42CPUs do, they had it just stay at a constant clock speed, it's actually two clock
1:54:46speeds because if you enable symmetric multithreading it goes slower or faster
1:54:50I forget which, but anyway, the point is you're in a particular mode and it just runs that speed the whole time
1:54:54no varying while it's running in whatever mode it's running in
1:54:58which is super weird and requires aggressive cooling
1:55:02but it makes a very consistent experience, like that was their pitch, we don't want you to have the console
1:55:06if it's in a place where it can't get a lot of cooling your game starts hitching
1:55:10Sony took a different approach, in some ways a more traditional
1:55:14approach, their thing does change clock speed, but they spent a while on their presentation
1:55:18talking about how it doesn't change clock speed based on temperature
1:55:22they didn't do a good job explaining, or at least I didn't do a good job absorbing why
1:55:26they don't want to do it based on temperature, the idea was like, well, temperature is
1:55:30one thing, but it can be a false signal maybe
1:55:34like, really, we don't want it to
1:55:38like, here's the thing, this is the reason I'm confused, you don't want
1:55:42your chip to melt, like temperature is important, like in the end that's what you're controlling for
1:55:46you know, if the thing is in the console under your TV
1:55:50and it's in a cabinet where it doesn't have good airflow, and you're playing a game
1:55:54and the clock speed is whatever, 2 point something gigahertz
1:55:58and stuff starts to get so hot, at a certain point
1:56:02chips stop working, like there's a safe temperature for chips operating in, if they get too hot
1:56:06they just don't work anymore, and worst case scenario you could even damage them, right?
1:56:10so in the end that's what you're controlling for, heat is the enemy here
1:56:14but I guess apparently you can't
1:56:18get accurate enough or reasonable enough temperature sensing
1:56:22to just have temperature be the only thing
1:56:26that you're measuring, or if you did you would end up clocking yourself slower than you technically needed to
1:56:30so it seems like what they're looking for is a situation where they
1:56:34run as hot as they possibly can, but not
1:56:38hotter, and they think the way that they can do that
1:56:42is instead of controlling for temperature, they have sort of assigned
1:56:46cost, sort of heat cost to certain
1:56:50operations, and they have special circuitry to say, if you're doing this
1:56:54type of operation, it takes this much of the heat budget, and this type of operation takes this much of the heat budget
1:56:58and they sort of real time measure what the actual CPU
1:57:02and GPU are doing, and from that come up with
1:57:06they try to assemble that all together to
1:57:10come up with a profile of how much energy are we using
1:57:14and then just, they have sort of a balanced equation that says we can do this amount of work
1:57:18with, this is the amount of cooling we have, and this is how
1:57:22our cooling relates to our clock speed and we can do this amount of work with this amount of heat, so they're
1:57:26trying to, I guess get closer to that ragged edge
1:57:30don't back off just because this one part's going to get really hot for a second
1:57:34because we know based on the workload across the entire system on a chip that you'll be fine
1:57:38because overall we're not doing that much work, so
1:57:42a very different strategy from Microsoft, which committed fully
1:57:46to just constant clock speed, again two different clock speeds for whether symmetric multithreading
1:57:50is enabled or not, and Playstation 5 is saying we're going to be variable but we're going to do
1:57:54variable clock speed better than anyone else has done it, so good luck to
1:57:58Sony, it was a little bit scary, I hope it works out better than my
1:58:02description of it made it sound. And the second weird thing Sony is doing, Sony being Sony
1:58:06they have this big part of the presentation still about 3D audio, or
1:58:10I don't know, audio that sounds more interesting in games, having more different audio
1:58:14sources, I'm fully sold on the idea that this can be more immersive
1:58:18because sound, you don't appreciate it until you've played
1:58:22a really good game with sound, but point sources and sound
1:58:26in games, in first person shooters and stuff you want to hear footsteps of enemies
1:58:30coming, are they behind me, are they to my left, to my right, sound can be super important
1:58:34but there's a small number of those sound sources and localizing them is difficult
1:58:38most interestingly, most
1:58:42movie based surround sound things
1:58:46only support a small number of sources, because movie soundtracks when they're mastered
1:58:50I say a small number of sources, you know they have like
1:58:547 speakers or whatever around you, or whatever it is, 8 or 9, there's a lot of speakers around you
1:58:58but it's not the number of speakers that determines the number of sources, the sources are like
1:59:02how many different individual point sources of sound can there be in the
1:59:06scene, and I think the movie ones are like 32 or something, it's a big number
1:59:10but it's not a huge number, and Sony wants to have a situation where you can have like
1:59:145000 point sources of sound, the example they gave was like if you're in the middle
1:59:18of a rainstorm and each raindrop was its own sound source with a drop hitting the ground, that's what it's like in real life
1:59:22each raindrop that hits the ground is its own sound source, whereas
1:59:26in games today, they just record the sound of the rain and put it in a channel, or make it
1:59:30come from all around you or whatever, but that rain sound is like flattened out
1:59:34so anyway, Sony's super into the 3D audio, they want to have
1:59:38this new audio engine, they have dedicated hardware for it, it's very powerful, they've spent time
1:59:42bragging about it, Marco you would like this part of the presentation, you should watch it, they talked about
1:59:46ring buffers and all sorts of... I'm a stereo guy
1:59:50well that's why I think you'll like this part of the presentation, first of all, first they talk about hardware
1:59:54that's tailored for processing sound, which you must be familiar with, like in terms of
1:59:58how is sound processing different than general
2:00:02computing, so they made a compute unit that technically has as much power
2:00:06as like the entire Playstation 4 CPU
2:00:10but it's just for sound, and because it's just for sound, you could remove all the caching and everything
2:00:14because you just want the data to flow through it, right, so it's very small and purpose-built
2:00:18anyway, yeah, so they have dedicated hardware for the sound and
2:00:22it allows them to have these thousands of sound sources and stuff like that, and the
2:00:26they're also focusing on output devices, as you would imagine, and their first output device
2:00:30that they're focusing on is not a surround sound system or a soundbar or whatever, it is
2:00:34headphones, because this system is not about "hey, how can we play
2:00:38sound on a bunch of different speakers?" in fact that makes it more difficult for them, you have
2:00:42two ears, and they just want the sound to go into your two ears, they can
2:00:46simulate all these sound sources and figure out the culmination
2:00:50of these sound sources, what sound hits your ears at what time
2:00:54and that's their first use case, to make you feel like you're
2:00:58really there just with headphones, just with stereo sound going directly into your ears
2:01:02the tricky part about that is how do your ears
2:01:06tell sound is like exactly to my left, or a little bit behind me
2:01:10or up or down, like all the sort of fine details of where the sound comes from
2:01:14if you just try to do that with timing, or when the sound hits your left ear
2:01:18and your right ear, you can get pretty far with that, but it's
2:01:22difficult, there is a thing in the world of sound design
2:01:26that Sony talked about a lot in their presentation, which is called
2:01:30HRTF, which stands for Head-Related Transfer Function
2:01:34and it's basically, what does
2:01:38your head do to the sound that affects how you hear it
2:01:42and your head being like all of the stuff that's attached to your body
2:01:46surrounding your eardrum, which senses the sound. In other words, it's the shape of your
2:01:50head and your ear, and to figure out
2:01:54every individual person has their own head-related transfer function, which is like when sound comes and it hits your head
2:01:58how does it bounce off all the little wrinkly weird parts of your ear and stuff
2:02:02and get inside your head to your eardrum and you hear it
2:02:06and so what they're trying to do is
2:02:10come up with a set of head-related transfer functions, kind of like the AirPod exercise
2:02:14you've got to come up with a thing that fits in everybody's ear, but it can't fit in everybody's ear, so they've got to come up with
2:02:18a set of head-related transfer functions that is representative of most of the
2:02:22population, and they're thinking of like, maybe you could have a game where you try each one
2:02:26and you try to pinpoint where the sound sounds like it's coming from and the one that is
2:02:30closest to the real virtual sound source is the one that you pick
2:02:34they even talked about maybe you could take a picture of your ear or a video
2:02:38of your ear and send it out over the network and it would make
2:02:42a custom head-related transfer function for your ear. The way they make the real ones
2:02:46by the way, they showed this in the video, is they put little sensors way deep in your ears
2:02:50and then they make you sit in a fixed position in this giant sound chamber
2:02:54and they play point source sounds from all over the place and then they record from
2:02:58inside your ear what it sounds like to the inside of your ear
2:03:02right, and every different person has their own head-related transfer function that's
2:03:06custom to them, so if they can do a custom one for individual people, like they're
2:03:10not going to put you in a sound chamber, but if they know the shape of your ear they can sort of use
2:03:14machine learning to simulate the head-related transfer function
2:03:18who knows if they'll ever do that, I think what they'll just do is ship with say
2:03:2220 different head-related transfer functions that are representative of a range of human ears
2:03:26and then have some kind of game-like experience where you pick the one that is closest to your ear
2:03:30but it's fascinating, they spent so much time in the presentation
2:03:34on this, it could all come to nothing, but it sounds super
2:03:38cool, like I'm all ready to be in a rainstorm where every raindrop is its own
2:03:42sound source and I'm all ready to be able to hear footsteps of approaching people
2:03:46in a first person shooter with much more accuracy than I can hear it right now
2:03:50so I once again recommend to all you people look into this, these videos
2:03:54we'll link them again, if you're not interested in consoles they may sound boring, but check it out, cool tech stuff
2:03:58yeah, so first of all I think there's two angles of this, I actually, both
2:04:02of these things I think are actually kind of fascinating, so the
2:04:06heat management of the processor thing, I think actually based on
2:04:10your explanation I think I do understand what they're doing, and it's actually kind of ingenious
2:04:14so if you just fix the clock speed like what Microsoft is doing, you lose
2:04:18out on potential performance and potential power savings
2:04:22because you could, like, most
2:04:26chips have a little bit of headroom, they can run faster for a short
2:04:30time, so they can do certain things faster for short times
2:04:34and so if Microsoft is not allowing that because they can't sustain
2:04:38that in a predictable way, because that's
2:04:42the whole goal here is, with the game console you want extremely predictable
2:04:46performance for the developers, that way you can make games that work the exact same
2:04:50way on every Xbox 17 and every PS5
2:04:54or whatever, and it's fine, right, and that's one of the great
2:04:58advantages of working on game consoles, is that you have predictable hardware, which you don't have
2:05:02in the PC world necessarily, so the way Microsoft is
2:05:06making theirs predictable sounds like they're just fixing the clock speed, so it always runs at this speed
2:05:10period and, you know, the dev kit machines that you're going to write the games on
2:05:14are going to be running at the exact same speed as all the consumer machines out there
2:05:18well it certainly sounds like they're doing the exact same thing, just in a different way
2:05:22but they're achieving the same result. What you want is predictability
2:05:26not necessarily constant clock speed, so as long as, so if they
2:05:30know that they can boost the CPU up to this rate for this
2:05:34operation, up to this many times per millisecond or whatever, they
2:05:38can build that into the CPU's
2:05:42power management unit or whatever, however this is working at the low level
2:05:46they can have it be, like run these heuristics and clock itself up and down
2:05:50instead of being based on temperature, based on what it's doing, so it
2:05:54achieves many of the benefits of dynamic clock speed
2:05:58but in a predictable, deterministic way
2:06:02so that every single PS5 will run this at the exact
2:06:06in the exact same way with the same performance characteristics.
2:06:10Yeah, that's exactly what they said in the presentation. Yeah, that sounds awesome. Right, so it does
2:06:14but as I said, you can pretend that that's true, it's like
2:06:18well this will always be exactly the same, because it's a deterministic state machine, I guess that probably changed the
2:06:22profile, but anyway, it's deterministic, but the thing that's your enemy
2:06:26is heat, and heat doesn't care about the repeatability of your
2:06:30game thing. If your PlayStation 5 is in a really hot spot
2:06:34and the power thing says, I'm going to do the same thing I did, the same thing I
2:06:38always do based on my estimation of the cost of these operations, I believe we can run at this
2:06:42speed, and it may be wrong, and you make it too hot
2:06:46and getting your GPU too hot or your CPU too hot
2:06:50can make bad things happen. That's why people go off temperature, because ultimately that's what you're trying
2:06:54to control, so I totally get the pitch, right, but it scares
2:06:58me to think that no matter what temperature, I mean I'm sure there's some temperature cut off on the top
2:07:02of it, but no matter what environment this chip is in, it is going
2:07:06to do exactly the same thing it did during development, because whatever
2:07:10the fixed profile is for the cost of each operation, and that may not be
2:07:14the appropriate thing to do. Now you could say it's the same situation for a fixed clock speed, because if it's fixed clock
2:07:18speed, and you put that in a really hot situation, how do you know it's safe to go at that clock speed?
2:07:22Well, we think our cooling solution can handle yada yada yada, but you might be wrong. So I
2:07:26suppose in the end it's the same trade-off, it just seems to me that what
2:07:30Sony wants is, like you said, to be able to use more of the
2:07:34flight envelope to use aviation terminology. There's a certain range of
2:07:38operations that are safe at any given time. Can I go a little
2:07:42faster now? Do I have to go a little slower? They want to fill that envelope from top to bottom,
2:07:46and Microsoft is the more conservative approach, they're saying we're going to draw a straight line, that's the
2:07:50envelope, we hope our cooling will always keep you under that line,
2:07:54but you're just going to be at that speed all the time.
2:07:58In the end they're both trying to do the same thing, but totally different approaches, and
2:08:02honestly they're both a little bit scary to me, now that I think about it.
2:08:06Well, but I'm sure neither one of them would reach the point where it would kill itself, right?
2:08:10I'm sure every chip... Talk to Xbox owners.
2:08:14Well, yeah. The red ring of death was exactly a heat management
2:08:18issue. That's true. But in general, any modern
2:08:22chip from the last 15, 20 years has a thermal shutdown
2:08:26protection built in, so that if it's running above some maximum temperature,
2:08:30usually it's like 100 degrees Celsius, something like that, whatever it is, it has a thermal shutdown
2:08:34limit where it will just shut itself down. The machine will crash and it'll just
2:08:38that's it. It will refuse to fry itself, basically.
2:08:42But that's different from usually from the temperatures that it's going to
2:08:46throttle itself back clock speed wise at. But anyway, I think this works the same way,
2:08:50just using different metrics. Either way, when they're designing
2:08:54an enclosure and a system and everything, they probably have
2:08:58a certain heat budget of how many watts are we going to dissipate from the cooling
2:09:02solution? So how many watts can the CPU put out and remain
2:09:06within the design here? And either way, it's like
2:09:10if we run this at a constant clock speed, we know it's going to be x watts maximum
2:09:14no matter what you do to it. And I think what Sony is talking
2:09:18about sounds like a very similar kind of system. Just they know
2:09:22the power cost of everything so they can know roughly what wattage
2:09:26of heat they are generating. So it doesn't
2:09:30I think it's kind of the same. You get the same output
2:09:34or rather you have the same design constraint of like you need a cooling solution
2:09:38that will be able to dissipate x watts of heat in
2:09:42most conditions to keep the temperature under y.
2:09:46That's the design no matter, in either case. And if the
2:09:50Sony chip's power management stuff is intelligent enough to not rely
2:09:54on measuring the temperature and to just be like, look, I know that if I do
2:09:58this sequence of operations, I will generate this many watts
2:10:02of heat output. And that's kind of cool. It's a very clever way to do it.
2:10:06And in a situation like this where like predictability of performance is
2:10:10more important than either eking out every last
2:10:14like 5% of performance at the top end or
2:10:18having a certain ideal temperature that you always stay at, that actually makes a lot
2:10:22of sense. It's kind of clever. One other factor to throw in here
2:10:26this is not the part of the presentation that Sony wanted to emphasize, but Sony has way
2:10:30fewer GPU cores than the Xbox. They're both using AMD parts. They're both using
2:10:34very similar, if not exactly the same CPU
2:10:38and GPU cores with minor tweaks. But Sony has, I forget what the numbers are, but it's like
2:10:42they have 30 something and the Xbox has like 50 something. So it's interesting
2:10:46that the machine with just more to cool and
2:10:50you know, like nearly double the number of GPU cores is the one using
2:10:54the fixed clock speed. We haven't seen what the PlayStation 5 looks like. The Xbox
2:10:58cooling solution is no joke. It is a really big heatsink with a
2:11:02humongous fan in it. So, you know
2:11:06the PlayStation 5 seems like with this variable thing that they could get away
2:11:10with a console that has a less gargantuan cooling
2:11:14solution. But we'll see. I don't think anyone knows what that machine looks like yet.
2:11:18So, moving on to the audio thing. I haven't watched this presentation
2:11:22yet, but there's a lot of overlap here with binaural audio.
2:11:26This is something that's existed for a very long time, decades.
2:11:30I think people discovered binaural audio like in the 70s or 60s, like it's been around for a long time.
2:11:34But the idea is, as you mentioned, we have two ears
2:11:38but we hear in 3D. How can this be when stereo
2:11:42sound is allegedly only two dimensional? And the answer
2:11:46is very complicated. And the idea is like
2:11:50our ears are shaped in a certain way, the sound
2:11:54travels through them in a certain way, our head has this volume in space
2:11:58that affects sound that we hear in certain ways. It's all very complicated.
2:12:02But if you place
2:12:06two microphones in a space, like suppose you want to
2:12:10record a concert hall. You put two microphones next to each other
2:12:14about a head's width apart, and you play that back
2:12:18on headphones or even on regular speakers, it'll sound roughly like there's an orchestra
2:12:22playing in front of you. It's not going to be incredibly precise, but it'll sound great.
2:12:26It'll sound, you know, approximately realistic. But it's not going to
2:12:30be super precise 3D sound. But
2:12:34if you have microphones that
2:12:38basically are in simulated or real ears
2:12:42on the sides of a simulated or real head, and that have
2:12:46like the shape of the ear around it, like, because you know,
2:12:50obviously part of your ear is outside of your head, the whole shape
2:12:54of your ear affects the shape of the sound waves coming in and it alters the sound.
2:12:58So if you can have a very like
2:13:02human ear and human head like recording device,
2:13:06you can record binaural audio, which is a style of recording,
2:13:10that like the two microphones are in
2:13:14these ears, and the idea is then if you play it back
2:13:18with headphones that are in basically the same spot in your ears,
2:13:22relaying basically the same sound that hit them, which was partially altered
2:13:26by the head and ears around the recording microphones, then
2:13:30it will sound much more precisely 3D. And this is a wonderful
2:13:34thing, if you just go to YouTube or wherever, search for binaural
2:13:38audio and you'll find tons of recordings of binaural things,
2:13:42if you've never experienced this effect, it's
2:13:46quite something. Like it's kind of mind blowing how incredibly
2:13:503D and realistic audio can be when it's recorded in
2:13:54this way. Now the way this is usually recorded is either
2:13:58that you can actually buy a fake head with fake ears that have
2:14:02microphones in it, they're extremely expensive, I think they start at like
2:14:06$800, like they're very expensive, and
2:14:10this is actually, the market for these is actually increasing now with VR, there's like 3D microphones for VR
2:14:14stuff, but anyway, you can buy like a whole fake head, or you can do what
2:14:18I did to play around with this for like less than a hundred bucks,
2:14:22you can get what look basically like earbuds.
2:14:26But, and they are earbuds, but on
2:14:30the outside of each one is a microphone, facing out, and the inside
2:14:34is headphones, earbuds facing in, and these are inexpensive because they're basically
2:14:38just earbuds plus microphones, they're inexpensive, and they use
2:14:42your own head to make your own head transfer function because you're just having
2:14:46earbuds in your ears, and so you can, for not that much money, experiment
2:14:50with this, where you can basically record, you know,
2:14:54yourself in a room or walking through the
2:14:58world or whatever, record exactly the way your ears inside
2:15:02your head are receiving the sound with those microphones that are on the outside of the earbuds.
2:15:06And then you play it back, wearing those same earbuds, or pretty much
2:15:10any other headphones, but especially when you wear those same earbuds, and it sounds
2:15:14remarkable, like you can't believe how much it feels like
2:15:18you are actually there again, it's spooky how accurate it is.
2:15:22It's really cool. So some of these things were way inside your ears, because
2:15:26the earbuds necessarily have to be out of your ears, so they're catching the sound
2:15:30in your ear, right? So the Sony ones, if you look at the video, are like
2:15:34these little wires going into the poor person's ear canal, right, so that's what they're
2:15:38using to build their standard set of 20 head-related transfer
2:15:42functions. But yeah, you won't have one custom for yourself unless you go into that Sony
2:15:46chamber, but I think games are an interesting scenario, and you mentioned VR,
2:15:50it's similar, where the positioning is the most important part.
2:15:54People aren't really cared about the fidelity
2:15:58of the footsteps sounds you hear. You want to know exactly where the footsteps are coming from, or even
2:16:02just like environmental stuff where rain is falling on cobblestones,
2:16:06car tire going across pavement versus going across wood, someone
2:16:10crunching on gravel, are the walls made
2:16:14of wood, are they made of metal, how high is the ceiling in this room?
2:16:18Those type of things, this is all about both how the sound gets into
2:16:22your ears, where it's coming from, but also how does the sound bounce around, that's why they have this dedicated
2:16:26audio engine to have 5,000 sound sources. So if you
2:16:30get, if you're in an environment and something happens, not only does it
2:16:34happen in a particular place, but the room affects how it sounds. Games
2:16:38have had this forever, like even going back to Mario 64 or probably earlier,
2:16:42or think of any driving game where you go through a tunnel and they change the audio, like they just put some filter
2:16:46on the audio so it sounds different when you're in a tunnel. Yeah, some reverb filter.
2:16:50Yeah, we're at the stage now where instead of just having a filter when you go through
2:16:54a tunnel, like every sound source will be affected by the surrounding
2:16:58geometry in the game at all times, right? And so the tunnel, you don't have to write any
2:17:02special code for the tunnel, you just make a tunnel and you drive through it and you get the sound.
2:17:06And so, like, this part of the presentation, it's hard to do a presentation
2:17:10about sound, because people just want to see graphics or want to hear you talk about graphics,
2:17:14but I honestly think that if this stuff works and developers actually use
2:17:18it, it could be, and this is certainly what Sony hopes,
2:17:22as big a differentiator for the Sony platform as graphics have been
2:17:26in the past, and Sony better help, that's the case because their graphic power is
2:17:30significantly less than the Xbox. Marco's
2:17:34going to buy a PS5 just so he can hear this audio, and so he can play the head-related transfer
2:17:38function game, and pick the one that matches his weird ears.
2:17:42I had such an amazing sound experience with
2:17:46what's the Mario game for the Switch?
2:17:50Odyssey? That's it, yeah. I kept thinking Galaxy, I knew that was wrong. Yeah. Mario Odyssey has amazing
2:17:54sound. And that's without doing any of this fancy stuff. Right. Like, that's just
2:17:58doing the old way of, like, just programming it so that, oh, when you go into the 8-bit mode, it plays the 8-bit
2:18:02sound. When you go underwater, it muffles the other sound. It's like, it's just doing the
2:18:06old way, but man, that is such a great game for sound.
2:18:10Having this technology is good because Nintendo can afford to do that, like, the old
2:18:14manual way, essentially, because they have a lot of money and this game's going to sell a million copies.
2:18:18But if you got, like, basically for free by just placing sound sources
2:18:22and defining materials, you got that, you know, without having to
2:18:26hire armies of people to manually set up all your sound, that's, hopefully
2:18:30will make every game have better sound, not just the ones that can afford to
2:18:34manually mess with sound in every possible environment, and record
2:18:38500 different footsteps depending on whether you're walking on these different materials, you know.
2:18:42Right, like, it's almost like back in the days, like, when 3D was just
2:18:46getting started, like, if you wanted to make a 3D game, you generally had to write your own 3D
2:18:50renderer, right? And then eventually, as technology got better, and the
2:18:54APIs got better, and the hardware got better, you could then eventually just say, alright, just put a
2:18:58triangle at this geometry, and put the camera at this geometry, and you figure out how to make that
2:19:02happen, right? And then, of course, eventually now nobody writes their own engine anymore, but, you know, the point
2:19:06stands, like, there hasn't been things like that for audio that were very
2:19:10advanced, and so this sounds like a way to do that, which sounds very exciting.
2:19:14And the differentiator, I think, what Sony hopes is, like, Xbox can't do this, because
2:19:18if you try to do this sound processing on a general purpose CPU, you will use all your
2:19:22CPU power, like, so their brag was their audio engine on the PS5,
2:19:26you couldn't, you'd have to absorb every compute resource on the PS4
2:19:30to match it, because it is a purpose-built sound processing engine, that's all it does.
2:19:34And I don't think the hardware exists at all on Xbox, so Xbox can't really compete
2:19:38with this, because they can't spare the cycles to dedicate
2:19:42to audio, and they don't have dedicated hardware for it.
2:19:46(beeping)