Hypercritical

00:00:00 ◼   ► [Music] [TS]

00:00:02 ◼   ► this is hypercritical weekly talkshow [TS]

00:00:05 ◼   ► ruminating on exactly what is wrong in [TS]

00:00:07 ◼   ► the world of Apple and related [TS]

00:00:08 ◼   ► technologies and businesses [TS]

00:00:09 ◼   ► nothing is so perfect that it can't be [TS]

00:00:12 ◼   ► complained about by my co-host John [TS]

00:00:14 ◼   ► siracusa i'm dan benjamin this is [TS]

00:00:16 ◼   ► episode number 57 today's Friday March [TS]

00:00:19 ◼   ► 2nd want to make sure that we thank our [TS]

00:00:21 ◼   ► two sponsors right at the top of the [TS]

00:00:23 ◼   ► show tiny letter comm send email [TS]

00:00:26 ◼   ► newsletters the easiest most simple way [TS]

00:00:29 ◼   ► possible and Shopify create your own [TS]

00:00:32 ◼   ► online store today we also want to [TS]

00:00:35 ◼   ► mention the bandwidth for this episode [TS]

00:00:36 ◼   ► is provided by Mac Mini Kolo I host [TS]

00:00:39 ◼   ► Ansel the servers there to help run 5x5 [TS]

00:00:41 ◼   ► and you can - Mac Mini Colo net / 5x5 as [TS]

00:00:45 ◼   ► a special discount for this is why we [TS]

00:00:48 ◼   ► need faster file systems I'm learning [TS]

00:00:50 ◼   ► how to use computers today John how are [TS]

00:00:52 ◼   ► you I can relaunch my apps a little bit [TS]

00:00:55 ◼   ► faster you have all SSD you've got [TS]

00:00:57 ◼   ► mirrored and striped SSD drives in all [TS]

00:00:59 ◼   ► of your chips Dave nanion the guy who [TS]

00:01:02 ◼   ► makes SuperDuper I doesn't trust the SSD [TS]

00:01:05 ◼   ► drives typically he doesn't trust Lee [TS]

00:01:08 ◼   ► you know supposed to trust any drive Oh [TS]

00:01:10 ◼   ► point is I'm gonna have but he trusts [TS]

00:01:12 ◼   ► those less than he distraught he he [TS]

00:01:14 ◼   ► distrusts them even more than he [TS]

00:01:16 ◼   ► distrust his other what about the people [TS]

00:01:17 ◼   ► of emailed asking you what about SSD is [TS]

00:01:20 ◼   ► John yeah I probably distrust me too [TS]

00:01:22 ◼   ► well today's show will be a follow up [TS]

00:01:24 ◼   ► show on file systems so we will talk [TS]

00:01:28 ◼   ► about all these issues we don't have any [TS]

00:01:29 ◼   ► other thing to talk about today no no [TS]

00:01:32 ◼   ► iPad 3 comments or predictions maybe [TS]

00:01:35 ◼   ► we'll throw that in at the end ok [TS]

00:01:38 ◼   ► alright really going here now that my [TS]

00:01:40 ◼   ► audio is actually working I would like [TS]

00:01:41 ◼   ► to blame this audio stuff online but [TS]

00:01:43 ◼   ► this happened in Snow Leopard too like [TS]

00:01:45 ◼   ► you know everything's all set up and the [TS]

00:01:46 ◼   ► last thing I do is you know your testing [TS]

00:01:48 ◼   ► 1 2 or 3 into my mic and noting the [TS]

00:01:51 ◼   ► meters don't move at all I'm plugged mic [TS]

00:01:54 ◼   ► plugged back in nothing so the old [TS]

00:01:57 ◼   ► reboot this is the only time I ever [TS]

00:01:58 ◼   ► reboot my Mac you know is when the audio [TS]

00:02:00 ◼   ► stops working and yeah before you [TS]

00:02:01 ◼   ► started doing the show you had an uptime [TS]

00:02:03 ◼   ► of like three years and soon as you have [TS]

00:02:05 ◼   ► this show you're lucky to get the full [TS]

00:02:06 ◼   ► seven days well now as soon as I had my [TS]

00:02:09 ◼   ► uptime was the space between security [TS]

00:02:10 ◼   ► updates to Mac OS 10 the ones that made [TS]

00:02:12 ◼   ► you restart [TS]

00:02:14 ◼   ► alright let's go do it follow up I was [TS]

00:02:19 ◼   ► listening to myself in hypercritical [TS]

00:02:21 ◼   ► episode 56 and in the car this week as I [TS]

00:02:25 ◼   ► always do and was cringing as I heard [TS]

00:02:27 ◼   ► myself repeatedly used the word defer [TS]

00:02:29 ◼   ► about deferring the costs and I wasn't I [TS]

00:02:33 ◼   ► didn't mean deferring the costs as in [TS]

00:02:35 ◼   ► pushing them off in time this was in [TS]

00:02:38 ◼   ► connection to Apple's data centers and [TS]

00:02:41 ◼   ► the cost of running them the cost of [TS]

00:02:43 ◼   ► running iCloud and the developers using [TS]

00:02:45 ◼   ► iCloud Hippias and no sooner had I [TS]

00:02:49 ◼   ► gotten out of the car and logged onto [TS]

00:02:51 ◼   ► the computer that I got a tweet from [TS]

00:02:54 ◼   ► someone Nathan heretic I hope I got that [TS]

00:02:58 ◼   ► right telling me that I should not be [TS]

00:02:59 ◼   ► saying the word defer and I said I know [TS]

00:03:01 ◼   ► I just heard myself saying in the car [TS]

00:03:02 ◼   ► was horrible I should have said either [TS]

00:03:04 ◼   ► offset or defray defray is probably the [TS]

00:03:07 ◼   ► word that was getting mapped wrong in my [TS]

00:03:08 ◼   ► brain and the point I made to him on [TS]

00:03:10 ◼   ► Twitter is this interesting that [TS]

00:03:11 ◼   ► listening to myself as soon as I said it [TS]

00:03:14 ◼   ► you can hear all that's wrong with what [TS]

00:03:15 ◼   ► is this guy talking about right but [TS]

00:03:16 ◼   ► saying it I said it like 9 times without [TS]

00:03:19 ◼   ► my brain triggering the warning you were [TS]

00:03:22 ◼   ► using the wrong word [TS]

00:03:23 ◼   ► right so I was kind of like writing and [TS]

00:03:25 ◼   ► proofreading when I'm speaking things [TS]

00:03:28 ◼   ► come out of my mouth that don't match [TS]

00:03:30 ◼   ► what my brain was thinking so I [TS]

00:03:32 ◼   ► apologize for that and thanks for the [TS]

00:03:33 ◼   ► correction Nathan a correction from Mark [TS]

00:03:38 ◼   ► Hofman about notifications in [TS]

00:03:40 ◼   ► Notification Center in Mountain Lion [TS]

00:03:42 ◼   ► we're just asking how that was one of [TS]

00:03:46 ◼   ► the api's that was getting swept up into [TS]

00:03:48 ◼   ► the group of things that are only [TS]

00:03:51 ◼   ► available to Mac developers who put [TS]

00:03:53 ◼   ► their application on the Mac App Store [TS]

00:03:54 ◼   ► and I was saying how that was weird [TS]

00:03:56 ◼   ► because yes notifications can be [TS]

00:03:59 ◼   ► synchronized through the cloud from one [TS]

00:04:01 ◼   ► device to the other and show up on your [TS]

00:04:02 ◼   ► desktop for an event triggered on your [TS]

00:04:05 ◼   ► phone and vice versa but notifications [TS]

00:04:08 ◼   ► also work locally if you just have a [TS]

00:04:09 ◼   ► local Mac application that wants to put [TS]

00:04:11 ◼   ► up a little notification dialog says hey [TS]

00:04:14 ◼   ► something happened over here take a look [TS]

00:04:15 ◼   ► and they want to do something more than [TS]

00:04:17 ◼   ► just have a dock icon badge or bounce or [TS]

00:04:19 ◼   ► something and Notification Center is for [TS]

00:04:23 ◼   ► that as well I said boy it's a shame [TS]

00:04:24 ◼   ► that those guys that you can't [TS]

00:04:27 ◼   ► use notification center for that yeah if [TS]

00:04:29 ◼   ► you're not in the Mac App Store even [TS]

00:04:31 ◼   ► though it has nothing to do with that [TS]

00:04:32 ◼   ► cloud well it turns out that if you do [TS]

00:04:35 ◼   ► only local notifications you can use [TS]

00:04:37 ◼   ► Notification Center in Mountain Lion [TS]

00:04:39 ◼   ► even if you're not on the Mac App Store [TS]

00:04:40 ◼   ► so the only thing that you're barred [TS]

00:04:42 ◼   ► from is so-called push notifications [TS]

00:04:45 ◼   ► notifications that are synchronized [TS]

00:04:45 ◼   ► through Apple's servers so that's good [TS]

00:04:48 ◼   ► to know [TS]

00:04:52 ◼   ► Andrew Lawrence was the first person to [TS]

00:04:54 ◼   ► chime in to tell me that the NetApp [TS]

00:04:57 ◼   ► Oracle lawsuit that I mentioned as a [TS]

00:04:59 ◼   ► possible reason that Apple bailed on ZFS [TS]

00:05:01 ◼   ► was settled shortly after son was [TS]

00:05:04 ◼   ► acquired by Oracle I put a link to the [TS]

00:05:07 ◼   ► press release in the show notes it was [TS]

00:05:09 ◼   ► actually September 2010 that was still [TS]

00:05:11 ◼   ► after long after Apple had decided had [TS]

00:05:15 ◼   ► made his decision on ZFS oh I'm not sure [TS]

00:05:17 ◼   ► settling helped to hurt one with the [TS]

00:05:19 ◼   ► other but by then it was too late but [TS]

00:05:20 ◼   ► it's a good point that it actually has [TS]

00:05:22 ◼   ► been settled and it's not dragging on [TS]

00:05:23 ◼   ► the settlement was one of those ones [TS]

00:05:24 ◼   ► where it's you know dismissed without [TS]

00:05:27 ◼   ► prejudice and the terms of the agreement [TS]

00:05:29 ◼   ► are confidential so we don't know what [TS]

00:05:30 ◼   ► actually happened they just probably [TS]

00:05:31 ◼   ► exchanged a sufficient amount of money [TS]

00:05:33 ◼   ► between parties and everything's all [TS]

00:05:35 ◼   ► better JP I don't know I just have JP [TS]

00:05:40 ◼   ► maybe that's all I could get out of his [TS]

00:05:41 ◼   ► Twitter handle or email or something was [TS]

00:05:44 ◼   ► the first to point out that btrfs is [TS]

00:05:46 ◼   ► pronounced butter FS I had heard that [TS]

00:05:49 ◼   ► enunciation but wasn't aware it was the [TS]

00:05:50 ◼   ► official apparently it is the official [TS]

00:05:52 ◼   ► pronunciation I don't I think I don't [TS]

00:05:54 ◼   ► really like the idea of food mixing with [TS]

00:05:56 ◼   ► my file systems so butter if that sounds [TS]

00:05:58 ◼   ► a little weird to me but if it's butter [TS]

00:06:00 ◼   ► FS then it's butter FS so I will [TS]

00:06:02 ◼   ► endeavour to say butter FS from now on [TS]

00:06:08 ◼   ► JP has some other information he [TS]

00:06:12 ◼   ► apparently this is why he just says JP [TS]

00:06:14 ◼   ► hey he worked in the Genius Bar for a [TS]

00:06:16 ◼   ► while in an Apple Store and here's a [TS]

00:06:19 ◼   ► quoting from his message we got almost [TS]

00:06:21 ◼   ► as many okay hard drives according to [TS]

00:06:23 ◼   ► smart status which were unrepairable in [TS]

00:06:25 ◼   ► Disk Utility as we did actual fail to [TS]

00:06:27 ◼   ► harden so what he's what he's saying is [TS]

00:06:29 ◼   ► that you know people would come in to [TS]

00:06:30 ◼   ► the Genius Bar and they would have a [TS]

00:06:31 ◼   ► problem with their hard drive and half [TS]

00:06:33 ◼   ► the time there was a hardware problem [TS]

00:06:35 ◼   ► with the hard drive but the other half [TS]

00:06:37 ◼   ► of the time it was just hfs+ corruption [TS]

00:06:39 ◼   ► so [TS]

00:06:40 ◼   ► fully 50% of their hard drive related [TS]

00:06:43 ◼   ► problems were not due to hardware but [TS]

00:06:45 ◼   ► were due to HFS corruption and he says [TS]

00:06:47 ◼   ► all in all hard drive issues accounted [TS]

00:06:49 ◼   ► for about 30% of all issues so about 15% [TS]

00:06:52 ◼   ► for a directory failure so he's saying [TS]

00:06:54 ◼   ► 15% of all problems that come to the [TS]

00:06:56 ◼   ► Genius Bar and his experience were due [TS]

00:06:58 ◼   ► to hfs+ corruption and across all of [TS]

00:07:01 ◼   ► apples customers this is a pretty big [TS]

00:07:03 ◼   ► number I'm actually surprised that I'm [TS]

00:07:06 ◼   ► surprised that the 30% of all Genius Bar [TS]

00:07:08 ◼   ► t-shirts were hard drives a hard drive [TS]

00:07:11 ◼   ► related but it's really said that half [TS]

00:07:13 ◼   ► of the HOS 15% of all possible Guinness [TS]

00:07:16 ◼   ► bar things are due to HFS+ corruption so [TS]

00:07:18 ◼   ► it shows that this is really is a [TS]

00:07:19 ◼   ► problem an apple I mean apples gotta [TS]

00:07:20 ◼   ► track these stats and has to know geez [TS]

00:07:22 ◼   ► you know 50 percent of the people come [TS]

00:07:24 ◼   ► to the Genius Bar is due to bugs in our [TS]

00:07:26 ◼   ► own software we really need to address [TS]

00:07:28 ◼   ► that filesystem thing a couple people I [TS]

00:07:32 ◼   ► don't remember who sent this might have [TS]

00:07:34 ◼   ► been the chat room but I don't have a [TS]

00:07:35 ◼   ► attribution for it send me a link to an [TS]

00:07:37 ◼   ► lwn net article called a short history [TS]

00:07:40 ◼   ► of butter FS and that I hadn't known too [TS]

00:07:45 ◼   ► much of a butter of has except that it [TS]

00:07:46 ◼   ► was the kind of like a you know the GPL [TS]

00:07:49 ◼   ► Linux directed ZFS like filesystem and I [TS]

00:07:55 ◼   ► also knew that it wasn't quite done yet [TS]

00:07:56 ◼   ► so both of those are true but this this [TS]

00:07:59 ◼   ► article that it linked in the show notes [TS]

00:08:01 ◼   ► is written by someone who worked [TS]

00:08:03 ◼   ► extensively on ZFS and it goes through [TS]

00:08:05 ◼   ► the history of how butter FS came to be [TS]

00:08:07 ◼   ► and what's different about it than ZFS I [TS]

00:08:12 ◼   ► think it's worth I guess all worth [TS]

00:08:14 ◼   ► hitting the highlights of instead of [TS]

00:08:16 ◼   ► just saying go read this article because [TS]

00:08:17 ◼   ► it is a long article and maybe if you're [TS]

00:08:18 ◼   ► not really into this you won't read the [TS]

00:08:20 ◼   ► whole darn thing [TS]

00:08:22 ◼   ► so the be actually the be TR part in [TS]

00:08:26 ◼   ► butter FS stands for be trees it's [TS]

00:08:29 ◼   ► shorter be trees have been so it's [TS]

00:08:30 ◼   ► another victim of the great valve sort [TS]

00:08:32 ◼   ► shortage of 1972 which is stricken the [TS]

00:08:35 ◼   ► UNIX landscape and Keaton used to be a [TS]

00:08:36 ◼   ► problem they didn't even have an e [TS]

00:08:39 ◼   ► leftover for the create system call [TS]

00:08:41 ◼   ► that's how bad it was [TS]

00:08:42 ◼   ► but apparently it's still going on so a [TS]

00:08:47 ◼   ► file systems use b-trees because they're [TS]

00:08:51 ◼   ► an efficient data structure for looking [TS]

00:08:53 ◼   ► stuff up and [TS]

00:08:54 ◼   ► starting stuff and deleting stuff if you [TS]

00:08:55 ◼   ► take any computer science course you'll [TS]

00:08:57 ◼   ► eventually do a chapter on trees and [TS]

00:08:58 ◼   ► they're all different kinds of trees and [TS]

00:09:00 ◼   ► you can see in the diagram of them it's [TS]

00:09:01 ◼   ► like a circle with a bunch of lines [TS]

00:09:02 ◼   ► connecting and there's you know out of [TS]

00:09:04 ◼   ► each little circle there are you know [TS]

00:09:07 ◼   ► one or more lines and then there are [TS]

00:09:09 ◼   ► circles connected those and it just [TS]

00:09:10 ◼   ► makes a big tree structure bee trees are [TS]

00:09:13 ◼   ► a particular kind of tree that you know [TS]

00:09:16 ◼   ► the characteristics of trees are how [TS]

00:09:17 ◼   ► many how many different branches come [TS]

00:09:19 ◼   ► out of each one how do you insert [TS]

00:09:21 ◼   ► something into the tree how do you [TS]

00:09:22 ◼   ► remove something from the tree I'm not [TS]

00:09:24 ◼   ► going to go into the sto details of bee [TS]

00:09:26 ◼   ► trees in particular but suffice it to [TS]

00:09:28 ◼   ► say that B trees are specific variant of [TS]

00:09:31 ◼   ► B trees are among the most efficient [TS]

00:09:33 ◼   ► general-purpose tree structure for the [TS]

00:09:36 ◼   ► you know they do everything pretty well [TS]

00:09:37 ◼   ► you can insert things pretty quickly you [TS]

00:09:39 ◼   ► can delete things pretty quickly they [TS]

00:09:40 ◼   ► don't get unbalanced they don't require [TS]

00:09:43 ◼   ► a lot of maintenance to keep balanced [TS]

00:09:44 ◼   ► ever it's kind of like a good middle [TS]

00:09:46 ◼   ► ground for the types of things that you [TS]

00:09:47 ◼   ► might want to keep track of in a file [TS]

00:09:50 ◼   ► system you do because you don't want to [TS]

00:09:51 ◼   ► be something to be really quick to read [TS]

00:09:53 ◼   ► from but really slow to write because [TS]

00:09:54 ◼   ► file systems tend to do all that stuff [TS]

00:09:56 ◼   ► read write and delete in and generally [TS]

00:09:58 ◼   ► you know equal measures but B trees are [TS]

00:10:04 ◼   ► not particularly good when it comes to [TS]

00:10:06 ◼   ► the strategy the ZFS implements where [TS]

00:10:08 ◼   ► they never overwrite data in place you [TS]

00:10:11 ◼   ► know that's the this is the [TS]

00:10:12 ◼   ► copy-on-write strategy where if you have [TS]

00:10:14 ◼   ► new data to write write in to a new [TS]

00:10:16 ◼   ► location [TS]

00:10:17 ◼   ► rather than writing over on top of the [TS]

00:10:18 ◼   ► open and if you want a mod that's to put [TS]

00:10:20 ◼   ► the copy-on-write means and saying if [TS]

00:10:21 ◼   ► it's time for you to write some data [TS]

00:10:23 ◼   ► instead of overwriting the data find [TS]

00:10:25 ◼   ► where that data is and then make a new [TS]

00:10:27 ◼   ► copy of it somewhere else it's different [TS]

00:10:29 ◼   ► than the old one but B trees are not [TS]

00:10:32 ◼   ► compatible with that at all because if [TS]

00:10:34 ◼   ► you were to naively implement copy and [TS]

00:10:35 ◼   ► write with the B tree then anytime you [TS]

00:10:37 ◼   ► changed anything the entire tree from [TS]

00:10:39 ◼   ► top to bottom would have to be changed [TS]

00:10:40 ◼   ► because the changes would propagate back [TS]

00:10:43 ◼   ► up the tree because you're making a new [TS]

00:10:44 ◼   ► instance so almost all B trees and file [TS]

00:10:46 ◼   ► systems and by the way HFS+ and H of s [TS]

00:10:49 ◼   ► use B trees extensively B trees and file [TS]

00:10:52 ◼   ► systems tend to be updated in place [TS]

00:10:53 ◼   ► because that preserves all the desirable [TS]

00:10:55 ◼   ► performance characteristics of a B tree [TS]

00:10:57 ◼   ► and once you start saying oh we can't [TS]

00:10:58 ◼   ► update anything in place we always had [TS]

00:10:59 ◼   ► to write a new dated a new location and [TS]

00:11:01 ◼   ► B trees seemed useless so many years ago [TS]

00:11:05 ◼   ► someone named Oh had Road [TS]

00:11:07 ◼   ► a I don't know if I got this name right [TS]

00:11:08 ◼   ► OHA D ro deh wrote an academic paper as [TS]

00:11:12 ◼   ► people in academia tend to do about a [TS]

00:11:15 ◼   ► new kind of bee tree that was friendly [TS]

00:11:18 ◼   ► to copy-on-write he found a way that if [TS]

00:11:20 ◼   ► you all you change bee trees in this way [TS]

00:11:21 ◼   ► then we actually can get reasonably good [TS]

00:11:23 ◼   ► performance characteristics while still [TS]

00:11:25 ◼   ► not ever updating data in place and this [TS]

00:11:29 ◼   ► academic paper probably sat around for a [TS]

00:11:30 ◼   ► while until someone named Chris Mason [TS]

00:11:32 ◼   ► found this and said hey if I have a bee [TS]

00:11:36 ◼   ► tree structure that's also friendly - [TS]

00:11:37 ◼   ► copy-on-write maybe I can use that in a [TS]

00:11:40 ◼   ► new file system and his big insight for [TS]

00:11:44 ◼   ► creating a new file system was to decide [TS]

00:11:47 ◼   ► that everything in the file system will [TS]

00:11:50 ◼   ► be stored in one of these copy-on-write [TS]

00:11:51 ◼   ► friendly bee trees normally you have bee [TS]

00:11:54 ◼   ► trees first restoring like oh I'll keep [TS]

00:11:56 ◼   ► track of directory entries with the bee [TS]

00:11:59 ◼   ► tree but when I'll have a separate data [TS]

00:12:00 ◼   ► structure for keeping track of free [TS]

00:12:02 ◼   ► space like a bitmap or something and [TS]

00:12:03 ◼   ► I'll have a separate structure for [TS]

00:12:04 ◼   ► keeping track of extensors you know it [TS]

00:12:08 ◼   ► there are usually different kinds of [TS]

00:12:09 ◼   ► data structures tailored for each [TS]

00:12:10 ◼   ► purpose he decided well this data [TS]

00:12:12 ◼   ► structure looks so great what if [TS]

00:12:13 ◼   ► everything is stored in one of these [TS]

00:12:15 ◼   ► copy IRB trees I'll star the directory [TS]

00:12:17 ◼   ► entries the the free space lists the [TS]

00:12:19 ◼   ► extents that the file data itself [TS]

00:12:22 ◼   ► everything in one of these bee trees and [TS]

00:12:23 ◼   ► that's a nice unifying concept because [TS]

00:12:27 ◼   ► it means that you only really need to [TS]

00:12:28 ◼   ► have one code path once you get your [TS]

00:12:31 ◼   ► code for reading writing and modifying [TS]

00:12:32 ◼   ► these bee trees like down pat and [TS]

00:12:34 ◼   ► debugged you're all set you don't have [TS]

00:12:36 ◼   ► to say okay well what about the code [TS]

00:12:37 ◼   ► that manages the free space and what [TS]

00:12:38 ◼   ► about the code that manages the direct [TS]

00:12:40 ◼   ► Rangers well you not not 17 different [TS]

00:12:42 ◼   ► algorithms and 17 different data [TS]

00:12:43 ◼   ► structure just one thing and you know so [TS]

00:12:45 ◼   ► the code is simpler smaller and a nicely [TS]

00:12:48 ◼   ► uniform and programmers love this type [TS]

00:12:49 ◼   ► of thing so that's the basis of btrfs [TS]

00:12:53 ◼   ► building an entire file system on this [TS]

00:12:55 ◼   ► this new data structure and that's very [TS]

00:12:59 ◼   ► interesting to me I didn't I didn't know [TS]

00:13:01 ◼   ► this before I had read it definitely [TS]

00:13:02 ◼   ► hits all of my buttons of her computer [TS]

00:13:06 ◼   ► science and programmers like oh it's [TS]

00:13:08 ◼   ► beautiful and elegant and has great [TS]

00:13:09 ◼   ► performance characteristics and has all [TS]

00:13:10 ◼   ► these features of ZFS the article goes [TS]

00:13:12 ◼   ► into much more detail about how ZFS and [TS]

00:13:15 ◼   ► btrfs [TS]

00:13:16 ◼   ► are the same and how they differ they [TS]

00:13:19 ◼   ► both kind of arrived similar [TS]

00:13:20 ◼   ► from different directions and they do [TS]

00:13:23 ◼   ► have different advantages and [TS]

00:13:24 ◼   ► disadvantages or asset each other so I [TS]

00:13:25 ◼   ► encourage everyone to actually follow [TS]

00:13:26 ◼   ► that link and read it is not that long [TS]

00:13:28 ◼   ► and it's not too jargon inducing it's a [TS]

00:13:30 ◼   ► short history of BTR oh I almost added a [TS]

00:13:32 ◼   ► butter fs and we will be revisiting this [TS]

00:13:37 ◼   ► topic in a bit once we get through the [TS]

00:13:39 ◼   ► official follow up Lenny P Robert writes [TS]

00:13:44 ◼   ► to tell me that there is a program that [TS]

00:13:48 ◼   ► will look through your HFS+ volume and [TS]

00:13:54 ◼   ► try to detect if there are any errors [TS]

00:13:57 ◼   ► and how does it do this it check sums [TS]

00:14:01 ◼   ► everything and writes a bunch of little [TS]

00:14:03 ◼   ► checksum files to each directory and [TS]

00:14:05 ◼   ► then the next time you ask the check it [TS]

00:14:07 ◼   ► says ok I have this file here this has [TS]

00:14:09 ◼   ► the checksum supposed to be blah [TS]

00:14:10 ◼   ► I will recheck some all these files and [TS]

00:14:12 ◼   ► see if it matches a lot of people were [TS]

00:14:15 ◼   ► asking me hey I'm afraid that my data is [TS]

00:14:17 ◼   ► going bad on my HFS+ volumes how do I [TS]

00:14:19 ◼   ► tell and I did I had wasn't aware of [TS]

00:14:24 ◼   ► this program before but other people [TS]

00:14:26 ◼   ► wrote in with it so I replied so this is [TS]

00:14:28 ◼   ► one of the tools available to you now if [TS]

00:14:29 ◼   ► you think about it for too long you [TS]

00:14:31 ◼   ► realize that this program which is [TS]

00:14:32 ◼   ► called integrity checker by the way it's [TS]

00:14:36 ◼   ► kind of in a catch-22 because it's [TS]

00:14:41 ◼   ► storing the check sums on the same disk [TS]

00:14:44 ◼   ► that you're not trusting to make sure [TS]

00:14:48 ◼   ► your data is ok you know what I mean so [TS]

00:14:50 ◼   ► if you have a bit error and it happens [TS]

00:14:53 ◼   ► to be an error in the checksum file then [TS]

00:14:57 ◼   ► it would look like all your data is [TS]

00:14:57 ◼   ► corrupted and really it's the checksum [TS]

00:14:59 ◼   ► that's corrupted so the very least this [TS]

00:15:01 ◼   ► might tell you something is wrong what [TS]

00:15:04 ◼   ► exactly is wrong what you can do but I [TS]

00:15:06 ◼   ► don't know but you could use this for [TS]

00:15:08 ◼   ► example to keep proving to yourself that [TS]

00:15:10 ◼   ► your backups are the same as they were [TS]

00:15:12 ◼   ► the last time you know like I haven't [TS]

00:15:15 ◼   ► modified anything let me look at my time [TS]

00:15:16 ◼   ► machine back up as of three months ago [TS]

00:15:18 ◼   ► is that data still exactly what it was [TS]

00:15:20 ◼   ► when I wrote it three months ago [TS]

00:15:22 ◼   ► or is there something wrong with it when [TS]

00:15:24 ◼   ► there is something wrong you may not be [TS]

00:15:25 ◼   ► able to tell exactly what's wrong but at [TS]

00:15:26 ◼   ► least you know your data is bad so I [TS]

00:15:29 ◼   ► have a link to this product in the show [TS]

00:15:30 ◼   ► notes I have not tried it myself I don't [TS]

00:15:32 ◼   ► even know how much it costs [TS]

00:15:34 ◼   ► but for people who are paranoid about [TS]

00:15:36 ◼   ► data integrity it's it's seems that [TS]

00:15:39 ◼   ► third parties are taking up the mantle [TS]

00:15:40 ◼   ► of attempting to do something about this [TS]

00:15:44 ◼   ► problem as you might imagine anything [TS]

00:15:47 ◼   ► that's going to grind over your whole [TS]

00:15:48 ◼   ► disk and checksum every single thing and [TS]

00:15:50 ◼   ► check their the check sums are the same [TS]

00:15:51 ◼   ► it's going to be tremendously i/o [TS]

00:15:53 ◼   ► intensive and it probably takes a really [TS]

00:15:55 ◼   ► long time that's what you get when you [TS]

00:15:58 ◼   ► have to do it on top of an existing file [TS]

00:16:00 ◼   ► system instead of having it built in and [TS]

00:16:05 ◼   ► Gregory Williams pointed me to another [TS]

00:16:08 ◼   ► third-party product this one allows you [TS]

00:16:13 ◼   ► to tap into HFS+ native compression that [TS]

00:16:17 ◼   ► was introduced in Snow Leopard I've [TS]

00:16:18 ◼   ► talked about in last show how there was [TS]

00:16:20 ◼   ► no like UI for that in the in the OS and [TS]

00:16:24 ◼   ► Apple decided to compress a whole bunch [TS]

00:16:26 ◼   ► of files sort of out of the box but you [TS]

00:16:27 ◼   ► didn't really have any control over that [TS]

00:16:29 ◼   ► well this is a preference pane called [TS]

00:16:31 ◼   ► clusters from late night software and [TS]

00:16:35 ◼   ► with it you can tell it hey any files [TS]

00:16:39 ◼   ► under a particular folder flip that [TS]

00:16:41 ◼   ► little HFS+ compression bit and compress [TS]

00:16:42 ◼   ► them for you and it's complete [TS]

00:16:43 ◼   ► transparent encryption you don't know [TS]

00:16:44 ◼   ► they're encrypted there's no like [TS]

00:16:47 ◼   ► decryption phase that you can see it all [TS]

00:16:49 ◼   ► happens under the covers in the file [TS]

00:16:50 ◼   ► system so you can save some space with [TS]

00:16:53 ◼   ► this now don't turn this on for a folder [TS]

00:16:55 ◼   ► full of QuickTime movies or something [TS]

00:16:56 ◼   ► because they're already compressed and [TS]

00:16:58 ◼   ► you're not going to get much more out of [TS]

00:16:59 ◼   ► them if you have some easily [TS]

00:17:00 ◼   ► compressible data and you would like to [TS]

00:17:01 ◼   ► have it transparently compressed for you [TS]

00:17:03 ◼   ► this may be the ticket on the other hand [TS]

00:17:07 ◼   ► it's possible that there's a good reason [TS]

00:17:10 ◼   ► that Apple has not exposed the HFS+ [TS]

00:17:13 ◼   ► compression in the user interface so I [TS]

00:17:17 ◼   ► think this type of software is the type [TS]

00:17:19 ◼   ► of thing that exists on the borderline [TS]

00:17:20 ◼   ► between what Apple what Apple's [TS]

00:17:22 ◼   ► operating system is capable of and what [TS]

00:17:24 ◼   ► they recommend third-party developers [TS]

00:17:25 ◼   ► use and we will touch on that again [TS]

00:17:27 ◼   ► actually later in the show and I also [TS]

00:17:32 ◼   ► put a link in the show notes to the [TS]

00:17:34 ◼   ► section about hfs+ compression in my [TS]

00:17:37 ◼   ► snow leopard review from 2009 alright so [TS]

00:17:42 ◼   ► that's the end of the official follow-up [TS]

00:17:43 ◼   ► and now we begin the unofficial fall off [TS]

00:17:46 ◼   ► which is the main topic of the show [TS]

00:17:47 ◼   ► which is more [TS]

00:17:48 ◼   ► files isms because believe it or not the [TS]

00:17:50 ◼   ► end of the last show I thought of 20 [TS]

00:17:51 ◼   ► things that I didn't talk about related [TS]

00:17:54 ◼   ► to file systems and a lot of people [TS]

00:17:55 ◼   ► asked about them hmm well let's do our [TS]

00:17:57 ◼   ► first sponsor and then we'll kick that [TS]

00:17:58 ◼   ► part off what do you think sounds good [TS]

00:18:00 ◼   ► our first sponsor is a longtime sponsor [TS]

00:18:02 ◼   ► shopify calm love these guys [TS]

00:18:05 ◼   ► very simply I've told you about them [TS]

00:18:07 ◼   ► before create your online store today [TS]

00:18:09 ◼   ► with Shopify they make it super simple [TS]

00:18:12 ◼   ► to make your own store and you can sell [TS]

00:18:14 ◼   ► whatever you want one of the questions [TS]

00:18:16 ◼   ► that I had people were asking me about [TS]

00:18:17 ◼   ► this so well that's perfectly fine if [TS]

00:18:19 ◼   ► you're selling you know physical goods [TS]

00:18:20 ◼   ► if you're selling a t-shirt or you're [TS]

00:18:22 ◼   ► selling electronics or you have a [TS]

00:18:24 ◼   ► Kickstarter project that you know you [TS]

00:18:26 ◼   ► need the storefront for it what if I [TS]

00:18:28 ◼   ► want to sell something electronic what [TS]

00:18:29 ◼   ► if I wrote a book you know and I want to [TS]

00:18:32 ◼   ► sell the PDF of it or what if I have [TS]

00:18:34 ◼   ► something that I want to control down [TS]

00:18:35 ◼   ► yes of course you can do that with [TS]

00:18:38 ◼   ► Shopify dot-com there are tons of [TS]

00:18:39 ◼   ► add-ons and tons of little things that [TS]

00:18:41 ◼   ► that people have written it for exactly [TS]

00:18:43 ◼   ► this kind of thing in fact my friend [TS]

00:18:46 ◼   ► Jeffrey's elderman they have a company [TS]

00:18:48 ◼   ► called a book apart and all of the books [TS]

00:18:51 ◼   ► that they do for designers and [TS]

00:18:52 ◼   ► developers [TS]

00:18:52 ◼   ► that's a Shopify store and they [TS]

00:18:54 ◼   ► customized the heck out of it you [TS]

00:18:56 ◼   ► wouldn't even recognize it [TS]

00:18:57 ◼   ► it's totally so here's the deal normally [TS]

00:19:00 ◼   ► you go in there and you sign up on [TS]

00:19:02 ◼   ► Shopify com you get 30 days free use the [TS]

00:19:04 ◼   ► coupon code 5x5 when you're signing up [TS]

00:19:07 ◼   ► you'll get three months free and of [TS]

00:19:09 ◼   ► course you can use the site and sell [TS]

00:19:11 ◼   ► everything you want within that that [TS]

00:19:13 ◼   ► time frame of course you can so go there [TS]

00:19:16 ◼   ► today Shopify com coupon code 5x5 and [TS]

00:19:19 ◼   ► join more than 20,000 stores selling [TS]

00:19:22 ◼   ► their stuff on Shopify John himself [TS]

00:19:25 ◼   ► you're even opening up a store I am NOT [TS]

00:19:29 ◼   ► gonna be selling USB headsets [TS]

00:19:35 ◼   ► all right already main topic yes the [TS]

00:19:39 ◼   ► main topic is actually inspired by [TS]

00:19:40 ◼   ► follow-up and you mentioned at the very [TS]

00:19:41 ◼   ► top of the show because you've been [TS]

00:19:43 ◼   ► seeing the same emails as I have [TS]

00:19:44 ◼   ► everybody was asking in the last show [TS]

00:19:46 ◼   ► all you know so I listened your show [TS]

00:19:47 ◼   ► about file systems how does all this [TS]

00:19:50 ◼   ► change in light of solid state disks [TS]

00:19:52 ◼   ► that is the question of the week ah and [TS]

00:19:57 ◼   ► I didn't talk about it in last show so [TS]

00:19:59 ◼   ► we'll talk about it now I'll use it as a [TS]

00:20:00 ◼   ► jumping-off point to talk more about [TS]

00:20:01 ◼   ► these file systems in general one of the [TS]

00:20:05 ◼   ► reasons I didn't talk about it is [TS]

00:20:06 ◼   ► because there's no clear answer how does [TS]

00:20:11 ◼   ► it change and it's not even we can't [TS]

00:20:13 ◼   ► even say it changes completely or not at [TS]

00:20:15 ◼   ► all it's somewhere in the middle so I'm [TS]

00:20:17 ◼   ► trying to think of things that file [TS]

00:20:23 ◼   ► systems do that clearly have spinning [TS]

00:20:28 ◼   ► disks in mind as a way to get into the [TS]

00:20:32 ◼   ► topic of how that might change with SSDs [TS]

00:20:34 ◼   ► so the biggest one I could think of them [TS]

00:20:36 ◼   ► are you going to go some more is putting [TS]

00:20:40 ◼   ► related information close together on [TS]

00:20:42 ◼   ► the disk [TS]

00:20:43 ◼   ► because the big thing you want to avoid [TS]

00:20:44 ◼   ► on a spinning disk is moving the disk [TS]

00:20:47 ◼   ► head that is by far the most time [TS]

00:20:49 ◼   ► consuming thing you have to do because [TS]

00:20:51 ◼   ► especially with you know diskette disks [TS]

00:20:54 ◼   ► are getting faster but the speed with [TS]

00:20:55 ◼   ► which you can move a disk head from one [TS]

00:20:57 ◼   ► place to another on a disk has not been [TS]

00:20:59 ◼   ► getting faster nearly as much as the [TS]

00:21:01 ◼   ► size of disks have been increasing you [TS]

00:21:03 ◼   ► know because you have to move the little [TS]

00:21:05 ◼   ► head and then wait for it to stop it [TS]

00:21:07 ◼   ► exactly over some microscopic little [TS]

00:21:08 ◼   ► track on the disk and then wait for it [TS]

00:21:10 ◼   ► to stop wiggling back and forth and sort [TS]

00:21:12 ◼   ► of settle down so we can read again and [TS]

00:21:14 ◼   ► they can do that amazingly fast it's [TS]

00:21:16 ◼   ► like a seek times like nine milliseconds [TS]

00:21:18 ◼   ► or something like that but it's still [TS]

00:21:21 ◼   ► super slow compared to how fast they can [TS]

00:21:23 ◼   ► stream data off a single location on [TS]

00:21:25 ◼   ► disk so putting stuff together on a [TS]

00:21:28 ◼   ► spinning disk is really important you [TS]

00:21:31 ◼   ► can see this yourself I think regular [TS]

00:21:32 ◼   ► users have some idea about this when you [TS]

00:21:35 ◼   ► have a like a 100 megabyte folder and [TS]

00:21:39 ◼   ► but it's filled with thousands and [TS]

00:21:41 ◼   ► thousands of files there are a few bytes [TS]

00:21:42 ◼   ► each how long does it take to duplicate [TS]

00:21:45 ◼   ► that versus how long does it take [TS]

00:21:47 ◼   ► duplicate a single hundred megabyte [TS]

00:21:48 ◼   ► phile if your disc is not full and you [TS]

00:21:50 ◼   ► have lots of free space duplicating a [TS]

00:21:51 ◼   ► single hundred megabyte file can be [TS]

00:21:53 ◼   ► really fast amazingly fast basically if [TS]

00:21:55 ◼   ► the hundred megabyte file is kind of all [TS]

00:21:57 ◼   ► all the data of it is kind of in this [TS]

00:22:00 ◼   ► similar place on the disk you don't need [TS]

00:22:01 ◼   ► to do many seeks you read long sequences [TS]

00:22:03 ◼   ► of bytes off of the disk before you have [TS]

00:22:05 ◼   ► to move the disk head to get the rest of [TS]

00:22:07 ◼   ► the bytes that's really fast but moving [TS]

00:22:10 ◼   ► thousands and thousands of little files [TS]

00:22:12 ◼   ► even the total amount of data is the [TS]

00:22:14 ◼   ► same to get it thousands of thousands [TS]

00:22:15 ◼   ► little files well you have to go read [TS]

00:22:17 ◼   ► the directory entry and that's imagine [TS]

00:22:18 ◼   ► this is hfs+ to go to go to the catalog [TS]

00:22:20 ◼   ► file which is in a central location so [TS]

00:22:21 ◼   ► you read the directory entry and then [TS]

00:22:22 ◼   ► the catalog file points you'd okay [TS]

00:22:23 ◼   ► here's the first file and here's what [TS]

00:22:25 ◼   ► it's called and here's what the bytes [TS]

00:22:26 ◼   ► are so move this can get the bytes for [TS]

00:22:28 ◼   ► that file and that's really quick [TS]

00:22:29 ◼   ► because there's only a few bytes and [TS]

00:22:30 ◼   ► then write the file to the new location [TS]

00:22:32 ◼   ► so make a new directory entry that's [TS]

00:22:33 ◼   ► back to the catalog file and write the [TS]

00:22:35 ◼   ► name of the file there and then write [TS]

00:22:36 ◼   ► the bytes for the file okay let's write [TS]

00:22:38 ◼   ► those flights with a file someplace else [TS]

00:22:39 ◼   ► and dusk you've done like six seeks and [TS]

00:22:40 ◼   ► all you've copied is like two bytes of [TS]

00:22:42 ◼   ► data like if this was a tiny little file [TS]

00:22:44 ◼   ► in fact most your time might have been [TS]

00:22:47 ◼   ► spent just writing the metadata about [TS]

00:22:49 ◼   ► the files name the date you know all [TS]

00:22:50 ◼   ► these other other information but you're [TS]

00:22:52 ◼   ► seeking all over the disk get a little [TS]

00:22:54 ◼   ► bit of data to go back to the catalog [TS]

00:22:56 ◼   ► file write the new location write the [TS]

00:22:57 ◼   ► data it takes forever because seeking [TS]

00:22:59 ◼   ► time starts to dominate how long the [TS]

00:23:02 ◼   ► entire operation takes whereas the [TS]

00:23:03 ◼   ► actual writing data to disk when you [TS]

00:23:05 ◼   ► look at the stats on a disk like into a [TS]

00:23:06 ◼   ► 30 megabytes per second or 40 megabytes [TS]

00:23:08 ◼   ► per second yeah that's how much you can [TS]

00:23:10 ◼   ► write if the disk head is pretty much [TS]

00:23:12 ◼   ► not moving and the disk is spinning [TS]

00:23:13 ◼   ► underneath it once you start moving the [TS]

00:23:15 ◼   ► diskette around you're spending all your [TS]

00:23:17 ◼   ► time moving and very little time [TS]

00:23:18 ◼   ► actually writing data so the big thing [TS]

00:23:21 ◼   ► that file systems try to do in in the at [TS]

00:23:24 ◼   ► all times but especially in the modern [TS]

00:23:26 ◼   ► age is no disk seeks put related stuff [TS]

00:23:28 ◼   ► close together pack it all next to each [TS]

00:23:29 ◼   ► other don't make the disk ad move around [TS]

00:23:32 ◼   ► so how does that relate to SSDs the big [TS]

00:23:37 ◼   ► thing with us is these is you know [TS]

00:23:38 ◼   ► there's no moving parts there's no disk [TS]

00:23:40 ◼   ► kid moving back and forth trying to grab [TS]

00:23:44 ◼   ► little pieces just it's just a bunch of [TS]

00:23:45 ◼   ► memory chips and the naive view of [TS]

00:23:47 ◼   ► memory checks is like oh I can get at [TS]

00:23:48 ◼   ► any part of those memory chips instantly [TS]

00:23:50 ◼   ► I don't have to move anything anywhere I [TS]

00:23:52 ◼   ► just say give me byte number 56 and I [TS]

00:23:54 ◼   ► get it give me byte number 3000 and I [TS]

00:23:56 ◼   ► get it and asking for byte 56 directly [TS]

00:23:58 ◼   ► after I asked for but 3000 it doesn't [TS]

00:24:00 ◼   ► make any difference right [TS]

00:24:01 ◼   ► there's no there's nothing to move well [TS]

00:24:05 ◼   ► flash memory unfortunately is not as [TS]

00:24:07 ◼   ► simple as the naive view of memory might [TS]

00:24:10 ◼   ► be there are some limitations and the [TS]

00:24:14 ◼   ► limitations are different for flash [TS]

00:24:16 ◼   ► memory of different kinds a different [TS]

00:24:17 ◼   ► for DRAM differing for all sorts of [TS]

00:24:19 ◼   ► memory chips the stuff that goes on [TS]

00:24:20 ◼   ► inside the memory chips we like to think [TS]

00:24:21 ◼   ► of it as a black box that just stores [TS]

00:24:23 ◼   ► our bits and gives us back but it's [TS]

00:24:24 ◼   ► really fiendishly complex and you don't [TS]

00:24:26 ◼   ► know how for example DRAM works I [TS]

00:24:28 ◼   ► encourage you to go to Wikipedia and [TS]

00:24:30 ◼   ► read about how DRAM works you will [TS]

00:24:32 ◼   ► probably be horrified and amazed at any [TS]

00:24:34 ◼   ► Forks at all pieces it's terrifyingly [TS]

00:24:35 ◼   ► scary flash is similarly terrifying it [TS]

00:24:39 ◼   ► has these crazy limitations that don't [TS]

00:24:40 ◼   ► make any sense until you figure out how [TS]

00:24:42 ◼   ► like the chips are actually laid out and [TS]

00:24:43 ◼   ► how the physics work and the economics [TS]

00:24:45 ◼   ► of fabbing and why they're made this [TS]

00:24:47 ◼   ► play I'll try to give a reasonable [TS]

00:24:50 ◼   ► high-level summary of it but the short [TS]

00:24:53 ◼   ► answer is that it does help to have [TS]

00:24:57 ◼   ► things near each other when when using [TS]

00:25:01 ◼   ► flash memory because although it doesn't [TS]

00:25:04 ◼   ► take any time - there's nothing to seek [TS]

00:25:06 ◼   ► from one location to the other flash [TS]

00:25:09 ◼   ► memory cannot be arbitrarily updated in [TS]

00:25:12 ◼   ► exactly the size pieces that you want [TS]

00:25:14 ◼   ► there are larger chunks that you know [TS]

00:25:17 ◼   ► you can't just say okay I want I'm going [TS]

00:25:19 ◼   ► to write two bytes here I'm going to [TS]

00:25:20 ◼   ► write one byte there and about 1 byte [TS]

00:25:21 ◼   ► over there and everything's hunky-dory [TS]

00:25:23 ◼   ► if the flash chips themselves tend to [TS]

00:25:26 ◼   ► have to do operations in larger evenly [TS]

00:25:28 ◼   ► sized chunks than you would want so that [TS]

00:25:30 ◼   ► might mean erasing an entire block we [TS]

00:25:33 ◼   ► are reading the entire contents of [TS]

00:25:34 ◼   ► entire block erasing the whole thing and [TS]

00:25:37 ◼   ► writing it back plus the change instead [TS]

00:25:39 ◼   ► of just writing your change I tried to [TS]

00:25:42 ◼   ► highlight the one part that's relevant [TS]

00:25:43 ◼   ► most relevant from the Wikipedia page to [TS]

00:25:46 ◼   ► a NAND flash memory that's moved in most [TS]

00:25:49 ◼   ► flash drives and the upshot according to [TS]

00:25:52 ◼   ► Wikipedia and I don't know enough about [TS]

00:25:54 ◼   ► it to know whether this is accurate but [TS]

00:25:55 ◼   ► I assume it probably is because it's [TS]

00:25:57 ◼   ► just a technical matter is that it says [TS]

00:26:01 ◼   ► you can erase memory a block at a time [TS]

00:26:05 ◼   ► but so you can read it in a little [TS]

00:26:08 ◼   ► pieces and you can write it in little [TS]

00:26:10 ◼   ► pieces but you can only erase it a block [TS]

00:26:11 ◼   ► at the time and the next wrinkle to that [TS]

00:26:13 ◼   ► is that [TS]

00:26:15 ◼   ► when you erase you set all the bits but [TS]

00:26:18 ◼   ► when you set them you can only clear [TS]

00:26:19 ◼   ► bits so if you're setting something in a [TS]

00:26:22 ◼   ► region that has all ones in it all you [TS]

00:26:25 ◼   ► got to do is blank out the ones that are [TS]

00:26:26 ◼   ► zeros to make your bit patterns so if [TS]

00:26:28 ◼   ► you're writing one zero one zero and [TS]

00:26:29 ◼   ► you've got one one one one [TS]

00:26:31 ◼   ► you just blank out those two ones and [TS]

00:26:33 ◼   ► voila one zero one zero you just wrote [TS]

00:26:34 ◼   ► only the amount of data you needed ah [TS]

00:26:38 ◼   ► but when you erase it you have to erase [TS]

00:26:40 ◼   ► the entire thing the entire block and a [TS]

00:26:42 ◼   ► block is much bigger than the region [TS]

00:26:44 ◼   ► you're just interested in updating and [TS]

00:26:45 ◼   ► since you can only clear disks if a [TS]

00:26:48 ◼   ► region has more zeros than the data [TS]

00:26:49 ◼   ► you're about to write there like I can't [TS]

00:26:51 ◼   ► I can't write this because I all I can [TS]

00:26:53 ◼   ► do is change things it all I can do is [TS]

00:26:57 ◼   ► change things to zeros but it's already [TS]

00:26:58 ◼   ► got too many zeros I can't change that [TS]

00:27:00 ◼   ► zero to a one [TS]

00:27:00 ◼   ► so if I can't change a zero to one my [TS]

00:27:03 ◼   ► only option is to erase this entire [TS]

00:27:05 ◼   ► block set it all to ones we're going to [TS]

00:27:07 ◼   ► read the block first erase the entire [TS]

00:27:08 ◼   ► block and write the entire block back [TS]

00:27:10 ◼   ► plus my changes there are probably [TS]

00:27:14 ◼   ► better articles about this and someone [TS]

00:27:16 ◼   ► hasn't been the show at Stan and tech I [TS]

00:27:17 ◼   ► was trying to Google for better links to [TS]

00:27:19 ◼   ► this before I started so I will add this [TS]

00:27:21 ◼   ► one they came up in the show notes we [TS]

00:27:23 ◼   ► link to the print only version of it but [TS]

00:27:25 ◼   ► I guess I'll throw them in there so it [TS]

00:27:29 ◼   ► means the ETL details don't matter but [TS]

00:27:31 ◼   ► the point is that operations to flash [TS]

00:27:34 ◼   ► memory don't necessarily only modify the [TS]

00:27:36 ◼   ► bits that you're interested in sometimes [TS]

00:27:38 ◼   ► they have to modify entire large blocks [TS]

00:27:40 ◼   ► of bits even if you're only interested [TS]

00:27:43 ◼   ► in changing a small portion of so that [TS]

00:27:46 ◼   ► argues for trying to make your data [TS]

00:27:50 ◼   ► that's related close to each other [TS]

00:27:52 ◼   ► otherwise you're going to spend your [TS]

00:27:53 ◼   ► time erasing an entire block of data [TS]

00:27:55 ◼   ► just to put one byte there and then [TS]

00:27:57 ◼   ► erase it all totally unrelated block [TS]

00:27:59 ◼   ► about to put to put another byte there [TS]

00:28:02 ◼   ► now I'm not sure if this means that [TS]

00:28:05 ◼   ► putting stuff close together is like a [TS]

00:28:07 ◼   ► big win on flash certainly it's not as [TS]

00:28:09 ◼   ► big a win as it is on spinning disks but [TS]

00:28:12 ◼   ► at the very least it probably does no [TS]

00:28:14 ◼   ► harm and there are reasons to think that [TS]

00:28:17 ◼   ► it might be beneficial like these same [TS]

00:28:19 ◼   ► file system layouts that put things [TS]

00:28:23 ◼   ► close together would also benefit flash [TS]

00:28:25 ◼   ► memory because then [TS]

00:28:27 ◼   ► that's it do operations that modify more [TS]

00:28:31 ◼   ► of the block the whole block rhythm I [TS]

00:28:33 ◼   ► need to erase but the flipside of that [TS]

00:28:35 ◼   ► is if there is a file system that goes [TS]

00:28:37 ◼   ► through great lengths and bends over [TS]

00:28:38 ◼   ► backwards and has a really complicated [TS]

00:28:40 ◼   ► structure completely aimed at minimizing [TS]

00:28:42 ◼   ► seeks it's like all we just got to pack [TS]

00:28:44 ◼   ► everything together and we have to do [TS]

00:28:45 ◼   ► all this complicated stuff and the code [TS]

00:28:47 ◼   ► is complicated and the i/o scheduling is [TS]

00:28:49 ◼   ► complicated also it can avoid seeks it [TS]

00:28:52 ◼   ► could be that that complexity is not [TS]

00:28:55 ◼   ► worth the benefit that you get in SSDs [TS]

00:28:57 ◼   ► and a file system that doesn't go to [TS]

00:28:58 ◼   ► such great lengths to make sure data is [TS]

00:29:00 ◼   ► as close together but is much simpler [TS]

00:29:03 ◼   ► much more reliable and as fewer bugs or [TS]

00:29:05 ◼   ► whatever might be fine on an SSD because [TS]

00:29:08 ◼   ► the things that hurt it on spinning disk [TS]

00:29:10 ◼   ► don't hurt it as much there so that's [TS]

00:29:13 ◼   ► not a very good answer the answer is [TS]

00:29:14 ◼   ► basically it depends and and I don't [TS]

00:29:17 ◼   ► think anyone is really sure and all the [TS]

00:29:19 ◼   ► file systems that I've been talking [TS]

00:29:21 ◼   ► about with the possible section of [TS]

00:29:23 ◼   ► session of butter FS were conceived in a [TS]

00:29:26 ◼   ► time when people didn't realize that [TS]

00:29:28 ◼   ► SSDs would be the phenomenon that they [TS]

00:29:30 ◼   ► are as ZFS is but I think it's close to [TS]

00:29:34 ◼   ► ten years old now it's pretty old and [TS]

00:29:36 ◼   ► back then yes SSDs existed and they [TS]

00:29:39 ◼   ► certainly didn't want to do anything [TS]

00:29:40 ◼   ► that would be silly on an SSD and I [TS]

00:29:43 ◼   ► don't think they did but they wasn't [TS]

00:29:45 ◼   ► designed for an SSD many of the [TS]

00:29:47 ◼   ► considerations in the file system design [TS]

00:29:49 ◼   ► were based on spinning disks and butter [TS]

00:29:52 ◼   ► FS looks a lot like ZFS but it's newer [TS]

00:29:58 ◼   ► so they may have more SSD stuff in mind [TS]

00:30:00 ◼   ► certainly there is no file system that I [TS]

00:30:02 ◼   ► know of that's in widespread use it was [TS]

00:30:05 ◼   ► designed from the ground up to say [TS]

00:30:06 ◼   ► forget spinning disks in fact if you run [TS]

00:30:09 ◼   ► this on a spinning disk it might be [TS]

00:30:10 ◼   ► hideous and terrible this is entirely a [TS]

00:30:12 ◼   ► ended SSD Boyd Watters wrote in to say [TS]

00:30:16 ◼   ► that to quote him plopping NAND flash [TS]

00:30:21 ◼   ► behind a SATA interface is stupid so [TS]

00:30:24 ◼   ► what he's put is showing is that thing [TS]

00:30:25 ◼   ► of the storage hierarchy that exists in [TS]

00:30:27 ◼   ► current things you got the CPU register [TS]

00:30:30 ◼   ► file you got the the caching on the CPU [TS]

00:30:32 ◼   ► then you've got the RAM then you've got [TS]

00:30:33 ◼   ► the the i/o controller chip then you've [TS]

00:30:35 ◼   ► got the the SATA interface and then [TS]

00:30:38 ◼   ► you've got this flash disk and you put [TS]

00:30:39 ◼   ► discs and [TS]

00:30:40 ◼   ► scare quotes because it's not actually a [TS]

00:30:42 ◼   ► disk why does that big structure exist [TS]

00:30:47 ◼   ► why do these flash memory chips have to [TS]

00:30:51 ◼   ► pretend they're a spinning disk does [TS]

00:30:53 ◼   ► that seem like a silly kind of archaic [TS]

00:30:56 ◼   ► it's kind of like a computational [TS]

00:30:58 ◼   ► skeuomorphism where it's pretending to [TS]

00:31:02 ◼   ► be a disk because everything expects [TS]

00:31:03 ◼   ► there to be a disk there but it's [TS]

00:31:04 ◼   ► totally not this is an interesting issue [TS]

00:31:09 ◼   ► that has come up many times in the in [TS]

00:31:11 ◼   ► the past decades like why do we had disk [TS]

00:31:13 ◼   ► and memory wouldn't it be nice if [TS]

00:31:15 ◼   ► everything was just memory that's the [TS]

00:31:18 ◼   ► you know the computer science dream [TS]

00:31:19 ◼   ► everything is just a big open field of [TS]

00:31:21 ◼   ► bits and it can be addressed randomly [TS]

00:31:24 ◼   ► written land rent they write or read [TS]

00:31:26 ◼   ► randomly and there's no difference [TS]

00:31:29 ◼   ► between something's in disk and it's in [TS]

00:31:31 ◼   ► memory just one big continuum one way to [TS]

00:31:33 ◼   ► do that would be to have notice can just [TS]

00:31:34 ◼   ► have everything in memory because then [TS]

00:31:36 ◼   ► everything is memory right but that's [TS]

00:31:40 ◼   ► generally not feasible because memory is [TS]

00:31:42 ◼   ► measured in you know gigabytes and disk [TS]

00:31:45 ◼   ► space is memory measured in terabytes so [TS]

00:31:48 ◼   ► they're not close and the other thing is [TS]

00:31:49 ◼   ► just less I have a unified interface [TS]

00:31:50 ◼   ► that yes it's flat flash is slower than [TS]

00:31:55 ◼   ► then DRAM but the Ihram is also slower [TS]

00:31:58 ◼   ► than l3 cache which is slower than l2 [TS]

00:31:59 ◼   ► cache which is slower than l1 cache [TS]

00:32:01 ◼   ► which is slower than registers it's a [TS]

00:32:02 ◼   ► hierarchy of memory how close quote [TS]

00:32:05 ◼   ► unquote close is this memory to the CPU [TS]

00:32:08 ◼   ► how long does it take the CPU to get to [TS]

00:32:10 ◼   ► it that hierarchy exists and [TS]

00:32:12 ◼   ► traditionally if you were to graph how [TS]

00:32:14 ◼   ► you know how long does it take to get a [TS]

00:32:16 ◼   ► piece of memory out of l1 cache l2 cache [TS]

00:32:18 ◼   ► l3 cache it's like a hockey stick graph [TS]

00:32:20 ◼   ► and especially when you hit disk it's [TS]

00:32:21 ◼   ► like well now you might as well just go [TS]

00:32:23 ◼   ► on vacation because that that is not [TS]

00:32:24 ◼   ► going to be coming back for another 8 [TS]

00:32:25 ◼   ► billion cycles of the CPU so forget it [TS]

00:32:30 ◼   ► flash memory is faster than just what [TS]

00:32:32 ◼   ► that wall is still there that the gap [TS]

00:32:33 ◼   ► between the speed of memory is still [TS]

00:32:35 ◼   ► there but still the idea is that why not [TS]

00:32:38 ◼   ► just have this is a uniform storage [TS]

00:32:39 ◼   ► hierarchy with uniform interface and not [TS]

00:32:42 ◼   ► have disk i/o to be this totally [TS]

00:32:44 ◼   ► separate thing that's something that [TS]

00:32:48 ◼   ► someone like Apple is not going to [TS]

00:32:49 ◼   ► probably tackle anytime soon but it's [TS]

00:32:52 ◼   ► been in the air [TS]

00:32:54 ◼   ► probably since the 60s or 70s the idea [TS]

00:32:56 ◼   ► that a unified interface to memory would [TS]

00:32:58 ◼   ► be better than and probably a lot of old [TS]

00:33:00 ◼   ► machines did have a unified interface [TS]

00:33:01 ◼   ► but in the end the person computer age [TS]

00:33:05 ◼   ► the split weakened disk in memory is [TS]

00:33:06 ◼   ► pretty baked into the way that we do [TS]

00:33:09 ◼   ► things so let's consider what might [TS]

00:33:12 ◼   ► change if everything was unified well [TS]

00:33:17 ◼   ► one thing that would have to change is [TS]

00:33:19 ◼   ► that the operating system would have to [TS]

00:33:22 ◼   ► be different because in the operating [TS]

00:33:23 ◼   ► system the file IO calls are very [TS]

00:33:25 ◼   ► different from the calls to manipulate [TS]

00:33:26 ◼   ► memory file alcohols you've got you know [TS]

00:33:28 ◼   ► opening files reading writing files [TS]

00:33:30 ◼   ► they're just structured differently take [TS]

00:33:31 ◼   ► different arguments have different [TS]

00:33:32 ◼   ► semantics than the memory calls which [TS]

00:33:34 ◼   ► are you know setting memory getting [TS]

00:33:37 ◼   ► memory copying memory and just bit [TS]

00:33:39 ◼   ► operations that happen in memory in two [TS]

00:33:43 ◼   ► entirely different api's so you couldn't [TS]

00:33:46 ◼   ► just say ok memory and disk or all one [TS]

00:33:47 ◼   ► because everyone's programs wouldn't [TS]

00:33:49 ◼   ► work anymore because that's you know [TS]

00:33:51 ◼   ► you'd have to you'd have to make some [TS]

00:33:52 ◼   ► sort of shim layer there and ideally [TS]

00:33:54 ◼   ► you'd want one unified interface to a [TS]

00:33:56 ◼   ► new set of api's that looks the same for [TS]

00:34:00 ◼   ► both of those things and what does that [TS]

00:34:01 ◼   ► new wear interface look like and who is [TS]

00:34:03 ◼   ► going to adopt that new integration [TS]

00:34:04 ◼   ► who's going to design it that's a big [TS]

00:34:05 ◼   ► problem one thing that would go away are [TS]

00:34:08 ◼   ► these really complicated api's that we [TS]

00:34:10 ◼   ► have now for memory mapping files where [TS]

00:34:12 ◼   ► you say look I want to manipulate this [TS]

00:34:16 ◼   ► really big file but I want to read every [TS]

00:34:18 ◼   ► single byte of it off disk and I want to [TS]

00:34:19 ◼   ► manipulate it as if it's all in memory [TS]

00:34:21 ◼   ► but I can't fit it all in memory so can [TS]

00:34:22 ◼   ► i memory map the file and memory mapping [TS]

00:34:24 ◼   ► will automatically make it look like [TS]

00:34:25 ◼   ► your entire files in memory but just [TS]

00:34:26 ◼   ► pulling the parts that you need on [TS]

00:34:28 ◼   ► demand and that's a very complicated [TS]

00:34:30 ◼   ► system and the API is very complicated [TS]

00:34:33 ◼   ► that's kind of stuff would go away if [TS]

00:34:36 ◼   ► you had a single unified interface to [TS]

00:34:38 ◼   ► the entire storage hierarchy or maybe [TS]

00:34:40 ◼   ► you would just shift and that becomes [TS]

00:34:41 ◼   ► part of this unified interface but it's [TS]

00:34:43 ◼   ► a complexity that exists now because we [TS]

00:34:45 ◼   ► know that files are much bigger than [TS]

00:34:47 ◼   ► memory we know we can't fit an entire [TS]

00:34:48 ◼   ► file in memory so we have to make this [TS]

00:34:50 ◼   ► weird illusion that it's kind of in [TS]

00:34:52 ◼   ► memory when it's not ah a big one that I [TS]

00:34:55 ◼   ► thought of is that you'd have to [TS]

00:34:56 ◼   ► reconsider limits limits that we take [TS]

00:34:59 ◼   ► for granted now so for example when you [TS]

00:35:03 ◼   ► run out of disk space that's a dead end [TS]

00:35:05 ◼   ► right now [TS]

00:35:07 ◼   ► no space left on device right that's [TS]

00:35:10 ◼   ► that's the end you know this there's no [TS]

00:35:12 ◼   ► program in existence that says when I [TS]

00:35:15 ◼   ► run out it's like what can it what kind [TS]

00:35:16 ◼   ► of program do well jeez on that it can [TS]

00:35:18 ◼   ► tell you I try to do that for you but [TS]

00:35:20 ◼   ► your disk is full so and and by the way [TS]

00:35:23 ◼   ► UNIX does not deal well with the disk [TS]

00:35:24 ◼   ► being full for reasons we'll probably [TS]

00:35:26 ◼   ► get into in a bit later uh and you don't [TS]

00:35:30 ◼   ► want you just to be complete full lots [TS]

00:35:32 ◼   ► of bad things happen but running out of [TS]

00:35:34 ◼   ► RAM waiting it what do you say UNIX [TS]

00:35:36 ◼   ► doesn't handle it well what does handle [TS]

00:35:38 ◼   ► it well or is that is that part of that [TS]

00:35:41 ◼   ► next topic yeah that what does handle it [TS]

00:35:44 ◼   ► well in it in the days before UNIX for [TS]

00:35:46 ◼   ► example in classic Mac OS in like really [TS]

00:35:48 ◼   ► classic Mac OS like you know system 4.2 [TS]

00:35:52 ◼   ► you could have your disk filled up to [TS]

00:35:54 ◼   ► the brim and everything would still [TS]

00:35:57 ◼   ► pretty much work okay you know you [TS]

00:35:58 ◼   ► couldn't save stuff but you wouldn't be [TS]

00:36:01 ◼   ► using the thing and it would just blow [TS]

00:36:03 ◼   ► up whereas the UNIX uses the disk space [TS]

00:36:06 ◼   ► for all sorts of things and if your disk [TS]

00:36:10 ◼   ► even gets close to full lots of things [TS]

00:36:12 ◼   ► they're going to start falling down not [TS]

00:36:13 ◼   ► just the program you're using so you [TS]

00:36:14 ◼   ► can't just happily use your Mac with an [TS]

00:36:16 ◼   ► entirely full disk in fact I don't even [TS]

00:36:18 ◼   ► know if you can get your your disk [TS]

00:36:19 ◼   ► entirely full without sort of cheating [TS]

00:36:20 ◼   ► by putting it to single user mode and [TS]

00:36:22 ◼   ► using DD to fill up the disk I don't [TS]

00:36:24 ◼   ► even think you can boot with it with an [TS]

00:36:25 ◼   ► entirely full disk once you get anywhere [TS]

00:36:27 ◼   ► near close to disk full Mac os10 starts [TS]

00:36:29 ◼   ► throwing dialogues in your faces said [TS]

00:36:30 ◼   ► this is almost full seriously delete [TS]

00:36:32 ◼   ► stuff you've got a problem and there's a [TS]

00:36:34 ◼   ► good reason for that uh but in the [TS]

00:36:37 ◼   ► modern era running out of RAM is not a [TS]

00:36:40 ◼   ► dead end and I say the modern era being [TS]

00:36:43 ◼   ► like Mac OS 10 certainly but even in the [TS]

00:36:45 ◼   ► latter days of classic Mac OS what [TS]

00:36:47 ◼   ► happens when we run out of RAM is we're [TS]

00:36:49 ◼   ► not around all the time ah [TS]

00:36:52 ◼   ► you run out of RAM and you swap some [TS]

00:36:56 ◼   ► stuff to disk so you take some stuff [TS]

00:36:58 ◼   ► that's in memory though I don't have [TS]

00:36:59 ◼   ► room to put everything in memory that [TS]

00:37:00 ◼   ► need to put in a memory let me take [TS]

00:37:02 ◼   ► something that's in memory or write it [TS]

00:37:03 ◼   ► out to this temporarily now I've read up [TS]

00:37:05 ◼   ► some new room in memory and it tries to [TS]

00:37:07 ◼   ► put stuff on to disk [TS]

00:37:08 ◼   ► tries to swap stuff out that hasn't been [TS]

00:37:11 ◼   ► used in a while and this is the basis of [TS]

00:37:13 ◼   ► all modern virtual memory systems with [TS]

00:37:16 ◼   ► swap where they will [TS]

00:37:19 ◼   ► try to let it appear to your programs as [TS]

00:37:21 ◼   ► if you have way more RAM than you do so [TS]

00:37:23 ◼   ► programs back the old days and 32-bit [TS]

00:37:26 ◼   ► thing they would think that they had you [TS]

00:37:27 ◼   ► know two to four gigabytes of RAM when [TS]

00:37:30 ◼   ► maybe your computer only at 128 [TS]

00:37:32 ◼   ► megabytes of RAM and the program was [TS]

00:37:35 ◼   ► blissfully unaware of that it would just [TS]

00:37:36 ◼   ► try to address memory in the operating [TS]

00:37:38 ◼   ► system every time you filled up the [TS]

00:37:39 ◼   ► actual RAM chips [TS]

00:37:41 ◼   ► it would just swap things in and out to [TS]

00:37:43 ◼   ► disk to make enough room for the stuff [TS]

00:37:44 ◼   ► you currently using in ram now swapping [TS]

00:37:47 ◼   ► is bad and it's slow because if your [TS]

00:37:48 ◼   ► program thinks something's in memory and [TS]

00:37:50 ◼   ► it asks the operating system for on the [TS]

00:37:51 ◼   ► operating system says actually that's [TS]

00:37:53 ◼   ► not in memory I had to put that in disk [TS]

00:37:54 ◼   ► for you but I'll wait I'll go get it on [TS]

00:37:55 ◼   ► the disk you just wait there it takes [TS]

00:37:57 ◼   ► forever for the operating system to come [TS]

00:37:59 ◼   ► back so here's this thing and the [TS]

00:38:00 ◼   ► program expected that to come back [TS]

00:38:02 ◼   ► really quickly so when your computer is [TS]

00:38:05 ◼   ► thrashing or swapping or one of these [TS]

00:38:07 ◼   ► other terms it means that it's trying to [TS]

00:38:09 ◼   ► access a bunch of stuff that it thinks [TS]

00:38:10 ◼   ► is in memory but a lot of it turns out [TS]

00:38:12 ◼   ► to be on disk and your disk ad they're [TS]

00:38:13 ◼   ► seeking all around to pull this stuff [TS]

00:38:15 ◼   ► off of the disk and putting other stuff [TS]

00:38:16 ◼   ► onto this because if it's on if it's on [TS]

00:38:19 ◼   ► disk has to pull it out maybe that's to [TS]

00:38:20 ◼   ► evict something else from RAM and put it [TS]

00:38:21 ◼   ► down there so that's situation we're in [TS]

00:38:26 ◼   ► now where a disk space fills up total [TS]

00:38:28 ◼   ► dead end RAM fills up it's not a dead [TS]

00:38:30 ◼   ► end but how is RAM filling up not a dead [TS]

00:38:32 ◼   ► it uses disk space and that's why if [TS]

00:38:34 ◼   ► your disk is full everything falls down [TS]

00:38:37 ◼   ► because all these programs think that [TS]

00:38:38 ◼   ► they have you know much more memory than [TS]

00:38:41 ◼   ► they really have and if they did blondie [TS]

00:38:43 ◼   ► continued to put stuff in memory in the [TS]

00:38:44 ◼   ► operating system says well that actually [TS]

00:38:46 ◼   ► can't fit in memory so what I'm supposed [TS]

00:38:48 ◼   ► to do on behalf of this program is evict [TS]

00:38:50 ◼   ► something from memory put that on disk [TS]

00:38:53 ◼   ► and then put his thing in that spot and [TS]

00:38:54 ◼   ► when it goes to evict something from [TS]

00:38:55 ◼   ► memory and put it on disk the disk is [TS]

00:38:57 ◼   ► full and that's why bad things start [TS]

00:38:59 ◼   ► happening classic Mac OS way back and [TS]

00:39:01 ◼   ► they did not have virtual memory which [TS]

00:39:03 ◼   ► is different than swap but it certainly [TS]

00:39:04 ◼   ► didn't have swap space so it could [TS]

00:39:07 ◼   ► happily hum along with its you know 512 [TS]

00:39:09 ◼   ► K of RAM or whatever and if you hit the [TS]

00:39:12 ◼   ► 512 K Ram limit ah that was the end of [TS]

00:39:16 ◼   ► that it wouldn't swap stuff out to disk [TS]

00:39:17 ◼   ► so if your disk was full no big deal [TS]

00:39:20 ◼   ► it's the only time it has to write [TS]

00:39:22 ◼   ► something on to disk is if you ask it to [TS]

00:39:23 ◼   ► not in your normal operation it didn't [TS]

00:39:26 ◼   ► write stuff and UNIX of course has log [TS]

00:39:27 ◼   ► files that are constantly being appended [TS]

00:39:29 ◼   ► to and other things that are going on in [TS]

00:39:30 ◼   ► the background slowly filling in your [TS]

00:39:31 ◼   ► disk that [TS]

00:39:32 ◼   ► know about and if your disk is full all [TS]

00:39:33 ◼   ► those things freak out and say you know [TS]

00:39:36 ◼   ► we've got a problem here they would [TS]

00:39:37 ◼   ► probably they would probably try to log [TS]

00:39:38 ◼   ► the fact that they got a no space left [TS]

00:39:40 ◼   ► on device message and that would cause [TS]

00:39:41 ◼   ► more Disgaea which would also fail and [TS]

00:39:43 ◼   ► it's just a vaccine so there's an [TS]

00:39:47 ◼   ► exception to this in the modern error [TS]

00:39:48 ◼   ► which is iOS which some people also [TS]

00:39:51 ◼   ► asked about iOS if you run out of disk [TS]

00:39:55 ◼   ► space that's a dead end still if you run [TS]

00:39:58 ◼   ► out of RAM on iOS it's also a didn't [TS]

00:40:02 ◼   ► because iOS though it implements virtual [TS]

00:40:04 ◼   ► memory virtual memory is the mechanism [TS]

00:40:06 ◼   ► by which the programs think they have [TS]

00:40:08 ◼   ► much more memory and they do then they [TS]

00:40:10 ◼   ► really do and they think they all have [TS]

00:40:11 ◼   ► their own memory space to themselves [TS]

00:40:14 ◼   ► it has virtual memory but it does not [TS]

00:40:16 ◼   ► use swap so when you fill up all the RAM [TS]

00:40:19 ◼   ► on iOS it doesn't say oh let me put that [TS]

00:40:21 ◼   ► on quote-unquote disk for you it's disk [TS]

00:40:23 ◼   ► is obviously flash what it does is kills [TS]

00:40:25 ◼   ► your program in fact to kill your [TS]

00:40:26 ◼   ► program long before that iOS is a very [TS]

00:40:28 ◼   ► hostile environment for programs and the [TS]

00:40:30 ◼   ► the application environment for running [TS]

00:40:33 ◼   ► GUI applications in iOS will send your [TS]

00:40:37 ◼   ► application a low memory warning if it [TS]

00:40:38 ◼   ► starts using too much RAM and if the [TS]

00:40:42 ◼   ► program does not respond by freeing [TS]

00:40:43 ◼   ► memory eventually the operating system [TS]

00:40:45 ◼   ► will just kill that program say look I [TS]

00:40:46 ◼   ► sent you a message I told you you're [TS]

00:40:47 ◼   ► getting close to the memory limit I'm [TS]

00:40:48 ◼   ► gonna have to kill you because if you [TS]

00:40:49 ◼   ► ever hit the memory limit on iOS that's [TS]

00:40:53 ◼   ► it it's not you know I need more space [TS]

00:40:55 ◼   ► well RAM is full and there's no more RAM [TS]

00:40:57 ◼   ► we can't make new Ram out of thin air [TS]

00:40:59 ◼   ► and we're not going to swap the flash [TS]

00:41:00 ◼   ► now one of the reasons that iOS doesn't [TS]

00:41:01 ◼   ► swap to flash is because even though [TS]

00:41:04 ◼   ► flash is faster than disk it's still way [TS]

00:41:06 ◼   ► way WAY slower than Ram so that's one of [TS]

00:41:10 ◼   ► the reasons that iOS has always been so [TS]

00:41:11 ◼   ► responsive because to get that kind of [TS]

00:41:13 ◼   ► responsiveness that's how it's running a [TS]

00:41:15 ◼   ► variant of Mac OS 10 we're doing virtual [TS]

00:41:17 ◼   ► memory but we cannot afford to like oh [TS]

00:41:20 ◼   ► when I swipe something on the screen oh [TS]

00:41:21 ◼   ► hang on a second I gotta pull a bunch of [TS]

00:41:23 ◼   ► that stuff that you thought was in [TS]

00:41:24 ◼   ► memory actually it's on my flash disk [TS]

00:41:26 ◼   ► and I got to pull it off the disk and [TS]

00:41:28 ◼   ► it's it can take a thousand times longer [TS]

00:41:29 ◼   ► than you thought it would and then you [TS]

00:41:30 ◼   ► get the stuttery animation or anything [TS]

00:41:32 ◼   ► maybe someday that will change but for [TS]

00:41:35 ◼   ► now as far as I'm aware maybe someone [TS]

00:41:37 ◼   ► can correct me [TS]

00:41:38 ◼   ► Iowa still does not use swap space it [TS]

00:41:41 ◼   ► puts everything in RAM so if you're [TS]

00:41:44 ◼   ► going to unify disk [TS]

00:41:45 ◼   ► and ram into one type of thing [TS]

00:41:49 ◼   ► everything would have to become a dead [TS]

00:41:50 ◼   ► end [TS]

00:41:51 ◼   ► because you have no other place to put [TS]

00:41:53 ◼   ► stuff right like when you run out of [TS]

00:41:56 ◼   ► whatever you're going to call this thing [TS]

00:41:58 ◼   ► when you run out of memory that means [TS]

00:41:59 ◼   ► that your caches are full you know they [TS]

00:42:02 ◼   ► know you can put in a register can't put [TS]

00:42:04 ◼   ► it in a cache you can't put it in RAM [TS]

00:42:05 ◼   ► and uh your disk is also full and I [TS]

00:42:10 ◼   ► think that would be a bad place to be [TS]

00:42:13 ◼   ► because you could imagine getting [TS]

00:42:14 ◼   ► machine into a deadlock situation [TS]

00:42:15 ◼   ► hopefully not hopefully in a consistent [TS]

00:42:17 ◼   ► state of some kind but where do you go [TS]

00:42:19 ◼   ► from there you don't have an out you [TS]

00:42:20 ◼   ► don't have a place to put stuff maybe [TS]

00:42:22 ◼   ► we're just carve off it you would not do [TS]

00:42:23 ◼   ► the same thing where it would say if you [TS]

00:42:25 ◼   ► get close to the limit of my giant pool [TS]

00:42:27 ◼   ► of memory I'm going to warn you and then [TS]

00:42:30 ◼   ► I'm going to kill stuff off because we [TS]

00:42:31 ◼   ► cannot hit that limit sort of like you [TS]

00:42:35 ◼   ► know it does with disk space today where [TS]

00:42:36 ◼   ► you just can't get close to that limit [TS]

00:42:38 ◼   ► on disk pacer will start yelling about [TS]

00:42:39 ◼   ► you but Ram the operating system will [TS]

00:42:41 ◼   ► let you continue to use it and use it [TS]

00:42:42 ◼   ► when you have virtual memory with swap [TS]

00:42:44 ◼   ► until your computer becomes so slow that [TS]

00:42:46 ◼   ► you're discouraged from using it any [TS]

00:42:47 ◼   ► longer and you walk away and wait for [TS]

00:42:49 ◼   ► everything to swap in and out so I think [TS]

00:42:52 ◼   ► this is an interesting topic I don't [TS]

00:42:54 ◼   ► know what will happen in this area I [TS]

00:42:57 ◼   ► don't think any time in my lifetime [TS]

00:42:58 ◼   ► we're going to see this grand unified a [TS]

00:43:02 ◼   ► single pool of memory thing but if it is [TS]

00:43:07 ◼   ► going to happen I think it will happen [TS]

00:43:08 ◼   ► somewhere in the mobile space where the [TS]

00:43:10 ◼   ► sizes are smaller and the entrenched [TS]

00:43:14 ◼   ► system of dividing RAM and disk are not [TS]

00:43:19 ◼   ► as prevalent as they are on the PC [TS]

00:43:26 ◼   ► that's the end of my fake follow up [TS]

00:43:29 ◼   ► follow up I was what's fake about that I [TS]

00:43:32 ◼   ► thought it was very real well it was you [TS]

00:43:34 ◼   ► know it's talking about SSD is not [TS]

00:43:37 ◼   ► following up something we said last time [TS]

00:43:38 ◼   ► but it's a new top oh I see yeah so your [TS]

00:43:41 ◼   ► your confidence level of SSD over the [TS]

00:43:46 ◼   ► traditional Drive as far as a strategy [TS]

00:43:50 ◼   ► for Apple what are your thoughts on that [TS]

00:43:51 ◼   ► long-term strategy Apple's hardware [TS]

00:43:54 ◼   ► everything SSD yeah has to be mean it's [TS]

00:43:58 ◼   ► going to have [TS]

00:43:59 ◼   ► it's just a question of how long it [TS]

00:44:00 ◼   ► takes with Apple sells consumer hardware [TS]

00:44:02 ◼   ► so they don't have to deal with like [TS]

00:44:03 ◼   ► well I need to store petabytes of data [TS]

00:44:05 ◼   ► like they're not selling servers for the [TS]

00:44:07 ◼   ► data center and SSDs are way more [TS]

00:44:10 ◼   ► expensive than spinning disks we're just [TS]

00:44:14 ◼   ► kind of that inflection point where they [TS]

00:44:15 ◼   ► can say like when I got my MacBook Air [TS]

00:44:19 ◼   ► like the biggest s is the you get was [TS]

00:44:20 ◼   ► 256 and that's what I got and that was [TS]

00:44:23 ◼   ► still bigger than 160 gig spinning disk [TS]

00:44:25 ◼   ► that was in my old MacBook Pro so it was [TS]

00:44:27 ◼   ► still a step up but you can get like [TS]

00:44:30 ◼   ► terabyte now I think terabyte laptop [TS]

00:44:32 ◼   ► drives are available it's four times [TS]

00:44:34 ◼   ► smaller than than the the biggest lap [TS]

00:44:37 ◼   ► spinning laptop drive you can get and [TS]

00:44:39 ◼   ► way way way more expensive so they're [TS]

00:44:43 ◼   ► just on the borderline of like you can [TS]

00:44:45 ◼   ► make people choke down a much higher [TS]

00:44:48 ◼   ► price for less space just because the [TS]

00:44:50 ◼   ► performance is so much better but they [TS]

00:44:52 ◼   ► haven't quite crawled up to like can I [TS]

00:44:54 ◼   ► convince someone to buy a Mac Pro with [TS]

00:44:56 ◼   ► just SSDs in it someone who wants a Mac [TS]

00:44:59 ◼   ► Pro probably wants a lot more internal [TS]

00:45:00 ◼   ► storage space than 256 gigs and those [TS]

00:45:02 ◼   ► drives are so expensive if you buy for [TS]

00:45:04 ◼   ► them now the entire cost of your [TS]

00:45:06 ◼   ► computer is basically just an SSD so the [TS]

00:45:09 ◼   ► price keeps going down the storage space [TS]

00:45:10 ◼   ► keeps going up eventually there'll be a [TS]

00:45:13 ◼   ► point where spinning disks are just seen [TS]

00:45:17 ◼   ► as archaic and everyone has to have SSDs [TS]

00:45:19 ◼   ► and only the people who really need [TS]

00:45:21 ◼   ► gigantic amount of space need spinning [TS]

00:45:22 ◼   ► disks and eventually those will probably [TS]

00:45:24 ◼   ► go away - so that's inevitable you just [TS]

00:45:27 ◼   ► get used to the fact that as as these [TS]

00:45:29 ◼   ► are coming how long before every single [TS]

00:45:33 ◼   ► computer Apple makes has SSD by default [TS]

00:45:35 ◼   ► the kinds of if they discontinue the Mac [TS]

00:45:38 ◼   ► Pros thoughts on discontinuing the Mac [TS]

00:45:40 ◼   ► Pro don't do it that's my thought please [TS]

00:45:43 ◼   ► as you do it you'd like you like the Mac [TS]

00:45:45 ◼   ► Pro and you're you're a big-time Mac Pro [TS]

00:45:47 ◼   ► user yeah you don't want them to get rid [TS]

00:45:49 ◼   ► of it I was I saw an article from I [TS]

00:45:52 ◼   ► don't know if you've spoken with him but [TS]

00:45:54 ◼   ► he's a friend of mine he used to do a [TS]

00:45:56 ◼   ► show here as well James Duncan Davidson [TS]

00:45:59 ◼   ► he's a photographer and right now or [TS]

00:46:02 ◼   ► very recently he was out shooting Ted [TS]

00:46:04 ◼   ► and he was saying that every year that [TS]

00:46:06 ◼   ► he shot Ted which is a number of years [TS]

00:46:08 ◼   ► these guys they spare no expense when it [TS]

00:46:11 ◼   ► comes to having great [TS]

00:46:12 ◼   ► hardware for the people who are doing [TS]

00:46:14 ◼   ► the audio video editing and then the [TS]

00:46:17 ◼   ► photography editing and things like that [TS]

00:46:18 ◼   ► and they set up a whole lab I'll try and [TS]

00:46:21 ◼   ► put I'll try and put one of these [TS]

00:46:22 ◼   ► pictures that he's taken of the lab into [TS]

00:46:25 ◼   ► the show notes and by the way thanks to [TS]

00:46:28 ◼   ► help spot calm for the show notes and [TS]

00:46:31 ◼   ► this year instead of having two dozen [TS]

00:46:35 ◼   ► Mac Pros they now have two dozen IMAX [TS]

00:46:39 ◼   ► with Thunderbolt drives connected to [TS]

00:46:42 ◼   ► them raid Thunderbolt drives connected [TS]

00:46:44 ◼   ► to them and they have officially moved [TS]

00:46:46 ◼   ► away completely from Mac Pros and now [TS]

00:46:49 ◼   ► they just have IMAX and it makes you [TS]

00:46:51 ◼   ► think that if if a company like tad if a [TS]

00:46:54 ◼   ► group like Ted is as as serious and [TS]

00:46:58 ◼   ► dedicated as they are to doing real and [TS]

00:47:02 ◼   ► if you've ever watched the Ted videos [TS]

00:47:04 ◼   ► you know these are these are phenomenal [TS]

00:47:06 ◼   ► videos they have the best audio-video [TS]

00:47:08 ◼   ► photographer people in the business [TS]

00:47:09 ◼   ► making them and editing them and turning [TS]

00:47:12 ◼   ► them around in almost real-time it's [TS]

00:47:14 ◼   ► it's crazy how fast they put these [TS]

00:47:16 ◼   ► things out there they're content with [TS]

00:47:19 ◼   ► IMAX why why would Apple keep making the [TS]

00:47:23 ◼   ► Mac Pro who is that for isn't the [TS]

00:47:25 ◼   ► Thunderbolt the obvious indication that [TS]

00:47:27 ◼   ► they're serious about moving away from [TS]

00:47:29 ◼   ► the Mac Pro well that particular [TS]

00:47:33 ◼   ► application I think doesn't need the [TS]

00:47:36 ◼   ► massive amounts of storage because kind [TS]

00:47:38 ◼   ► of a way station for a workflow that [TS]

00:47:40 ◼   ► takes assets from the day and grinds [TS]

00:47:42 ◼   ► them up and spits out some new assets [TS]

00:47:44 ◼   ► where you need the big spinning disks [TS]

00:47:47 ◼   ► that can store so much more than SSDs [TS]

00:47:48 ◼   ► for so much less money is where does the [TS]

00:47:51 ◼   ► company that is responsible for [TS]

00:47:54 ◼   ► recording all of Ted and producing all [TS]

00:47:56 ◼   ► the content where does it distorts [TS]

00:47:58 ◼   ► information where does it start all the [TS]

00:48:00 ◼   ► raw videos all the raw images all the [TS]

00:48:02 ◼   ► processed files everything forever [TS]

00:48:04 ◼   ► archived maybe puts them all on tape and [TS]

00:48:09 ◼   ► that's the answer but I think companies [TS]

00:48:11 ◼   ► that have large storage requirements [TS]

00:48:13 ◼   ► like people doing special effects for [TS]

00:48:14 ◼   ► movies or something like that they'll [TS]

00:48:17 ◼   ► use SSDs for the parts where speed [TS]

00:48:19 ◼   ► matters but at some point that like they [TS]

00:48:21 ◼   ► don't want to ever throw anything out [TS]

00:48:22 ◼   ► you want to have all your assets and all [TS]

00:48:23 ◼   ► everything redundantly stored in a [TS]

00:48:24 ◼   ► million place and that takes up a [TS]

00:48:25 ◼   ► tremendous and [TS]

00:48:26 ◼   ► room even just for one movie let alone [TS]

00:48:28 ◼   ► movie after movie like how many assets [TS]

00:48:29 ◼   ► do you think Pixar has stored [TS]

00:48:31 ◼   ► permanently yeah it accessible not on [TS]

00:48:34 ◼   ► tape or some robot has to go pull a tape [TS]

00:48:36 ◼   ► out and shove it in but accessible in [TS]

00:48:38 ◼   ► semi real-time because you might want to [TS]

00:48:39 ◼   ► use that asset for a new production you [TS]

00:48:41 ◼   ► need some list to put all that stuff in [TS]

00:48:42 ◼   ► SSDs just cost too much money to to do [TS]

00:48:45 ◼   ► that now spinning disks are so much more [TS]

00:48:47 ◼   ► economical especially since you have to [TS]

00:48:49 ◼   ► have 20 of them because you have to have [TS]

00:48:50 ◼   ► everything redundantly stored and you [TS]

00:48:52 ◼   ► know I there is still a big market for [TS]

00:48:55 ◼   ► it for spinning discs now is that is [TS]

00:48:58 ◼   ► Apple does Apple have any part of that [TS]

00:48:59 ◼   ► market maybe they're going to say look [TS]

00:49:02 ◼   ► if you want to do that you're gonna have [TS]

00:49:03 ◼   ► to buy disk storage from EMC or [TS]

00:49:05 ◼   ► something just like we at Apple do for [TS]

00:49:07 ◼   ► our data centers because we're not [TS]

00:49:08 ◼   ► interested in giving you a machine that [TS]

00:49:11 ◼   ► can store all that stuff or as you said [TS]

00:49:12 ◼   ► our machines can't store it buy an iMac [TS]

00:49:15 ◼   ► and connect one of these big raid things [TS]

00:49:17 ◼   ► that we also don't make to a Thunderbolt [TS]

00:49:20 ◼   ► cable you have really fast access to to [TS]

00:49:22 ◼   ► lots of storage the Mac Pro as a product [TS]

00:49:25 ◼   ► exists for reasons other than just [TS]

00:49:27 ◼   ► internal drives though it's got the [TS]

00:49:28 ◼   ► slots where you can put cards that you [TS]

00:49:29 ◼   ► might need cards that don't fit inside [TS]

00:49:31 ◼   ► the iMac because there's no place for [TS]

00:49:32 ◼   ► cards at all and yet maybe you can use a [TS]

00:49:34 ◼   ► Thunderbolt external chassis but the [TS]

00:49:35 ◼   ► number of PCI Express Lanes over a [TS]

00:49:37 ◼   ► thunderbolt wire is not that that bus is [TS]

00:49:41 ◼   ► not as wide as an internal slot so there [TS]

00:49:44 ◼   ► are many reasons why the Mac Pro still [TS]

00:49:45 ◼   ► needs to exist the fact that they [TS]

00:49:46 ◼   ► switched from Mac Pro Stimac for that [TS]

00:49:48 ◼   ► Ted thing they probably could have [TS]

00:49:50 ◼   ► gotten away with imax even sooner [TS]

00:49:53 ◼   ► because it's just kind of a way station [TS]

00:49:54 ◼   ► and not something that needs to have [TS]

00:49:56 ◼   ► lots of storage and those big you know [TS]

00:49:58 ◼   ► big special-purpose cards and yeah and [TS]

00:50:01 ◼   ► also the other thing to keep in mind is [TS]

00:50:02 ◼   ► that Ted probably has a lot of money you [TS]

00:50:05 ◼   ► would think so I put I I did find the [TS]

00:50:07 ◼   ► article and the pictures that go along [TS]

00:50:09 ◼   ► with it I put it into the show notes and [TS]

00:50:10 ◼   ► he says for Ted 2012 there's a pretty do [TS]

00:50:13 ◼   ► you say SATA or sad at these days I say [TS]

00:50:16 ◼   ► both okay for Ted 2012 there's a pretty [TS]

00:50:19 ◼   ► big change in the media room instead of [TS]

00:50:20 ◼   ► dozens of Mac Pros and piles of SATA [TS]

00:50:22 ◼   ► drives like there have been in the last [TS]

00:50:24 ◼   ► eight Ted events I've been part of the [TS]

00:50:26 ◼   ► room is full of IMAX and Thunderbolt [TS]

00:50:28 ◼   ► drives lots of Thunderbolt drives and he [TS]

00:50:30 ◼   ► has a photograph here he says there are [TS]

00:50:31 ◼   ► 12 12 12 terabyte Pegasus are six arrays [TS]

00:50:36 ◼   ► that's a hundred and forty four [TS]

00:50:37 ◼   ► terabytes total [TS]

00:50:39 ◼   ► he says it which is about a tenth of a [TS]

00:50:41 ◼   ► petabyte of usual usable space for good [TS]

00:50:44 ◼   ► measure there are a few more pegasus [TS]

00:50:45 ◼   ► arrays including one that's dedicated to [TS]

00:50:47 ◼   ► photography and he goes on to detail [TS]

00:50:49 ◼   ► this and he says this is the biggest [TS]

00:50:52 ◼   ► appointment of thunderbolt based storage [TS]

00:50:53 ◼   ► that he's seen yeah I think Thunderbolt [TS]

00:50:56 ◼   ► has made it feasible to have external [TS]

00:50:59 ◼   ► storage that's really fast without [TS]

00:51:02 ◼   ► having some weird interface card like [TS]

00:51:04 ◼   ► fiber channel or something because [TS]

00:51:06 ◼   ► internal disk was the way the way how [TS]

00:51:08 ◼   ► you get the speed right because you know [TS]

00:51:10 ◼   ► it's fast as possible as you can get [TS]

00:51:11 ◼   ► it's connected right to the thing [TS]

00:51:12 ◼   ► there's no long wire there's no weird [TS]

00:51:14 ◼   ► interface it's you know sad is pretty [TS]

00:51:15 ◼   ► fast and they would keep cranking up the [TS]

00:51:16 ◼   ► speed and then once you went out over [TS]

00:51:19 ◼   ► firewire like firewire 400 in particular [TS]

00:51:21 ◼   ► was not enough to handle the output of a [TS]

00:51:24 ◼   ► arrayed with many spindles or any large [TS]

00:51:27 ◼   ► array like that firewire 800 was an [TS]

00:51:29 ◼   ► improvement but still I think as the the [TS]

00:51:32 ◼   ► Thunderbolt devices they're available [TS]

00:51:34 ◼   ► today show they'll take like a four disc [TS]

00:51:36 ◼   ► firewire 800 Drive and then I'll take a [TS]

00:51:39 ◼   ► four disc Thunderbolt array maybe it's [TS]

00:51:42 ◼   ► the same exact four mechanisms in each [TS]

00:51:44 ◼   ► set light but that by eight mechanisms [TS]

00:51:45 ◼   ► but form one form the other and the [TS]

00:51:46 ◼   ► Thunderbolt ones just transit because [TS]

00:51:48 ◼   ► Thunderbolt is like an external [TS]

00:51:50 ◼   ► extension of an internal bus in this [TS]

00:51:52 ◼   ► case it's the PCI Express bus so that [TS]

00:51:54 ◼   ► that probably makes it feasible for them [TS]

00:51:56 ◼   ► to have storage that's fast enough to do [TS]

00:51:59 ◼   ► all this HD video streaming or whatever [TS]

00:52:00 ◼   ► they're doing whilst just hooking it up [TS]

00:52:03 ◼   ► to an iMac with that little Thunderbolt [TS]

00:52:04 ◼   ► cable but the Mac pros are still an open [TS]

00:52:07 ◼   ► question I think let's do our second [TS]

00:52:10 ◼   ► sponsor its tiny letter comm they're [TS]

00:52:12 ◼   ► just they're putting the finishing [TS]

00:52:13 ◼   ► touches on a brand-new version and this [TS]

00:52:16 ◼   ► is very exciting they have been working [TS]

00:52:18 ◼   ► hard on this I talked to them about it [TS]

00:52:20 ◼   ► and a tiny letter in general the best [TS]

00:52:23 ◼   ► way for me to describe it to you is it [TS]

00:52:25 ◼   ► is the most simple straightforward [TS]

00:52:27 ◼   ► newsletter tool that has ever existed in [TS]

00:52:30 ◼   ► the history of the earth it's elegant [TS]

00:52:33 ◼   ► it's simple its straightforward it gets [TS]

00:52:36 ◼   ► rid of all of the extra features and [TS]

00:52:38 ◼   ► cruft and user interface layers and [TS]

00:52:41 ◼   ► throws all that out the window now some [TS]

00:52:43 ◼   ► people might want that go to MailChimp [TS]

00:52:45 ◼   ► com if you want that you don't want that [TS]

00:52:47 ◼   ► tiny letter super simple elegant [TS]

00:52:51 ◼   ► straightforward tiny letter com [TS]

00:52:53 ◼   ► they have a sneak peak video that they [TS]

00:52:55 ◼   ► just put up to give you an idea of what [TS]

00:52:57 ◼   ► it's going to be like within the next [TS]

00:53:01 ◼   ► few days when this thing comes out so go [TS]

00:53:03 ◼   ► check it out thanks very much two tiny [TS]

00:53:05 ◼   ► letter calm for making the show possible [TS]

00:53:06 ◼   ► I do actually have some more files of [TS]

00:53:10 ◼   ► some stuff lurking at the bottom here [TS]

00:53:12 ◼   ► we're going to talk about BOS no okay [TS]

00:53:15 ◼   ► would be good ID now you give me more [TS]

00:53:16 ◼   ► stuff I tried the Kappas to shows I had [TS]

00:53:18 ◼   ► to assume well I actually I wanted to [TS]

00:53:21 ◼   ► talk more about ZFS because all the [TS]

00:53:23 ◼   ► stuff we talked about was EFS was [TS]

00:53:25 ◼   ► focused on the particular failings of [TS]

00:53:27 ◼   ► HFS+ and how they're addressed by ZFS [TS]

00:53:29 ◼   ► with data integrity and all that but [TS]

00:53:32 ◼   ► there are so many more things as e of us [TS]

00:53:33 ◼   ► that I realize they didn't even get to [TS]

00:53:34 ◼   ► that are interesting and other reasons [TS]

00:53:36 ◼   ► why you might be interest why you might [TS]

00:53:39 ◼   ► want this process like things that H of [TS]

00:53:41 ◼   ► us aren't even a glimmer in HFS+ is I [TS]

00:53:44 ◼   ► but that our most modern bus system cell [TS]

00:53:46 ◼   ► and I imagine all sorry F SS but we [TS]

00:53:49 ◼   ► didn't quite get to it okay so the first [TS]

00:53:51 ◼   ► one which I think I mentioned this is [TS]

00:53:53 ◼   ► these are advantages of ZFS or neat [TS]

00:53:55 ◼   ► features of ZFS S and there a lot of [TS]

00:53:56 ◼   ► them are not unique to ZFS but I'm using [TS]

00:53:58 ◼   ► it as the poster boy for a modern ish [TS]

00:54:01 ◼   ► file system first one is constant time [TS]

00:54:04 ◼   ► snapshots and clones that is a mouthful [TS]

00:54:07 ◼   ► there so I'll explain the pros also in [TS]

00:54:11 ◼   ► the middle part snapshots so a snapshot [TS]

00:54:13 ◼   ► is like the word says kind of at [TS]

00:54:17 ◼   ► capturing the state of a disk at a [TS]

00:54:20 ◼   ► moment in time and obviously we talked [TS]

00:54:24 ◼   ► about this before it's a consistent [TS]

00:54:25 ◼   ► state so the file system might be [TS]

00:54:28 ◼   ► applications might be in the middle of [TS]

00:54:30 ◼   ► doing stuff might be in the middle of [TS]

00:54:31 ◼   ► writing files out or deleting stuff or [TS]

00:54:33 ◼   ► whatever but you're going to take a [TS]

00:54:34 ◼   ► snapshot and say look at that [TS]

00:54:35 ◼   ► application might be in the null writing [TS]

00:54:36 ◼   ► that file and it might have written half [TS]

00:54:38 ◼   ► the file but as far as the file system [TS]

00:54:40 ◼   ► concerned at this moment in time it [TS]

00:54:42 ◼   ► knows where everything is it knows [TS]

00:54:43 ◼   ► exactly the size of everything [TS]

00:54:44 ◼   ► everything is all the bookkeeping agrees [TS]

00:54:46 ◼   ► with each other that half written file [TS]

00:54:49 ◼   ► it knows that that file is is one [TS]

00:54:51 ◼   ► megabyte ah maybe the file will [TS]

00:54:53 ◼   ► eventually be two megabytes when the [TS]

00:54:54 ◼   ► applications done but right now one [TS]

00:54:56 ◼   ► megabyte is on disk the metadata says [TS]

00:54:57 ◼   ► one megabyte it is a moment in time [TS]

00:54:59 ◼   ► snapshot ah [TS]

00:55:00 ◼   ► and what ZFS can do is freeze that and [TS]

00:55:05 ◼   ► say okay now at any [TS]

00:55:07 ◼   ► in the future if you ever want to see [TS]

00:55:08 ◼   ► what this file system was like at this [TS]

00:55:10 ◼   ► particular moment in time it's available [TS]

00:55:12 ◼   ► to you you can look at what it looked [TS]

00:55:14 ◼   ► like at this moment in time the constant [TS]

00:55:18 ◼   ► time part means that a time required to [TS]

00:55:22 ◼   ► take a snapshot doesn't scale with the [TS]

00:55:25 ◼   ► size of the disk so it's not like it [TS]

00:55:26 ◼   ► makes a big giant copy like you so you [TS]

00:55:29 ◼   ► have a terabyte drive when you say ok [TS]

00:55:31 ◼   ► take a snapshot doesn't say ok I'm going [TS]

00:55:32 ◼   ► to take the entire terabyte you know say [TS]

00:55:34 ◼   ► this 500 you know gigabytes on there I [TS]

00:55:36 ◼   ► take this entire 500 gigabytes and copy [TS]

00:55:38 ◼   ► it someplace else it doesn't do that [TS]

00:55:41 ◼   ► because of the way the file system is [TS]

00:55:43 ◼   ► structured it can just make a note that [TS]

00:55:46 ◼   ► this particular moment in time is [TS]

00:55:47 ◼   ► important and it should be preserved [TS]

00:55:48 ◼   ► remember that ZFS like many other file [TS]

00:55:50 ◼   ► systems avoids overriding data in place [TS]

00:55:53 ◼   ► it writes all new changes to a new [TS]

00:55:54 ◼   ► location when it does that the old [TS]

00:55:57 ◼   ► location eventually falls out of use [TS]

00:55:59 ◼   ► when no more processes have that file [TS]

00:56:03 ◼   ► open when nobody else is using it said [TS]

00:56:05 ◼   ► that was the old version of this file [TS]

00:56:06 ◼   ► the new one is way over there so now [TS]

00:56:09 ◼   ► we'll just say ok this the space [TS]

00:56:10 ◼   ► previously occupied by the old file is [TS]

00:56:12 ◼   ► free to be thrown back into the pool all [TS]

00:56:13 ◼   ► a snapshot does is tell the filesystem [TS]

00:56:17 ◼   ► hey all of these things that make up [TS]

00:56:20 ◼   ► this moment in time of the filesystem [TS]

00:56:21 ◼   ► don't throw them out even when [TS]

00:56:23 ◼   ► everybody's done with them keep them [TS]

00:56:24 ◼   ► around so the literature on ZFS will say [TS]

00:56:27 ◼   ► that this is instantaneous instantaneous [TS]

00:56:29 ◼   ► snapshot Gration it's not instantaneous [TS]

00:56:31 ◼   ► but the point is that it's constant time [TS]

00:56:33 ◼   ► that it doesn't take twice as long to [TS]

00:56:35 ◼   ► take a snapshot of a one terabyte discus [TS]

00:56:37 ◼   ► as that is a 500 gig disk because the [TS]

00:56:39 ◼   ► operation is basically a simple [TS]

00:56:40 ◼   ► bookkeeping operation of saying all the [TS]

00:56:43 ◼   ► stuff that you need with the snapshot is [TS]

00:56:44 ◼   ► already on disk all I'm telling you is [TS]

00:56:47 ◼   ► don't throw that stuff out later so it's [TS]

00:56:50 ◼   ► very quick to note that like this moment [TS]

00:56:52 ◼   ► in time it's a snapshot and you just [TS]

00:56:54 ◼   ► store a little bit of metadata about [TS]

00:56:55 ◼   ► here's a snapshot and here's you know [TS]

00:56:56 ◼   ► that here's where it is in the time [TS]

00:56:58 ◼   ► sequence of events and don't delete that [TS]

00:56:59 ◼   ► stuff because it's preserving as the [TS]

00:57:01 ◼   ► snapshot so it's really just a promise [TS]

00:57:05 ◼   ► in the future to not ditch those old [TS]

00:57:08 ◼   ► blocks that have new versions written to [TS]

00:57:10 ◼   ► them and that's something you can't even [TS]

00:57:12 ◼   ► imagine doing hfs+ because in nature I [TS]

00:57:14 ◼   ► suppose everything is written in place [TS]

00:57:15 ◼   ► and if you wanted to know what your disk [TS]

00:57:16 ◼   ► looks like in a moment in time there's [TS]

00:57:17 ◼   ► nothing available to you to do that [TS]

00:57:19 ◼   ► the only thing you could possibly do [TS]

00:57:20 ◼   ► is stopped all processes but you can't [TS]

00:57:23 ◼   ► really do butts is always something [TS]

00:57:24 ◼   ► vlogging in the background and UNIX or [TS]

00:57:25 ◼   ► whatever but try to freeze it you know [TS]

00:57:27 ◼   ► quit every application kill all [TS]

00:57:28 ◼   ► processes is that the possible minimal [TS]

00:57:30 ◼   ► maybe rebooting single user mode or [TS]

00:57:32 ◼   ► something and then make an exact clone [TS]

00:57:34 ◼   ► of this disk to a second location either [TS]

00:57:36 ◼   ► a second disk or a second partition as [TS]

00:57:39 ◼   ► you can imagine making an exact clone of [TS]

00:57:41 ◼   ► a disk takes a long time and if it takes [TS]

00:57:43 ◼   ► twice as long to make clone of a [TS]

00:57:45 ◼   ► terabyte disk as a 500 gig disk and that [TS]

00:57:47 ◼   ► amount of time is like measured in hours [TS]