57: Computational Skeuomorphism


  [Music] [TS]

  this is hypercritical weekly talkshow [TS]

  ruminating on exactly what is wrong in [TS]

  the world of Apple and related [TS]

  technologies and businesses [TS]

  nothing is so perfect that it can't be [TS]

  complained about by my co-host John [TS]

  siracusa i'm dan benjamin this is [TS]

  episode number 57 today's Friday March [TS]

  2nd want to make sure that we thank our [TS]

  two sponsors right at the top of the [TS]

  show tiny letter comm send email [TS]

  newsletters the easiest most simple way [TS]

  possible and Shopify create your own [TS]

  online store today we also want to [TS]

  mention the bandwidth for this episode [TS]

  is provided by Mac Mini Kolo I host [TS]

  Ansel the servers there to help run 5x5 [TS]

  and you can - Mac Mini Colo net / 5x5 as [TS]

  a special discount for this is why we [TS]

  need faster file systems I'm learning [TS]

  how to use computers today John how are [TS]

  you I can relaunch my apps a little bit [TS]

  faster you have all SSD you've got [TS]

  mirrored and striped SSD drives in all [TS]

  of your chips Dave nanion the guy who [TS]

  makes SuperDuper I doesn't trust the SSD [TS]

  drives typically he doesn't trust Lee [TS]

  you know supposed to trust any drive Oh [TS]

  point is I'm gonna have but he trusts [TS]

  those less than he distraught he he [TS]

  distrusts them even more than he [TS]

  distrust his other what about the people [TS]

  of emailed asking you what about SSD is [TS]

  John yeah I probably distrust me too [TS]

  well today's show will be a follow up [TS]

  show on file systems so we will talk [TS]

  about all these issues we don't have any [TS]

  other thing to talk about today no no [TS]

  iPad 3 comments or predictions maybe [TS]

  we'll throw that in at the end ok [TS]

  alright really going here now that my [TS]

  audio is actually working I would like [TS]

  to blame this audio stuff online but [TS]

  this happened in Snow Leopard too like [TS]

  you know everything's all set up and the [TS]

  last thing I do is you know your testing [TS]

  1 2 or 3 into my mic and noting the [TS]

  meters don't move at all I'm plugged mic [TS]

  plugged back in nothing so the old [TS]

  reboot this is the only time I ever [TS]

  reboot my Mac you know is when the audio [TS]

  stops working and yeah before you [TS]

  started doing the show you had an uptime [TS]

  of like three years and soon as you have [TS]

  this show you're lucky to get the full [TS]

  seven days well now as soon as I had my [TS]

  uptime was the space between security [TS]

  updates to Mac OS 10 the ones that made [TS]

  you restart [TS]

  alright let's go do it follow up I was [TS]

  listening to myself in hypercritical [TS]

  episode 56 and in the car this week as I [TS]

  always do and was cringing as I heard [TS]

  myself repeatedly used the word defer [TS]

  about deferring the costs and I wasn't I [TS]

  didn't mean deferring the costs as in [TS]

  pushing them off in time this was in [TS]

  connection to Apple's data centers and [TS]

  the cost of running them the cost of [TS]

  running iCloud and the developers using [TS]

  iCloud Hippias and no sooner had I [TS]

  gotten out of the car and logged onto [TS]

  the computer that I got a tweet from [TS]

  someone Nathan heretic I hope I got that [TS]

  right telling me that I should not be [TS]

  saying the word defer and I said I know [TS]

  I just heard myself saying in the car [TS]

  was horrible I should have said either [TS]

  offset or defray defray is probably the [TS]

  word that was getting mapped wrong in my [TS]

  brain and the point I made to him on [TS]

  Twitter is this interesting that [TS]

  listening to myself as soon as I said it [TS]

  you can hear all that's wrong with what [TS]

  is this guy talking about right but [TS]

  saying it I said it like 9 times without [TS]

  my brain triggering the warning you were [TS]

  using the wrong word [TS]

  right so I was kind of like writing and [TS]

  proofreading when I'm speaking things [TS]

  come out of my mouth that don't match [TS]

  what my brain was thinking so I [TS]

  apologize for that and thanks for the [TS]

  correction Nathan a correction from Mark [TS]

  Hofman about notifications in [TS]

  Notification Center in Mountain Lion [TS]

  we're just asking how that was one of [TS]

  the api's that was getting swept up into [TS]

  the group of things that are only [TS]

  available to Mac developers who put [TS]

  their application on the Mac App Store [TS]

  and I was saying how that was weird [TS]

  because yes notifications can be [TS]

  synchronized through the cloud from one [TS]

  device to the other and show up on your [TS]

  desktop for an event triggered on your [TS]

  phone and vice versa but notifications [TS]

  also work locally if you just have a [TS]

  local Mac application that wants to put [TS]

  up a little notification dialog says hey [TS]

  something happened over here take a look [TS]

  and they want to do something more than [TS]

  just have a dock icon badge or bounce or [TS]

  something and Notification Center is for [TS]

  that as well I said boy it's a shame [TS]

  that those guys that you can't [TS]

  use notification center for that yeah if [TS]

  you're not in the Mac App Store even [TS]

  though it has nothing to do with that [TS]

  cloud well it turns out that if you do [TS]

  only local notifications you can use [TS]

  Notification Center in Mountain Lion [TS]

  even if you're not on the Mac App Store [TS]

  so the only thing that you're barred [TS]

  from is so-called push notifications [TS]

  notifications that are synchronized [TS]

  through Apple's servers so that's good [TS]

  to know [TS]

  Andrew Lawrence was the first person to [TS]

  chime in to tell me that the NetApp [TS]

  Oracle lawsuit that I mentioned as a [TS]

  possible reason that Apple bailed on ZFS [TS]

  was settled shortly after son was [TS]

  acquired by Oracle I put a link to the [TS]

  press release in the show notes it was [TS]

  actually September 2010 that was still [TS]

  after long after Apple had decided had [TS]

  made his decision on ZFS oh I'm not sure [TS]

  settling helped to hurt one with the [TS]

  other but by then it was too late but [TS]

  it's a good point that it actually has [TS]

  been settled and it's not dragging on [TS]

  the settlement was one of those ones [TS]

  where it's you know dismissed without [TS]

  prejudice and the terms of the agreement [TS]

  are confidential so we don't know what [TS]

  actually happened they just probably [TS]

  exchanged a sufficient amount of money [TS]

  between parties and everything's all [TS]

  better JP I don't know I just have JP [TS]

  maybe that's all I could get out of his [TS]

  Twitter handle or email or something was [TS]

  the first to point out that btrfs is [TS]

  pronounced butter FS I had heard that [TS]

  enunciation but wasn't aware it was the [TS]

  official apparently it is the official [TS]

  pronunciation I don't I think I don't [TS]

  really like the idea of food mixing with [TS]

  my file systems so butter if that sounds [TS]

  a little weird to me but if it's butter [TS]

  FS then it's butter FS so I will [TS]

  endeavour to say butter FS from now on [TS]

  JP has some other information he [TS]

  apparently this is why he just says JP [TS]

  hey he worked in the Genius Bar for a [TS]

  while in an Apple Store and here's a [TS]

  quoting from his message we got almost [TS]

  as many okay hard drives according to [TS]

  smart status which were unrepairable in [TS]

  Disk Utility as we did actual fail to [TS]

  harden so what he's what he's saying is [TS]

  that you know people would come in to [TS]

  the Genius Bar and they would have a [TS]

  problem with their hard drive and half [TS]

  the time there was a hardware problem [TS]

  with the hard drive but the other half [TS]

  of the time it was just hfs+ corruption [TS]

  so [TS]

  fully 50% of their hard drive related [TS]

  problems were not due to hardware but [TS]

  were due to HFS corruption and he says [TS]

  all in all hard drive issues accounted [TS]

  for about 30% of all issues so about 15% [TS]

  for a directory failure so he's saying [TS]

  15% of all problems that come to the [TS]

  Genius Bar and his experience were due [TS]

  to hfs+ corruption and across all of [TS]

  apples customers this is a pretty big [TS]

  number I'm actually surprised that I'm [TS]

  surprised that the 30% of all Genius Bar [TS]

  t-shirts were hard drives a hard drive [TS]

  related but it's really said that half [TS]

  of the HOS 15% of all possible Guinness [TS]

  bar things are due to HFS+ corruption so [TS]

  it shows that this is really is a [TS]

  problem an apple I mean apples gotta [TS]

  track these stats and has to know geez [TS]

  you know 50 percent of the people come [TS]

  to the Genius Bar is due to bugs in our [TS]

  own software we really need to address [TS]

  that filesystem thing a couple people I [TS]

  don't remember who sent this might have [TS]

  been the chat room but I don't have a [TS]

  attribution for it send me a link to an [TS]

  lwn net article called a short history [TS]

  of butter FS and that I hadn't known too [TS]

  much of a butter of has except that it [TS]

  was the kind of like a you know the GPL [TS]

  Linux directed ZFS like filesystem and I [TS]

  also knew that it wasn't quite done yet [TS]

  so both of those are true but this this [TS]

  article that it linked in the show notes [TS]

  is written by someone who worked [TS]

  extensively on ZFS and it goes through [TS]

  the history of how butter FS came to be [TS]

  and what's different about it than ZFS I [TS]

  think it's worth I guess all worth [TS]

  hitting the highlights of instead of [TS]

  just saying go read this article because [TS]

  it is a long article and maybe if you're [TS]

  not really into this you won't read the [TS]

  whole darn thing [TS]

  so the be actually the be TR part in [TS]

  butter FS stands for be trees it's [TS]

  shorter be trees have been so it's [TS]

  another victim of the great valve sort [TS]

  shortage of 1972 which is stricken the [TS]

  UNIX landscape and Keaton used to be a [TS]

  problem they didn't even have an e [TS]

  leftover for the create system call [TS]

  that's how bad it was [TS]

  but apparently it's still going on so a [TS]

  file systems use b-trees because they're [TS]

  an efficient data structure for looking [TS]

  stuff up and [TS]

  starting stuff and deleting stuff if you [TS]

  take any computer science course you'll [TS]

  eventually do a chapter on trees and [TS]

  they're all different kinds of trees and [TS]

  you can see in the diagram of them it's [TS]

  like a circle with a bunch of lines [TS]

  connecting and there's you know out of [TS]

  each little circle there are you know [TS]

  one or more lines and then there are [TS]

  circles connected those and it just [TS]

  makes a big tree structure bee trees are [TS]

  a particular kind of tree that you know [TS]

  the characteristics of trees are how [TS]

  many how many different branches come [TS]

  out of each one how do you insert [TS]

  something into the tree how do you [TS]

  remove something from the tree I'm not [TS]

  going to go into the sto details of bee [TS]

  trees in particular but suffice it to [TS]

  say that B trees are specific variant of [TS]

  B trees are among the most efficient [TS]

  general-purpose tree structure for the [TS]

  you know they do everything pretty well [TS]

  you can insert things pretty quickly you [TS]

  can delete things pretty quickly they [TS]

  don't get unbalanced they don't require [TS]

  a lot of maintenance to keep balanced [TS]

  ever it's kind of like a good middle [TS]

  ground for the types of things that you [TS]

  might want to keep track of in a file [TS]

  system you do because you don't want to [TS]

  be something to be really quick to read [TS]

  from but really slow to write because [TS]

  file systems tend to do all that stuff [TS]

  read write and delete in and generally [TS]

  you know equal measures but B trees are [TS]

  not particularly good when it comes to [TS]

  the strategy the ZFS implements where [TS]

  they never overwrite data in place you [TS]

  know that's the this is the [TS]

  copy-on-write strategy where if you have [TS]

  new data to write write in to a new [TS]

  location [TS]

  rather than writing over on top of the [TS]

  open and if you want a mod that's to put [TS]

  the copy-on-write means and saying if [TS]

  it's time for you to write some data [TS]

  instead of overwriting the data find [TS]

  where that data is and then make a new [TS]

  copy of it somewhere else it's different [TS]

  than the old one but B trees are not [TS]

  compatible with that at all because if [TS]

  you were to naively implement copy and [TS]

  write with the B tree then anytime you [TS]

  changed anything the entire tree from [TS]

  top to bottom would have to be changed [TS]

  because the changes would propagate back [TS]

  up the tree because you're making a new [TS]

  instance so almost all B trees and file [TS]

  systems and by the way HFS+ and H of s [TS]

  use B trees extensively B trees and file [TS]

  systems tend to be updated in place [TS]

  because that preserves all the desirable [TS]

  performance characteristics of a B tree [TS]

  and once you start saying oh we can't [TS]

  update anything in place we always had [TS]

  to write a new dated a new location and [TS]

  B trees seemed useless so many years ago [TS]

  someone named Oh had Road [TS]

  a I don't know if I got this name right [TS]

  OHA D ro deh wrote an academic paper as [TS]

  people in academia tend to do about a [TS]

  new kind of bee tree that was friendly [TS]

  to copy-on-write he found a way that if [TS]

  you all you change bee trees in this way [TS]

  then we actually can get reasonably good [TS]

  performance characteristics while still [TS]

  not ever updating data in place and this [TS]

  academic paper probably sat around for a [TS]

  while until someone named Chris Mason [TS]

  found this and said hey if I have a bee [TS]

  tree structure that's also friendly - [TS]

  copy-on-write maybe I can use that in a [TS]

  new file system and his big insight for [TS]

  creating a new file system was to decide [TS]

  that everything in the file system will [TS]

  be stored in one of these copy-on-write [TS]

  friendly bee trees normally you have bee [TS]

  trees first restoring like oh I'll keep [TS]

  track of directory entries with the bee [TS]

  tree but when I'll have a separate data [TS]

  structure for keeping track of free [TS]

  space like a bitmap or something and [TS]

  I'll have a separate structure for [TS]

  keeping track of extensors you know it [TS]

  there are usually different kinds of [TS]

  data structures tailored for each [TS]

  purpose he decided well this data [TS]

  structure looks so great what if [TS]

  everything is stored in one of these [TS]

  copy IRB trees I'll star the directory [TS]

  entries the the free space lists the [TS]

  extents that the file data itself [TS]

  everything in one of these bee trees and [TS]

  that's a nice unifying concept because [TS]

  it means that you only really need to [TS]

  have one code path once you get your [TS]

  code for reading writing and modifying [TS]

  these bee trees like down pat and [TS]

  debugged you're all set you don't have [TS]

  to say okay well what about the code [TS]

  that manages the free space and what [TS]

  about the code that manages the direct [TS]

  Rangers well you not not 17 different [TS]

  algorithms and 17 different data [TS]

  structure just one thing and you know so [TS]

  the code is simpler smaller and a nicely [TS]

  uniform and programmers love this type [TS]

  of thing so that's the basis of btrfs [TS]

  building an entire file system on this [TS]

  this new data structure and that's very [TS]

  interesting to me I didn't I didn't know [TS]

  this before I had read it definitely [TS]

  hits all of my buttons of her computer [TS]

  science and programmers like oh it's [TS]

  beautiful and elegant and has great [TS]

  performance characteristics and has all [TS]

  these features of ZFS the article goes [TS]

  into much more detail about how ZFS and [TS]

  btrfs [TS]

  are the same and how they differ they [TS]

  both kind of arrived similar [TS]

  from different directions and they do [TS]

  have different advantages and [TS]

  disadvantages or asset each other so I [TS]

  encourage everyone to actually follow [TS]

  that link and read it is not that long [TS]

  and it's not too jargon inducing it's a [TS]

  short history of BTR oh I almost added a [TS]

  butter fs and we will be revisiting this [TS]

  topic in a bit once we get through the [TS]

  official follow up Lenny P Robert writes [TS]

  to tell me that there is a program that [TS]

  will look through your HFS+ volume and [TS]

  try to detect if there are any errors [TS]

  and how does it do this it check sums [TS]

  everything and writes a bunch of little [TS]

  checksum files to each directory and [TS]

  then the next time you ask the check it [TS]

  says ok I have this file here this has [TS]

  the checksum supposed to be blah [TS]

  I will recheck some all these files and [TS]

  see if it matches a lot of people were [TS]

  asking me hey I'm afraid that my data is [TS]

  going bad on my HFS+ volumes how do I [TS]

  tell and I did I had wasn't aware of [TS]

  this program before but other people [TS]

  wrote in with it so I replied so this is [TS]

  one of the tools available to you now if [TS]

  you think about it for too long you [TS]

  realize that this program which is [TS]

  called integrity checker by the way it's [TS]

  kind of in a catch-22 because it's [TS]

  storing the check sums on the same disk [TS]

  that you're not trusting to make sure [TS]

  your data is ok you know what I mean so [TS]

  if you have a bit error and it happens [TS]

  to be an error in the checksum file then [TS]

  it would look like all your data is [TS]

  corrupted and really it's the checksum [TS]

  that's corrupted so the very least this [TS]

  might tell you something is wrong what [TS]

  exactly is wrong what you can do but I [TS]

  don't know but you could use this for [TS]

  example to keep proving to yourself that [TS]

  your backups are the same as they were [TS]

  the last time you know like I haven't [TS]

  modified anything let me look at my time [TS]

  machine back up as of three months ago [TS]

  is that data still exactly what it was [TS]

  when I wrote it three months ago [TS]

  or is there something wrong with it when [TS]

  there is something wrong you may not be [TS]

  able to tell exactly what's wrong but at [TS]

  least you know your data is bad so I [TS]

  have a link to this product in the show [TS]

  notes I have not tried it myself I don't [TS]

  even know how much it costs [TS]

  but for people who are paranoid about [TS]

  data integrity it's it's seems that [TS]

  third parties are taking up the mantle [TS]

  of attempting to do something about this [TS]

  problem as you might imagine anything [TS]

  that's going to grind over your whole [TS]

  disk and checksum every single thing and [TS]

  check their the check sums are the same [TS]

  it's going to be tremendously i/o [TS]

  intensive and it probably takes a really [TS]

  long time that's what you get when you [TS]

  have to do it on top of an existing file [TS]

  system instead of having it built in and [TS]

  Gregory Williams pointed me to another [TS]

  third-party product this one allows you [TS]

  to tap into HFS+ native compression that [TS]

  was introduced in Snow Leopard I've [TS]

  talked about in last show how there was [TS]

  no like UI for that in the in the OS and [TS]

  Apple decided to compress a whole bunch [TS]

  of files sort of out of the box but you [TS]

  didn't really have any control over that [TS]

  well this is a preference pane called [TS]

  clusters from late night software and [TS]

  with it you can tell it hey any files [TS]

  under a particular folder flip that [TS]

  little HFS+ compression bit and compress [TS]

  them for you and it's complete [TS]

  transparent encryption you don't know [TS]

  they're encrypted there's no like [TS]

  decryption phase that you can see it all [TS]

  happens under the covers in the file [TS]

  system so you can save some space with [TS]

  this now don't turn this on for a folder [TS]

  full of QuickTime movies or something [TS]

  because they're already compressed and [TS]

  you're not going to get much more out of [TS]

  them if you have some easily [TS]

  compressible data and you would like to [TS]

  have it transparently compressed for you [TS]

  this may be the ticket on the other hand [TS]

  it's possible that there's a good reason [TS]

  that Apple has not exposed the HFS+ [TS]

  compression in the user interface so I [TS]

  think this type of software is the type [TS]

  of thing that exists on the borderline [TS]

  between what Apple what Apple's [TS]

  operating system is capable of and what [TS]

  they recommend third-party developers [TS]

  use and we will touch on that again [TS]

  actually later in the show and I also [TS]

  put a link in the show notes to the [TS]

  section about hfs+ compression in my [TS]

  snow leopard review from 2009 alright so [TS]

  that's the end of the official follow-up [TS]

  and now we begin the unofficial fall off [TS]

  which is the main topic of the show [TS]

  which is more [TS]

  files isms because believe it or not the [TS]

  end of the last show I thought of 20 [TS]

  things that I didn't talk about related [TS]

  to file systems and a lot of people [TS]

  asked about them hmm well let's do our [TS]

  first sponsor and then we'll kick that [TS]

  part off what do you think sounds good [TS]

  our first sponsor is a longtime sponsor [TS]

  shopify calm love these guys [TS]

  very simply I've told you about them [TS]

  before create your online store today [TS]

  with Shopify they make it super simple [TS]

  to make your own store and you can sell [TS]

  whatever you want one of the questions [TS]

  that I had people were asking me about [TS]

  this so well that's perfectly fine if [TS]

  you're selling you know physical goods [TS]

  if you're selling a t-shirt or you're [TS]

  selling electronics or you have a [TS]

  Kickstarter project that you know you [TS]

  need the storefront for it what if I [TS]

  want to sell something electronic what [TS]

  if I wrote a book you know and I want to [TS]

  sell the PDF of it or what if I have [TS]

  something that I want to control down [TS]

  yes of course you can do that with [TS]

  Shopify dot-com there are tons of [TS]

  add-ons and tons of little things that [TS]

  that people have written it for exactly [TS]

  this kind of thing in fact my friend [TS]

  Jeffrey's elderman they have a company [TS]

  called a book apart and all of the books [TS]

  that they do for designers and [TS]

  developers [TS]

  that's a Shopify store and they [TS]

  customized the heck out of it you [TS]

  wouldn't even recognize it [TS]

  it's totally so here's the deal normally [TS]

  you go in there and you sign up on [TS]

  Shopify com you get 30 days free use the [TS]

  coupon code 5x5 when you're signing up [TS]

  you'll get three months free and of [TS]

  course you can use the site and sell [TS]

  everything you want within that that [TS]

  time frame of course you can so go there [TS]

  today Shopify com coupon code 5x5 and [TS]

  join more than 20,000 stores selling [TS]

  their stuff on Shopify John himself [TS]

  you're even opening up a store I am NOT [TS]

  gonna be selling USB headsets [TS]

  all right already main topic yes the [TS]

  main topic is actually inspired by [TS]

  follow-up and you mentioned at the very [TS]

  top of the show because you've been [TS]

  seeing the same emails as I have [TS]

  everybody was asking in the last show [TS]

  all you know so I listened your show [TS]

  about file systems how does all this [TS]

  change in light of solid state disks [TS]

  that is the question of the week ah and [TS]

  I didn't talk about it in last show so [TS]

  we'll talk about it now I'll use it as a [TS]

  jumping-off point to talk more about [TS]

  these file systems in general one of the [TS]

  reasons I didn't talk about it is [TS]

  because there's no clear answer how does [TS]

  it change and it's not even we can't [TS]

  even say it changes completely or not at [TS]

  all it's somewhere in the middle so I'm [TS]

  trying to think of things that file [TS]

  systems do that clearly have spinning [TS]

  disks in mind as a way to get into the [TS]

  topic of how that might change with SSDs [TS]

  so the biggest one I could think of them [TS]

  are you going to go some more is putting [TS]

  related information close together on [TS]

  the disk [TS]

  because the big thing you want to avoid [TS]

  on a spinning disk is moving the disk [TS]

  head that is by far the most time [TS]

  consuming thing you have to do because [TS]

  especially with you know diskette disks [TS]

  are getting faster but the speed with [TS]

  which you can move a disk head from one [TS]

  place to another on a disk has not been [TS]

  getting faster nearly as much as the [TS]

  size of disks have been increasing you [TS]

  know because you have to move the little [TS]

  head and then wait for it to stop it [TS]

  exactly over some microscopic little [TS]

  track on the disk and then wait for it [TS]

  to stop wiggling back and forth and sort [TS]

  of settle down so we can read again and [TS]

  they can do that amazingly fast it's [TS]

  like a seek times like nine milliseconds [TS]

  or something like that but it's still [TS]

  super slow compared to how fast they can [TS]

  stream data off a single location on [TS]

  disk so putting stuff together on a [TS]

  spinning disk is really important you [TS]

  can see this yourself I think regular [TS]

  users have some idea about this when you [TS]

  have a like a 100 megabyte folder and [TS]

  but it's filled with thousands and [TS]

  thousands of files there are a few bytes [TS]

  each how long does it take to duplicate [TS]

  that versus how long does it take [TS]

  duplicate a single hundred megabyte [TS]

  phile if your disc is not full and you [TS]

  have lots of free space duplicating a [TS]

  single hundred megabyte file can be [TS]

  really fast amazingly fast basically if [TS]

  the hundred megabyte file is kind of all [TS]

  all the data of it is kind of in this [TS]

  similar place on the disk you don't need [TS]

  to do many seeks you read long sequences [TS]

  of bytes off of the disk before you have [TS]

  to move the disk head to get the rest of [TS]

  the bytes that's really fast but moving [TS]

  thousands and thousands of little files [TS]

  even the total amount of data is the [TS]

  same to get it thousands of thousands [TS]

  little files well you have to go read [TS]

  the directory entry and that's imagine [TS]

  this is hfs+ to go to go to the catalog [TS]

  file which is in a central location so [TS]

  you read the directory entry and then [TS]

  the catalog file points you'd okay [TS]

  here's the first file and here's what [TS]

  it's called and here's what the bytes [TS]

  are so move this can get the bytes for [TS]

  that file and that's really quick [TS]

  because there's only a few bytes and [TS]

  then write the file to the new location [TS]

  so make a new directory entry that's [TS]

  back to the catalog file and write the [TS]

  name of the file there and then write [TS]

  the bytes for the file okay let's write [TS]

  those flights with a file someplace else [TS]

  and dusk you've done like six seeks and [TS]

  all you've copied is like two bytes of [TS]

  data like if this was a tiny little file [TS]

  in fact most your time might have been [TS]

  spent just writing the metadata about [TS]

  the files name the date you know all [TS]

  these other other information but you're [TS]

  seeking all over the disk get a little [TS]

  bit of data to go back to the catalog [TS]

  file write the new location write the [TS]

  data it takes forever because seeking [TS]

  time starts to dominate how long the [TS]

  entire operation takes whereas the [TS]

  actual writing data to disk when you [TS]

  look at the stats on a disk like into a [TS]

  30 megabytes per second or 40 megabytes [TS]

  per second yeah that's how much you can [TS]

  write if the disk head is pretty much [TS]

  not moving and the disk is spinning [TS]

  underneath it once you start moving the [TS]

  diskette around you're spending all your [TS]

  time moving and very little time [TS]

  actually writing data so the big thing [TS]

  that file systems try to do in in the at [TS]

  all times but especially in the modern [TS]

  age is no disk seeks put related stuff [TS]

  close together pack it all next to each [TS]

  other don't make the disk ad move around [TS]

  so how does that relate to SSDs the big [TS]

  thing with us is these is you know [TS]

  there's no moving parts there's no disk [TS]

  kid moving back and forth trying to grab [TS]

  little pieces just it's just a bunch of [TS]

  memory chips and the naive view of [TS]

  memory checks is like oh I can get at [TS]

  any part of those memory chips instantly [TS]

  I don't have to move anything anywhere I [TS]

  just say give me byte number 56 and I [TS]

  get it give me byte number 3000 and I [TS]

  get it and asking for byte 56 directly [TS]

  after I asked for but 3000 it doesn't [TS]

  make any difference right [TS]

  there's no there's nothing to move well [TS]

  flash memory unfortunately is not as [TS]

  simple as the naive view of memory might [TS]

  be there are some limitations and the [TS]

  limitations are different for flash [TS]

  memory of different kinds a different [TS]

  for DRAM differing for all sorts of [TS]

  memory chips the stuff that goes on [TS]

  inside the memory chips we like to think [TS]

  of it as a black box that just stores [TS]

  our bits and gives us back but it's [TS]

  really fiendishly complex and you don't [TS]

  know how for example DRAM works I [TS]

  encourage you to go to Wikipedia and [TS]

  read about how DRAM works you will [TS]

  probably be horrified and amazed at any [TS]

  Forks at all pieces it's terrifyingly [TS]

  scary flash is similarly terrifying it [TS]

  has these crazy limitations that don't [TS]

  make any sense until you figure out how [TS]

  like the chips are actually laid out and [TS]

  how the physics work and the economics [TS]

  of fabbing and why they're made this [TS]

  play I'll try to give a reasonable [TS]

  high-level summary of it but the short [TS]

  answer is that it does help to have [TS]

  things near each other when when using [TS]

  flash memory because although it doesn't [TS]

  take any time - there's nothing to seek [TS]

  from one location to the other flash [TS]

  memory cannot be arbitrarily updated in [TS]

  exactly the size pieces that you want [TS]

  there are larger chunks that you know [TS]

  you can't just say okay I want I'm going [TS]

  to write two bytes here I'm going to [TS]

  write one byte there and about 1 byte [TS]

  over there and everything's hunky-dory [TS]

  if the flash chips themselves tend to [TS]

  have to do operations in larger evenly [TS]

  sized chunks than you would want so that [TS]

  might mean erasing an entire block we [TS]

  are reading the entire contents of [TS]

  entire block erasing the whole thing and [TS]

  writing it back plus the change instead [TS]

  of just writing your change I tried to [TS]

  highlight the one part that's relevant [TS]

  most relevant from the Wikipedia page to [TS]

  a NAND flash memory that's moved in most [TS]

  flash drives and the upshot according to [TS]

  Wikipedia and I don't know enough about [TS]

  it to know whether this is accurate but [TS]

  I assume it probably is because it's [TS]

  just a technical matter is that it says [TS]

  you can erase memory a block at a time [TS]

  but so you can read it in a little [TS]

  pieces and you can write it in little [TS]

  pieces but you can only erase it a block [TS]

  at the time and the next wrinkle to that [TS]

  is that [TS]

  when you erase you set all the bits but [TS]

  when you set them you can only clear [TS]

  bits so if you're setting something in a [TS]

  region that has all ones in it all you [TS]

  got to do is blank out the ones that are [TS]

  zeros to make your bit patterns so if [TS]

  you're writing one zero one zero and [TS]

  you've got one one one one [TS]

  you just blank out those two ones and [TS]

  voila one zero one zero you just wrote [TS]

  only the amount of data you needed ah [TS]

  but when you erase it you have to erase [TS]

  the entire thing the entire block and a [TS]

  block is much bigger than the region [TS]

  you're just interested in updating and [TS]

  since you can only clear disks if a [TS]

  region has more zeros than the data [TS]

  you're about to write there like I can't [TS]

  I can't write this because I all I can [TS]

  do is change things it all I can do is [TS]

  change things to zeros but it's already [TS]

  got too many zeros I can't change that [TS]

  zero to a one [TS]

  so if I can't change a zero to one my [TS]

  only option is to erase this entire [TS]

  block set it all to ones we're going to [TS]

  read the block first erase the entire [TS]

  block and write the entire block back [TS]

  plus my changes there are probably [TS]

  better articles about this and someone [TS]

  hasn't been the show at Stan and tech I [TS]

  was trying to Google for better links to [TS]

  this before I started so I will add this [TS]

  one they came up in the show notes we [TS]

  link to the print only version of it but [TS]

  I guess I'll throw them in there so it [TS]

  means the ETL details don't matter but [TS]

  the point is that operations to flash [TS]

  memory don't necessarily only modify the [TS]

  bits that you're interested in sometimes [TS]

  they have to modify entire large blocks [TS]

  of bits even if you're only interested [TS]

  in changing a small portion of so that [TS]

  argues for trying to make your data [TS]

  that's related close to each other [TS]

  otherwise you're going to spend your [TS]

  time erasing an entire block of data [TS]

  just to put one byte there and then [TS]

  erase it all totally unrelated block [TS]

  about to put to put another byte there [TS]

  now I'm not sure if this means that [TS]

  putting stuff close together is like a [TS]

  big win on flash certainly it's not as [TS]

  big a win as it is on spinning disks but [TS]

  at the very least it probably does no [TS]

  harm and there are reasons to think that [TS]

  it might be beneficial like these same [TS]

  file system layouts that put things [TS]

  close together would also benefit flash [TS]

  memory because then [TS]

  that's it do operations that modify more [TS]

  of the block the whole block rhythm I [TS]

  need to erase but the flipside of that [TS]

  is if there is a file system that goes [TS]

  through great lengths and bends over [TS]

  backwards and has a really complicated [TS]

  structure completely aimed at minimizing [TS]

  seeks it's like all we just got to pack [TS]

  everything together and we have to do [TS]

  all this complicated stuff and the code [TS]

  is complicated and the i/o scheduling is [TS]

  complicated also it can avoid seeks it [TS]

  could be that that complexity is not [TS]

  worth the benefit that you get in SSDs [TS]

  and a file system that doesn't go to [TS]

  such great lengths to make sure data is [TS]

  as close together but is much simpler [TS]

  much more reliable and as fewer bugs or [TS]

  whatever might be fine on an SSD because [TS]

  the things that hurt it on spinning disk [TS]

  don't hurt it as much there so that's [TS]

  not a very good answer the answer is [TS]

  basically it depends and and I don't [TS]

  think anyone is really sure and all the [TS]

  file systems that I've been talking [TS]

  about with the possible section of [TS]

  session of butter FS were conceived in a [TS]

  time when people didn't realize that [TS]

  SSDs would be the phenomenon that they [TS]

  are as ZFS is but I think it's close to [TS]

  ten years old now it's pretty old and [TS]

  back then yes SSDs existed and they [TS]

  certainly didn't want to do anything [TS]

  that would be silly on an SSD and I [TS]

  don't think they did but they wasn't [TS]

  designed for an SSD many of the [TS]

  considerations in the file system design [TS]

  were based on spinning disks and butter [TS]

  FS looks a lot like ZFS but it's newer [TS]

  so they may have more SSD stuff in mind [TS]

  certainly there is no file system that I [TS]

  know of that's in widespread use it was [TS]

  designed from the ground up to say [TS]

  forget spinning disks in fact if you run [TS]

  this on a spinning disk it might be [TS]

  hideous and terrible this is entirely a [TS]

  ended SSD Boyd Watters wrote in to say [TS]

  that to quote him plopping NAND flash [TS]

  behind a SATA interface is stupid so [TS]

  what he's put is showing is that thing [TS]

  of the storage hierarchy that exists in [TS]

  current things you got the CPU register [TS]

  file you got the the caching on the CPU [TS]

  then you've got the RAM then you've got [TS]

  the the i/o controller chip then you've [TS]

  got the the SATA interface and then [TS]

  you've got this flash disk and you put [TS]

  discs and [TS]

  scare quotes because it's not actually a [TS]

  disk why does that big structure exist [TS]

  why do these flash memory chips have to [TS]

  pretend they're a spinning disk does [TS]

  that seem like a silly kind of archaic [TS]

  it's kind of like a computational [TS]

  skeuomorphism where it's pretending to [TS]

  be a disk because everything expects [TS]

  there to be a disk there but it's [TS]

  totally not this is an interesting issue [TS]

  that has come up many times in the in [TS]

  the past decades like why do we had disk [TS]

  and memory wouldn't it be nice if [TS]

  everything was just memory that's the [TS]

  you know the computer science dream [TS]

  everything is just a big open field of [TS]

  bits and it can be addressed randomly [TS]

  written land rent they write or read [TS]

  randomly and there's no difference [TS]

  between something's in disk and it's in [TS]

  memory just one big continuum one way to [TS]

  do that would be to have notice can just [TS]

  have everything in memory because then [TS]

  everything is memory right but that's [TS]

  generally not feasible because memory is [TS]

  measured in you know gigabytes and disk [TS]

  space is memory measured in terabytes so [TS]

  they're not close and the other thing is [TS]

  just less I have a unified interface [TS]

  that yes it's flat flash is slower than [TS]

  then DRAM but the Ihram is also slower [TS]

  than l3 cache which is slower than l2 [TS]

  cache which is slower than l1 cache [TS]

  which is slower than registers it's a [TS]

  hierarchy of memory how close quote [TS]

  unquote close is this memory to the CPU [TS]

  how long does it take the CPU to get to [TS]

  it that hierarchy exists and [TS]

  traditionally if you were to graph how [TS]

  you know how long does it take to get a [TS]

  piece of memory out of l1 cache l2 cache [TS]

  l3 cache it's like a hockey stick graph [TS]

  and especially when you hit disk it's [TS]

  like well now you might as well just go [TS]

  on vacation because that that is not [TS]

  going to be coming back for another 8 [TS]

  billion cycles of the CPU so forget it [TS]

  flash memory is faster than just what [TS]

  that wall is still there that the gap [TS]

  between the speed of memory is still [TS]

  there but still the idea is that why not [TS]

  just have this is a uniform storage [TS]

  hierarchy with uniform interface and not [TS]

  have disk i/o to be this totally [TS]

  separate thing that's something that [TS]

  someone like Apple is not going to [TS]

  probably tackle anytime soon but it's [TS]

  been in the air [TS]

  probably since the 60s or 70s the idea [TS]

  that a unified interface to memory would [TS]

  be better than and probably a lot of old [TS]

  machines did have a unified interface [TS]

  but in the end the person computer age [TS]

  the split weakened disk in memory is [TS]

  pretty baked into the way that we do [TS]

  things so let's consider what might [TS]

  change if everything was unified well [TS]

  one thing that would have to change is [TS]

  that the operating system would have to [TS]

  be different because in the operating [TS]

  system the file IO calls are very [TS]

  different from the calls to manipulate [TS]

  memory file alcohols you've got you know [TS]

  opening files reading writing files [TS]

  they're just structured differently take [TS]

  different arguments have different [TS]

  semantics than the memory calls which [TS]

  are you know setting memory getting [TS]

  memory copying memory and just bit [TS]

  operations that happen in memory in two [TS]

  entirely different api's so you couldn't [TS]

  just say ok memory and disk or all one [TS]

  because everyone's programs wouldn't [TS]

  work anymore because that's you know [TS]

  you'd have to you'd have to make some [TS]

  sort of shim layer there and ideally [TS]

  you'd want one unified interface to a [TS]

  new set of api's that looks the same for [TS]

  both of those things and what does that [TS]

  new wear interface look like and who is [TS]

  going to adopt that new integration [TS]

  who's going to design it that's a big [TS]

  problem one thing that would go away are [TS]

  these really complicated api's that we [TS]

  have now for memory mapping files where [TS]

  you say look I want to manipulate this [TS]

  really big file but I want to read every [TS]

  single byte of it off disk and I want to [TS]

  manipulate it as if it's all in memory [TS]

  but I can't fit it all in memory so can [TS]

  i memory map the file and memory mapping [TS]

  will automatically make it look like [TS]

  your entire files in memory but just [TS]

  pulling the parts that you need on [TS]

  demand and that's a very complicated [TS]

  system and the API is very complicated [TS]

  that's kind of stuff would go away if [TS]

  you had a single unified interface to [TS]

  the entire storage hierarchy or maybe [TS]

  you would just shift and that becomes [TS]

  part of this unified interface but it's [TS]

  a complexity that exists now because we [TS]

  know that files are much bigger than [TS]

  memory we know we can't fit an entire [TS]

  file in memory so we have to make this [TS]

  weird illusion that it's kind of in [TS]

  memory when it's not ah a big one that I [TS]

  thought of is that you'd have to [TS]

  reconsider limits limits that we take [TS]

  for granted now so for example when you [TS]

  run out of disk space that's a dead end [TS]

  right now [TS]

  no space left on device right that's [TS]

  that's the end you know this there's no [TS]

  program in existence that says when I [TS]

  run out it's like what can it what kind [TS]

  of program do well jeez on that it can [TS]

  tell you I try to do that for you but [TS]

  your disk is full so and and by the way [TS]

  UNIX does not deal well with the disk [TS]

  being full for reasons we'll probably [TS]

  get into in a bit later uh and you don't [TS]

  want you just to be complete full lots [TS]

  of bad things happen but running out of [TS]

  RAM waiting it what do you say UNIX [TS]

  doesn't handle it well what does handle [TS]

  it well or is that is that part of that [TS]

  next topic yeah that what does handle it [TS]

  well in it in the days before UNIX for [TS]

  example in classic Mac OS in like really [TS]

  classic Mac OS like you know system 4.2 [TS]

  you could have your disk filled up to [TS]

  the brim and everything would still [TS]

  pretty much work okay you know you [TS]

  couldn't save stuff but you wouldn't be [TS]

  using the thing and it would just blow [TS]

  up whereas the UNIX uses the disk space [TS]

  for all sorts of things and if your disk [TS]

  even gets close to full lots of things [TS]

  they're going to start falling down not [TS]

  just the program you're using so you [TS]

  can't just happily use your Mac with an [TS]

  entirely full disk in fact I don't even [TS]

  know if you can get your your disk [TS]

  entirely full without sort of cheating [TS]

  by putting it to single user mode and [TS]

  using DD to fill up the disk I don't [TS]

  even think you can boot with it with an [TS]

  entirely full disk once you get anywhere [TS]

  near close to disk full Mac os10 starts [TS]

  throwing dialogues in your faces said [TS]

  this is almost full seriously delete [TS]

  stuff you've got a problem and there's a [TS]

  good reason for that uh but in the [TS]

  modern era running out of RAM is not a [TS]

  dead end and I say the modern era being [TS]

  like Mac OS 10 certainly but even in the [TS]

  latter days of classic Mac OS what [TS]

  happens when we run out of RAM is we're [TS]

  not around all the time ah [TS]

  you run out of RAM and you swap some [TS]

  stuff to disk so you take some stuff [TS]

  that's in memory though I don't have [TS]

  room to put everything in memory that [TS]

  need to put in a memory let me take [TS]

  something that's in memory or write it [TS]

  out to this temporarily now I've read up [TS]

  some new room in memory and it tries to [TS]

  put stuff on to disk [TS]

  tries to swap stuff out that hasn't been [TS]

  used in a while and this is the basis of [TS]

  all modern virtual memory systems with [TS]

  swap where they will [TS]

  try to let it appear to your programs as [TS]

  if you have way more RAM than you do so [TS]

  programs back the old days and 32-bit [TS]

  thing they would think that they had you [TS]

  know two to four gigabytes of RAM when [TS]

  maybe your computer only at 128 [TS]

  megabytes of RAM and the program was [TS]

  blissfully unaware of that it would just [TS]

  try to address memory in the operating [TS]

  system every time you filled up the [TS]

  actual RAM chips [TS]

  it would just swap things in and out to [TS]

  disk to make enough room for the stuff [TS]

  you currently using in ram now swapping [TS]

  is bad and it's slow because if your [TS]

  program thinks something's in memory and [TS]

  it asks the operating system for on the [TS]

  operating system says actually that's [TS]

  not in memory I had to put that in disk [TS]

  for you but I'll wait I'll go get it on [TS]

  the disk you just wait there it takes [TS]

  forever for the operating system to come [TS]

  back so here's this thing and the [TS]

  program expected that to come back [TS]

  really quickly so when your computer is [TS]

  thrashing or swapping or one of these [TS]

  other terms it means that it's trying to [TS]

  access a bunch of stuff that it thinks [TS]

  is in memory but a lot of it turns out [TS]

  to be on disk and your disk ad they're [TS]

  seeking all around to pull this stuff [TS]

  off of the disk and putting other stuff [TS]

  onto this because if it's on if it's on [TS]

  disk has to pull it out maybe that's to [TS]

  evict something else from RAM and put it [TS]

  down there so that's situation we're in [TS]

  now where a disk space fills up total [TS]

  dead end RAM fills up it's not a dead [TS]

  end but how is RAM filling up not a dead [TS]

  it uses disk space and that's why if [TS]

  your disk is full everything falls down [TS]

  because all these programs think that [TS]

  they have you know much more memory than [TS]

  they really have and if they did blondie [TS]

  continued to put stuff in memory in the [TS]

  operating system says well that actually [TS]

  can't fit in memory so what I'm supposed [TS]

  to do on behalf of this program is evict [TS]

  something from memory put that on disk [TS]

  and then put his thing in that spot and [TS]

  when it goes to evict something from [TS]

  memory and put it on disk the disk is [TS]

  full and that's why bad things start [TS]

  happening classic Mac OS way back and [TS]

  they did not have virtual memory which [TS]

  is different than swap but it certainly [TS]

  didn't have swap space so it could [TS]

  happily hum along with its you know 512 [TS]

  K of RAM or whatever and if you hit the [TS]

  512 K Ram limit ah that was the end of [TS]

  that it wouldn't swap stuff out to disk [TS]

  so if your disk was full no big deal [TS]

  it's the only time it has to write [TS]

  something on to disk is if you ask it to [TS]

  not in your normal operation it didn't [TS]

  write stuff and UNIX of course has log [TS]

  files that are constantly being appended [TS]

  to and other things that are going on in [TS]

  the background slowly filling in your [TS]

  disk that [TS]

  know about and if your disk is full all [TS]

  those things freak out and say you know [TS]

  we've got a problem here they would [TS]

  probably they would probably try to log [TS]

  the fact that they got a no space left [TS]

  on device message and that would cause [TS]

  more Disgaea which would also fail and [TS]

  it's just a vaccine so there's an [TS]

  exception to this in the modern error [TS]

  which is iOS which some people also [TS]

  asked about iOS if you run out of disk [TS]

  space that's a dead end still if you run [TS]

  out of RAM on iOS it's also a didn't [TS]

  because iOS though it implements virtual [TS]

  memory virtual memory is the mechanism [TS]

  by which the programs think they have [TS]

  much more memory and they do then they [TS]

  really do and they think they all have [TS]

  their own memory space to themselves [TS]

  it has virtual memory but it does not [TS]

  use swap so when you fill up all the RAM [TS]

  on iOS it doesn't say oh let me put that [TS]

  on quote-unquote disk for you it's disk [TS]

  is obviously flash what it does is kills [TS]

  your program in fact to kill your [TS]

  program long before that iOS is a very [TS]

  hostile environment for programs and the [TS]

  the application environment for running [TS]

  GUI applications in iOS will send your [TS]

  application a low memory warning if it [TS]

  starts using too much RAM and if the [TS]

  program does not respond by freeing [TS]

  memory eventually the operating system [TS]

  will just kill that program say look I [TS]

  sent you a message I told you you're [TS]

  getting close to the memory limit I'm [TS]

  gonna have to kill you because if you [TS]

  ever hit the memory limit on iOS that's [TS]

  it it's not you know I need more space [TS]

  well RAM is full and there's no more RAM [TS]

  we can't make new Ram out of thin air [TS]

  and we're not going to swap the flash [TS]

  now one of the reasons that iOS doesn't [TS]

  swap to flash is because even though [TS]

  flash is faster than disk it's still way [TS]

  way WAY slower than Ram so that's one of [TS]

  the reasons that iOS has always been so [TS]

  responsive because to get that kind of [TS]

  responsiveness that's how it's running a [TS]

  variant of Mac OS 10 we're doing virtual [TS]

  memory but we cannot afford to like oh [TS]

  when I swipe something on the screen oh [TS]

  hang on a second I gotta pull a bunch of [TS]

  that stuff that you thought was in [TS]

  memory actually it's on my flash disk [TS]

  and I got to pull it off the disk and [TS]

  it's it can take a thousand times longer [TS]

  than you thought it would and then you [TS]

  get the stuttery animation or anything [TS]

  maybe someday that will change but for [TS]

  now as far as I'm aware maybe someone [TS]

  can correct me [TS]

  Iowa still does not use swap space it [TS]

  puts everything in RAM so if you're [TS]

  going to unify disk [TS]

  and ram into one type of thing [TS]

  everything would have to become a dead [TS]

  end [TS]

  because you have no other place to put [TS]

  stuff right like when you run out of [TS]

  whatever you're going to call this thing [TS]

  when you run out of memory that means [TS]

  that your caches are full you know they [TS]

  know you can put in a register can't put [TS]

  it in a cache you can't put it in RAM [TS]

  and uh your disk is also full and I [TS]

  think that would be a bad place to be [TS]

  because you could imagine getting [TS]

  machine into a deadlock situation [TS]

  hopefully not hopefully in a consistent [TS]

  state of some kind but where do you go [TS]

  from there you don't have an out you [TS]

  don't have a place to put stuff maybe [TS]

  we're just carve off it you would not do [TS]

  the same thing where it would say if you [TS]

  get close to the limit of my giant pool [TS]

  of memory I'm going to warn you and then [TS]

  I'm going to kill stuff off because we [TS]

  cannot hit that limit sort of like you [TS]

  know it does with disk space today where [TS]

  you just can't get close to that limit [TS]

  on disk pacer will start yelling about [TS]

  you but Ram the operating system will [TS]

  let you continue to use it and use it [TS]

  when you have virtual memory with swap [TS]

  until your computer becomes so slow that [TS]

  you're discouraged from using it any [TS]

  longer and you walk away and wait for [TS]

  everything to swap in and out so I think [TS]

  this is an interesting topic I don't [TS]

  know what will happen in this area I [TS]

  don't think any time in my lifetime [TS]

  we're going to see this grand unified a [TS]

  single pool of memory thing but if it is [TS]

  going to happen I think it will happen [TS]

  somewhere in the mobile space where the [TS]

  sizes are smaller and the entrenched [TS]

  system of dividing RAM and disk are not [TS]

  as prevalent as they are on the PC [TS]

  that's the end of my fake follow up [TS]

  follow up I was what's fake about that I [TS]

  thought it was very real well it was you [TS]

  know it's talking about SSD is not [TS]

  following up something we said last time [TS]

  but it's a new top oh I see yeah so your [TS]

  your confidence level of SSD over the [TS]

  traditional Drive as far as a strategy [TS]

  for Apple what are your thoughts on that [TS]

  long-term strategy Apple's hardware [TS]

  everything SSD yeah has to be mean it's [TS]

  going to have [TS]

  it's just a question of how long it [TS]

  takes with Apple sells consumer hardware [TS]

  so they don't have to deal with like [TS]

  well I need to store petabytes of data [TS]

  like they're not selling servers for the [TS]

  data center and SSDs are way more [TS]

  expensive than spinning disks we're just [TS]

  kind of that inflection point where they [TS]

  can say like when I got my MacBook Air [TS]

  like the biggest s is the you get was [TS]

  256 and that's what I got and that was [TS]

  still bigger than 160 gig spinning disk [TS]

  that was in my old MacBook Pro so it was [TS]

  still a step up but you can get like [TS]

  terabyte now I think terabyte laptop [TS]

  drives are available it's four times [TS]

  smaller than than the the biggest lap [TS]

  spinning laptop drive you can get and [TS]

  way way way more expensive so they're [TS]

  just on the borderline of like you can [TS]

  make people choke down a much higher [TS]

  price for less space just because the [TS]

  performance is so much better but they [TS]

  haven't quite crawled up to like can I [TS]

  convince someone to buy a Mac Pro with [TS]

  just SSDs in it someone who wants a Mac [TS]

  Pro probably wants a lot more internal [TS]

  storage space than 256 gigs and those [TS]

  drives are so expensive if you buy for [TS]

  them now the entire cost of your [TS]

  computer is basically just an SSD so the [TS]

  price keeps going down the storage space [TS]

  keeps going up eventually there'll be a [TS]

  point where spinning disks are just seen [TS]

  as archaic and everyone has to have SSDs [TS]

  and only the people who really need [TS]

  gigantic amount of space need spinning [TS]

  disks and eventually those will probably [TS]

  go away - so that's inevitable you just [TS]

  get used to the fact that as as these [TS]

  are coming how long before every single [TS]

  computer Apple makes has SSD by default [TS]

  the kinds of if they discontinue the Mac [TS]

  Pros thoughts on discontinuing the Mac [TS]

  Pro don't do it that's my thought please [TS]

  as you do it you'd like you like the Mac [TS]

  Pro and you're you're a big-time Mac Pro [TS]

  user yeah you don't want them to get rid [TS]

  of it I was I saw an article from I [TS]

  don't know if you've spoken with him but [TS]

  he's a friend of mine he used to do a [TS]

  show here as well James Duncan Davidson [TS]

  he's a photographer and right now or [TS]

  very recently he was out shooting Ted [TS]

  and he was saying that every year that [TS]

  he shot Ted which is a number of years [TS]

  these guys they spare no expense when it [TS]

  comes to having great [TS]

  hardware for the people who are doing [TS]

  the audio video editing and then the [TS]

  photography editing and things like that [TS]

  and they set up a whole lab I'll try and [TS]

  put I'll try and put one of these [TS]

  pictures that he's taken of the lab into [TS]

  the show notes and by the way thanks to [TS]

  help spot calm for the show notes and [TS]

  this year instead of having two dozen [TS]

  Mac Pros they now have two dozen IMAX [TS]

  with Thunderbolt drives connected to [TS]

  them raid Thunderbolt drives connected [TS]

  to them and they have officially moved [TS]

  away completely from Mac Pros and now [TS]

  they just have IMAX and it makes you [TS]

  think that if if a company like tad if a [TS]

  group like Ted is as as serious and [TS]

  dedicated as they are to doing real and [TS]

  if you've ever watched the Ted videos [TS]

  you know these are these are phenomenal [TS]

  videos they have the best audio-video [TS]

  photographer people in the business [TS]

  making them and editing them and turning [TS]

  them around in almost real-time it's [TS]

  it's crazy how fast they put these [TS]

  things out there they're content with [TS]

  IMAX why why would Apple keep making the [TS]

  Mac Pro who is that for isn't the [TS]

  Thunderbolt the obvious indication that [TS]

  they're serious about moving away from [TS]

  the Mac Pro well that particular [TS]

  application I think doesn't need the [TS]

  massive amounts of storage because kind [TS]

  of a way station for a workflow that [TS]

  takes assets from the day and grinds [TS]

  them up and spits out some new assets [TS]

  where you need the big spinning disks [TS]

  that can store so much more than SSDs [TS]

  for so much less money is where does the [TS]

  company that is responsible for [TS]

  recording all of Ted and producing all [TS]

  the content where does it distorts [TS]

  information where does it start all the [TS]

  raw videos all the raw images all the [TS]

  processed files everything forever [TS]

  archived maybe puts them all on tape and [TS]

  that's the answer but I think companies [TS]

  that have large storage requirements [TS]

  like people doing special effects for [TS]

  movies or something like that they'll [TS]

  use SSDs for the parts where speed [TS]

  matters but at some point that like they [TS]

  don't want to ever throw anything out [TS]

  you want to have all your assets and all [TS]

  everything redundantly stored in a [TS]

  million place and that takes up a [TS]

  tremendous and [TS]

  room even just for one movie let alone [TS]

  movie after movie like how many assets [TS]

  do you think Pixar has stored [TS]

  permanently yeah it accessible not on [TS]

  tape or some robot has to go pull a tape [TS]

  out and shove it in but accessible in [TS]

  semi real-time because you might want to [TS]

  use that asset for a new production you [TS]

  need some list to put all that stuff in [TS]

  SSDs just cost too much money to to do [TS]

  that now spinning disks are so much more [TS]

  economical especially since you have to [TS]

  have 20 of them because you have to have [TS]

  everything redundantly stored and you [TS]

  know I there is still a big market for [TS]

  it for spinning discs now is that is [TS]

  Apple does Apple have any part of that [TS]

  market maybe they're going to say look [TS]

  if you want to do that you're gonna have [TS]

  to buy disk storage from EMC or [TS]

  something just like we at Apple do for [TS]

  our data centers because we're not [TS]

  interested in giving you a machine that [TS]

  can store all that stuff or as you said [TS]

  our machines can't store it buy an iMac [TS]

  and connect one of these big raid things [TS]

  that we also don't make to a Thunderbolt [TS]

  cable you have really fast access to to [TS]

  lots of storage the Mac Pro as a product [TS]

  exists for reasons other than just [TS]

  internal drives though it's got the [TS]

  slots where you can put cards that you [TS]

  might need cards that don't fit inside [TS]

  the iMac because there's no place for [TS]

  cards at all and yet maybe you can use a [TS]

  Thunderbolt external chassis but the [TS]

  number of PCI Express Lanes over a [TS]

  thunderbolt wire is not that that bus is [TS]

  not as wide as an internal slot so there [TS]

  are many reasons why the Mac Pro still [TS]

  needs to exist the fact that they [TS]

  switched from Mac Pro Stimac for that [TS]

  Ted thing they probably could have [TS]

  gotten away with imax even sooner [TS]

  because it's just kind of a way station [TS]

  and not something that needs to have [TS]

  lots of storage and those big you know [TS]

  big special-purpose cards and yeah and [TS]

  also the other thing to keep in mind is [TS]

  that Ted probably has a lot of money you [TS]

  would think so I put I I did find the [TS]

  article and the pictures that go along [TS]

  with it I put it into the show notes and [TS]

  he says for Ted 2012 there's a pretty do [TS]

  you say SATA or sad at these days I say [TS]

  both okay for Ted 2012 there's a pretty [TS]

  big change in the media room instead of [TS]

  dozens of Mac Pros and piles of SATA [TS]

  drives like there have been in the last [TS]

  eight Ted events I've been part of the [TS]

  room is full of IMAX and Thunderbolt [TS]

  drives lots of Thunderbolt drives and he [TS]

  has a photograph here he says there are [TS]

  12 12 12 terabyte Pegasus are six arrays [TS]

  that's a hundred and forty four [TS]

  terabytes total [TS]

  he says it which is about a tenth of a [TS]

  petabyte of usual usable space for good [TS]

  measure there are a few more pegasus [TS]

  arrays including one that's dedicated to [TS]

  photography and he goes on to detail [TS]

  this and he says this is the biggest [TS]

  appointment of thunderbolt based storage [TS]

  that he's seen yeah I think Thunderbolt [TS]

  has made it feasible to have external [TS]

  storage that's really fast without [TS]

  having some weird interface card like [TS]

  fiber channel or something because [TS]

  internal disk was the way the way how [TS]

  you get the speed right because you know [TS]

  it's fast as possible as you can get [TS]

  it's connected right to the thing [TS]

  there's no long wire there's no weird [TS]

  interface it's you know sad is pretty [TS]

  fast and they would keep cranking up the [TS]

  speed and then once you went out over [TS]

  firewire like firewire 400 in particular [TS]

  was not enough to handle the output of a [TS]

  arrayed with many spindles or any large [TS]

  array like that firewire 800 was an [TS]

  improvement but still I think as the the [TS]

  Thunderbolt devices they're available [TS]

  today show they'll take like a four disc [TS]

  firewire 800 Drive and then I'll take a [TS]

  four disc Thunderbolt array maybe it's [TS]

  the same exact four mechanisms in each [TS]

  set light but that by eight mechanisms [TS]

  but form one form the other and the [TS]

  Thunderbolt ones just transit because [TS]

  Thunderbolt is like an external [TS]

  extension of an internal bus in this [TS]

  case it's the PCI Express bus so that [TS]

  that probably makes it feasible for them [TS]

  to have storage that's fast enough to do [TS]

  all this HD video streaming or whatever [TS]

  they're doing whilst just hooking it up [TS]

  to an iMac with that little Thunderbolt [TS]

  cable but the Mac pros are still an open [TS]

  question I think let's do our second [TS]

  sponsor its tiny letter comm they're [TS]

  just they're putting the finishing [TS]

  touches on a brand-new version and this [TS]

  is very exciting they have been working [TS]

  hard on this I talked to them about it [TS]

  and a tiny letter in general the best [TS]

  way for me to describe it to you is it [TS]

  is the most simple straightforward [TS]

  newsletter tool that has ever existed in [TS]

  the history of the earth it's elegant [TS]

  it's simple its straightforward it gets [TS]

  rid of all of the extra features and [TS]

  cruft and user interface layers and [TS]

  throws all that out the window now some [TS]

  people might want that go to MailChimp [TS]

  com if you want that you don't want that [TS]

  tiny letter super simple elegant [TS]

  straightforward tiny letter com [TS]

  they have a sneak peak video that they [TS]

  just put up to give you an idea of what [TS]

  it's going to be like within the next [TS]

  few days when this thing comes out so go [TS]

  check it out thanks very much two tiny [TS]

  letter calm for making the show possible [TS]

  I do actually have some more files of [TS]

  some stuff lurking at the bottom here [TS]

  we're going to talk about BOS no okay [TS]

  would be good ID now you give me more [TS]

  stuff I tried the Kappas to shows I had [TS]

  to assume well I actually I wanted to [TS]

  talk more about ZFS because all the [TS]

  stuff we talked about was EFS was [TS]

  focused on the particular failings of [TS]

  HFS+ and how they're addressed by ZFS [TS]

  with data integrity and all that but [TS]

  there are so many more things as e of us [TS]

  that I realize they didn't even get to [TS]

  that are interesting and other reasons [TS]

  why you might be interest why you might [TS]

  want this process like things that H of [TS]

  us aren't even a glimmer in HFS+ is I [TS]

  but that our most modern bus system cell [TS]

  and I imagine all sorry F SS but we [TS]

  didn't quite get to it okay so the first [TS]

  one which I think I mentioned this is [TS]

  these are advantages of ZFS or neat [TS]

  features of ZFS S and there a lot of [TS]

  them are not unique to ZFS but I'm using [TS]

  it as the poster boy for a modern ish [TS]

  file system first one is constant time [TS]

  snapshots and clones that is a mouthful [TS]

  there so I'll explain the pros also in [TS]

  the middle part snapshots so a snapshot [TS]

  is like the word says kind of at [TS]

  capturing the state of a disk at a [TS]

  moment in time and obviously we talked [TS]

  about this before it's a consistent [TS]

  state so the file system might be [TS]

  applications might be in the middle of [TS]

  doing stuff might be in the middle of [TS]

  writing files out or deleting stuff or [TS]

  whatever but you're going to take a [TS]

  snapshot and say look at that [TS]

  application might be in the null writing [TS]

  that file and it might have written half [TS]

  the file but as far as the file system [TS]

  concerned at this moment in time it [TS]

  knows where everything is it knows [TS]

  exactly the size of everything [TS]

  everything is all the bookkeeping agrees [TS]

  with each other that half written file [TS]

  it knows that that file is is one [TS]

  megabyte ah maybe the file will [TS]

  eventually be two megabytes when the [TS]

  applications done but right now one [TS]

  megabyte is on disk the metadata says [TS]

  one megabyte it is a moment in time [TS]

  snapshot ah [TS]

  and what ZFS can do is freeze that and [TS]

  say okay now at any [TS]

  in the future if you ever want to see [TS]

  what this file system was like at this [TS]

  particular moment in time it's available [TS]

  to you you can look at what it looked [TS]

  like at this moment in time the constant [TS]

  time part means that a time required to [TS]

  take a snapshot doesn't scale with the [TS]

  size of the disk so it's not like it [TS]

  makes a big giant copy like you so you [TS]

  have a terabyte drive when you say ok [TS]

  take a snapshot doesn't say ok I'm going [TS]

  to take the entire terabyte you know say [TS]

  this 500 you know gigabytes on there I [TS]

  take this entire 500 gigabytes and copy [TS]

  it someplace else it doesn't do that [TS]

  because of the way the file system is [TS]

  structured it can just make a note that [TS]

  this particular moment in time is [TS]

  important and it should be preserved [TS]

  remember that ZFS like many other file [TS]

  systems avoids overriding data in place [TS]

  it writes all new changes to a new [TS]

  location when it does that the old [TS]

  location eventually falls out of use [TS]

  when no more processes have that file [TS]

  open when nobody else is using it said [TS]

  that was the old version of this file [TS]

  the new one is way over there so now [TS]

  we'll just say ok this the space [TS]

  previously occupied by the old file is [TS]

  free to be thrown back into the pool all [TS]

  a snapshot does is tell the filesystem [TS]

  hey all of these things that make up [TS]

  this moment in time of the filesystem [TS]

  don't throw them out even when [TS]

  everybody's done with them keep them [TS]

  around so the literature on ZFS will say [TS]

  that this is instantaneous instantaneous [TS]

  snapshot Gration it's not instantaneous [TS]

  but the point is that it's constant time [TS]

  that it doesn't take twice as long to [TS]

  take a snapshot of a one terabyte discus [TS]

  as that is a 500 gig disk because the [TS]

  operation is basically a simple [TS]

  bookkeeping operation of saying all the [TS]

  stuff that you need with the snapshot is [TS]

  already on disk all I'm telling you is [TS]

  don't throw that stuff out later so it's [TS]

  very quick to note that like this moment [TS]

  in time it's a snapshot and you just [TS]

  store a little bit of metadata about [TS]

  here's a snapshot and here's you know [TS]

  that here's where it is in the time [TS]

  sequence of events and don't delete that [TS]

  stuff because it's preserving as the [TS]

  snapshot so it's really just a promise [TS]

  in the future to not ditch those old [TS]

  blocks that have new versions written to [TS]

  them and that's something you can't even [TS]

  imagine doing hfs+ because in nature I [TS]

  suppose everything is written in place [TS]

  and if you wanted to know what your disk [TS]

  looks like in a moment in time there's [TS]

  nothing available to you to do that [TS]

  the only thing you could possibly do [TS]

  is stopped all processes but you can't [TS]

  really do butts is always something [TS]

  vlogging in the background and UNIX or [TS]

  whatever but try to freeze it you know [TS]

  quit every application kill all [TS]

  processes is that the possible minimal [TS]

  maybe rebooting single user mode or [TS]

  something and then make an exact clone [TS]

  of this disk to a second location either [TS]

  a second disk or a second partition as [TS]

  you can imagine making an exact clone of [TS]

  a disk takes a long time and if it takes [TS]

  twice as long to make clone of a [TS]

  terabyte disk as a 500 gig disk and that [TS]

  amount of time is like measured in hours [TS]

  probably this is versus ZFS where you [TS]

  can just snap your fingers and set up [TS]

  make a snapshot of that so I can get [TS]

  back to it anytime now clones are [TS]

  writable snapshots clones let you say [TS]

  okay take a snapshot of it but actually [TS]

  let people make future changes to that [TS]

  snapshot so if you took a snapshot if [TS]

  you took if you made a clone right now [TS]

  what you essentially have our two file [TS]

  systems that look exactly identical and [TS]

  you can go into one of them and start [TS]

  messing with stuff of course the other [TS]

  one you know your main one also [TS]

  continues to go along and they start to [TS]

  diverge from each other this is all [TS]

  efficient because they share all the [TS]

  blocks in this because they have on [TS]

  count in common likes when you're making [TS]

  a snapshot all the disks are in common [TS]

  and then as time goes on something [TS]

  changes and what it does is okay well [TS]

  you know the same thing that always does [TS]

  someone wants to make a change this [TS]

  right the new version of it over there [TS]

  and to leave the old version over there [TS]

  because that's part of a snapshot so the [TS]

  the two resulting logical file system [TS]

  slowly diverged as as that more blocks [TS]

  changed as they differ snapshots don't [TS]

  let you modify that moment in time image [TS]

  of the thing but clones say just make it [TS]

  you know make a note here that this was [TS]

  a moment in time and now future changes [TS]

  to this clone should be separate from [TS]

  the future changes to the operating [TS]

  system that it was clone as if until the [TS]

  operation to the file system that it was [TS]

  cloned from these are two very powerful [TS]

  and very interesting technologies that [TS]

  I'm sure people who have never heard of [TS]

  them can think of many possible uses for [TS]

  them already but the fact that they are [TS]

  not giant copy operations and don't [TS]

  scale with the size of the file system [TS]

  is key the number of snapshots you can [TS]

  have on ZFS is supposedly unlimited but [TS]

  as you can imagine if you keeps taking [TS]

  snapshots what you're saying is the [TS]

  blocks used [TS]

  by the data currently on the disk can [TS]

  never be free eventually you'll fill [TS]

  your disk because you're you're telling [TS]

  it you can't delete this old stuff even [TS]

  if like so you take a snapshot then you [TS]

  delete a 1 gigabyte file off your disk [TS]

  well it can't actually give you one 2 [TS]

  gigabyte of free space left because that [TS]

  gigabyte file has still exist in the [TS]

  snapshot so that will disappear from [TS]

  your current disk but if you were to [TS]

  look back in the snapshot it would still [TS]

  be there so eventually you will fill [TS]

  your entire disk by taking snapshots you [TS]

  can course [TS]

  can course [TS]

  trim old snapshots off the end and say [TS]

  okay this snapshot I'm done with it you [TS]

  can free up all the blocks there or [TS]

  whatever but the whole point of all this [TS]

  is is that it happens very very quickly [TS]

  basically instantaneously which is a far [TS]

  cry from how long it takes to do a [TS]

  super-duper backup or a Time Machine [TS]

  backup or anything like that the second [TS]

  related feature is sending and receiving [TS]

  block level deltas between snapshots [TS]

  this is where you can say okay I've got [TS]

  a snapshot over here and I've got a [TS]

  snapshot over there what is it is [TS]

  different between those two things find [TS]

  not the files that are different but [TS]

  find the individual blocks on disk that [TS]

  are different between snapshot a and [TS]

  snapshot B and send them from snapshot a [TS]

  snapshot B to make B match up with a now [TS]

  this is should bring to mind a backup [TS]

  type of strategy incremental backup [TS]

  strategy and it's a big contrast with [TS]

  the way time machine works this is a [TS]

  review of some of the stuff they wrote [TS]

  about in the leopard review and time [TS]

  machine was introduced the way time [TS]

  machine works is it also has to send the [TS]

  differences between the last time you [TS]

  backed up and the current state to your [TS]

  time machine drive so if you run time [TS]

  between the first time it copies [TS]

  everything to a second Drive right the [TS]

  second time you run it you don't want it [TS]

  to copy everything again you just wanted [TS]

  to say what has changed since the last [TS]

  time you did a Time Machine backup and [TS]

  then write those changes to the Time [TS]

  Machine backup so the first backup takes [TS]

  like hours and the second backup takes [TS]

  maybe five minutes ah the tricky bit is [TS]

  how do you tell what has changed since [TS]

  the last time you made a backup in the [TS]

  in Mac OS 10 and time machine what Mac [TS]

  OS 10 does is anytime anything makes a [TS]

  change to your disk it writes a little [TS]

  log of it there's a mechanism called FS [TS]

  events that maintains this highly [TS]

  compressed log that tries to coalesce [TS]

  multiple updates and all sorts of other [TS]

  things to try to make it so it's not [TS]

  filling your disk but right now as [TS]

  you're if you're sitting here on your [TS]

  computer or your Mac is doing stuff at [TS]

  home anytime something happens to the [TS]

  file system the operating system is [TS]

  writing to a little log file of what [TS]

  happened to the file system at a file [TS]

  granularity so it's going to say this [TS]

  file change [TS]

  file was deleted this file was created [TS]

  when Time Machine runs it reads it keeps [TS]

  track of where it wasn't that log last [TS]

  time so it says I did a time machine [TS]

  back up and I did all the updates up to [TS]

  file system event number 1 2 3 so I'm [TS]

  going to start from file since event [TS]

  number 1 2 4 and it says this file was [TS]

  deleted ok so I know that's the change [TS]

  you have to make in the back and it said [TS]

  this file was modified ok so I've got to [TS]

  grab the copy that file put it over [TS]

  there and says this file was deleted so [TS]

  you know it can go back go through the [TS]

  file system event log and find the [TS]

  things that have changed and make the [TS]

  new backup with them when it makes the [TS]

  backup it doesn't make it does use as [TS]

  hard links on the target destination to [TS]

  say all right so everything is exactly [TS]

  the same as the last backup except this [TS]

  one document change so it makes hard [TS]

  links to all of the rest of the disk [TS]

  except for the directory that has that [TS]

  one changed file in it and then it makes [TS]

  hard links to all the files in that [TS]

  directory except for the one file to [TS]

  change and then finally it copies the [TS]

  one file to changed over to the [TS]

  time-machine disk so it doesn't have to [TS]

  have to complete copies every date it's [TS]

  got one copy of all your data plus two [TS]

  copies of that one file that changed you [TS]

  can see this is kind of a very different [TS]

  way to do what copy and write does it a [TS]

  block level in something like ZFS the [TS]

  disadvantage is that many people use [TS]

  time as you know if you have a big file [TS]

  like a 2 gigabyte mail database and you [TS]

  get a single new piece of email and it [TS]

  modifies your 2 gigabyte mail database [TS]

  file the next time time machine runs is [TS]

  going to see a file system event that [TS]

  says aw since the last time I ran the [TS]

  mail database has changed and then it's [TS]

  going to copy the entire 2 gigabyte mail [TS]

  database all 2 gigabytes of it over to [TS]

  the time machine disk because that's the [TS]

  only thing it can do it's got the old [TS]

  version of the 2 gigabyte mail database [TS]

  on the time machine disk and now there's [TS]

  a new version it's got to copy the [TS]

  entire version over there if you have [TS]

  any large files that change frequently [TS]

  your time machine backups are going to [TS]

  be humongous you're going to be doing [TS]

  gigs and gigs of i/o when may be only a [TS]

  few kilobytes changed because it works [TS]

  at the file level the FS immense [TS]

  mechanism works at the file level and [TS]

  time machine works at the file level [TS]

  this is why mail programs that used to [TS]

  have 2k by database [TS]

  have since been changed to use thousands [TS]

  and thousands of little tiny files [TS]

  because when a little tiny file changes [TS]

  just a little file has to be copied not [TS]

  the entire file but there are still [TS]

  large files that are modified fairly [TS]

  frequently and just so wasteful to have [TS]

  to you know I add I add two three words [TS]

  to the end of this big Word document so [TS]

  I got to copy the two megabyte word [TS]

  document again if that happens over and [TS]

  over again you're filling your backup [TS]

  disk way fat way faster than you think [TS]

  you should uh and you're doing a lot [TS]

  more i/o you know you're copying tons [TS]

  and tons of stuff uh you know from one [TS]

  place to the other so everything takes [TS]

  longer now this this mechanism of [TS]

  figuring out what has changed through [TS]

  the file system the vent thing this was [TS]

  introduced uh I believe was introduced [TS]

  when spotlight was introduced because [TS]

  spotlight uses the same thing spotlight [TS]

  is the thing that indexes all the files [TS]

  on your just so you can search them and [TS]

  doesn't just index the file names it [TS]

  also indexes the file contents so [TS]

  anytime something happens on your disk [TS]

  in a file is written the spotlight [TS]

  daemon else on the spotlight demo one of [TS]

  the daemons related to spotlight is [TS]

  listening in on that file system event [TS]

  stream listening for every single thing [TS]

  that happens on your disk and when [TS]

  something happens it goes AHA a file has [TS]

  been modified or created or deleted I [TS]

  have to do something so say a file was [TS]

  created says up new file is created and [TS]

  it's notified in real time when that [TS]

  happens and it says well I gotta go [TS]

  index that file so it's six it's little [TS]

  index around that file and the index R [TS]

  reads some portion of the file and [TS]

  updates his index of stuff and says okay [TS]

  I read this file and I index and it's in [TS]

  there same thing when a file is deleted [TS]

  so this file was deleted I got to remove [TS]

  that file from my index so when I search [TS]

  for it later they don't find it because [TS]

  it's gone when this mechanism is reduced [TS]

  this firehose of file system events [TS]

  real-time streams of what's happening in [TS]

  the file system was only available to [TS]

  selected apple processes so the [TS]

  spotlight could do it and a few other [TS]

  things we're listening on the the [TS]

  firehose of file system events and the [TS]

  reason it was limited to only a few [TS]

  listeners was because this is all [TS]

  happening in the kernel of the operating [TS]

  system it's the only way to catch all I [TS]

  owe that happens to a disk is to talk [TS]

  into the kernel the file systems in that [TS]

  magazine was hooking to the crown so [TS]

  anytime any i/o happened it could go [TS]

  through this list of people who are [TS]

  interested hey is anybody listening for [TS]

  files as events I got tell them this [TS]

  happen Hey [TS]

  this file was created oh hey this file [TS]

  was deleted hey this file was modified [TS]

  things that happen inside the kernel [TS]

  first of all things that happen inside [TS]

  the kernel tend not to maybe able to be [TS]

  swapped out to disk for what I hope are [TS]

  obvious reasons because the kernels [TS]

  where you implement the swap stuff so if [TS]

  for example the code that swaps virtual [TS]

  memory to disk is swapped to disk and [TS]

  you need it to get the stuff off disk [TS]

  you're in a bad situation [TS]

  so most kernel memories wire down wire [TS]

  down means that it can't be swapped disk [TS]

  and the kernel has buffers for storing [TS]

  like messages that it's going to send [TS]

  and most of those buffers are fixed in [TS]

  size and small because that you know [TS]

  growing memory and in the kernel is [TS]

  another thing that tends to be frowned [TS]

  upon you want a bunch of small fixed [TS]

  size buffers so when you're sending [TS]

  messages to these things that are [TS]

  listening for filesystem events you've [TS]

  got a window of time when things have to [TS]

  happen [TS]

  the kernels got to put the notification [TS]

  in in some buffer queue for say okay by [TS]

  the way spotlight thing you want to know [TS]

  when something happens something [TS]

  happened well say tons of things are [TS]

  happening really really fast like 100 [TS]

  files are created as fast as you [TS]

  possibly can the thing listening for [TS]

  filesystem events has a responsibility [TS]

  to consume those events to pull them off [TS]

  the queue and do something with them at [TS]

  a speed that's close to the speed that [TS]

  they're being filled in otherwise the [TS]

  buffer is going to get full and what [TS]

  happens when the buffer is full like oh [TS]

  it's it's like the Lucy with the [TS]

  assembly line chocolates thing right [TS]

  what episode of ice from I Love Lucy I [TS]

  don't know there's the one where she [TS]

  works in a chocolate factory in the [TS]

  chocolates are coming out faster than [TS]

  she can box him up yeah she was also [TS]

  funny she's she's the consumer in this [TS]

  she's supposed to be putting them in [TS]

  boxes and putting away but it's the [TS]

  producer at the chocolates cannot coming [TS]

  down the assembly line too fast you know [TS]

  what happens is the chocolates get [TS]

  dropped on the floor so that can happen [TS]

  with file system event consumers if the [TS]

  consumers are not pulling the events off [TS]

  the queue as fast as they can are [TS]

  they're coming in they will end up [TS]

  missing events and the whole point of [TS]

  this mechanism is not to miss any I have [TS]

  to know everything that happened so [TS]

  imagine for for the thing that writes [TS]

  the log of things that happen that's the [TS]

  whole point is it has to make a note of [TS]

  everything that happened and it really [TS]

  can't [TS]

  an event because then say files have [TS]

  been created really really fast and [TS]

  things like I'm alright I'm keeping up [TS]

  on pulling these events off the queue [TS]

  I'm keeping track of stuff I'm it would [TS]

  consolidate events to say like this if [TS]

  this file was changed 17 times in [TS]

  sequence the event log just needs to [TS]

  know the file was changed doesn't need [TS]

  to know how many times it was changed so [TS]

  we consolidate all those 17 changes into [TS]

  a single change event but eventually [TS]

  it's got to write out to its little [TS]

  compressed log file and disk this file [TS]

  change and it's got to do that fast [TS]

  enough that it doesn't get overwhelmed [TS]

  by the events that are coming in because [TS]

  if the kernel can't put another event on [TS]

  the queue for this listener [TS]

  it's going to send out a message and say [TS]

  look I had another event for you but [TS]

  there's no place to put it your queue is [TS]

  full I can't put it in your queue if I [TS]

  did that have to delete one of the [TS]

  things that's there so I'm just going to [TS]

  send you a message it says dude you [TS]

  didn't keep up you missed an event sorry [TS]

  about that [TS]

  if that happens the next time Time [TS]

  Machine runs it may not know about some [TS]

  file has been modified and then when you [TS]

  disk goes bad and you restore from your [TS]

  time machine back up you're like wait [TS]

  where's that file and if it's not there [TS]

  that's bad so there are certain [TS]

  constraints especially we're backup [TS]

  software but even for something like [TS]

  spotlight where like say you make a new [TS]

  file and something couldn't keep up and [TS]

  you go searching for it and you can't [TS]

  find it it's never going to get into the [TS]

  index because nobody knows that you know [TS]

  all you know all I can do is send to the [TS]

  the process you missed an event what is [TS]

  the recourse that time machine and [TS]

  spotlight have if they miss an event [TS]

  they the alternative strategy is really [TS]

  really bad because like our so I missed [TS]

  an event something happened and I don't [TS]

  know about the kernel is never going to [TS]

  tell me again because it's long since [TS]

  forgotten about that and moved on to [TS]

  other things and I don't know what [TS]

  happened maybe a really important file [TS]

  was modified or deleted or created or [TS]

  something and if I can't add to remove [TS]

  it to my index and I can't put it in my [TS]

  logs at a time machine knows to copy it [TS]

  to another disk we've got a bad [TS]

  situation here so what most of those [TS]

  programs do in response to you missed an [TS]

  event is to say oh I guess I got to scan [TS]

  the whole disk again from top to bottom [TS]

  maybe you've seen this if you do [TS]

  something that angers either Time [TS]

  Machine your spotlight where you log in [TS]

  and spotlight says it's indexing your [TS]

  disk again it's got to read every single [TS]

  file in every single directory starting [TS]

  from the beginning right now so it's [TS]

  gotta say forget it I'm totally screwed [TS]

  up I can do everything over again [TS]

  re-indexing with spotlight a one [TS]

  terabyte disk gets filled or [TS]

  are doing a brand-new time machine back [TS]

  up from the beginning and figuring out [TS]

  on a file-by-file basis is this file [TS]

  change no is this file change now it [TS]

  takes forever and grinds your disk to [TS]

  death [TS]

  this is all very very bad so that's why [TS]

  in when this mechanism introduced the [TS]

  public interface for applications was [TS]

  are you can't drink from the fire hose [TS]

  because you can't be held responsible [TS]

  for being able to keep up with the [TS]

  stream of stuff and furthermore if we [TS]

  have too many listeners listening for [TS]

  every single filesystem event that's [TS]

  happening we're going to have to slow [TS]

  down and we might not have enough kernel [TS]

  buffers for them and it's just you know [TS]

  this is not for you don't touch us but [TS]

  we'll have a public interface that will [TS]

  give you a much lower granularity of [TS]

  updates instead of telling you any every [TS]

  time something changes all we're going [TS]

  to tell you is something changed in this [TS]

  directory and we're not going to tell [TS]

  you what it was and those events can [TS]

  come kind of in a leisurely manner so if [TS]

  there's some huge amount of filesystem [TS]

  activity going on your application using [TS]

  this API is only going to find out about [TS]

  it perhaps after a whole bunch of stuff [TS]

  has changed and it's just and all you're [TS]

  gonna get is something changed in this [TS]

  directory and what happened I don't know [TS]

  that's not up to us so that is a much [TS]

  lower granularity of event then this [TS]

  file was deleted this file was created [TS]

  this file was modified this file was [TS]

  deleted you know that type of thing and [TS]

  that was the public interface to file [TS]

  system events now and time machine can [TS]

  consume that as well because but what [TS]

  that means is that what you get an event [TS]

  that says something changed in this [TS]

  directory you have to have a way to [TS]

  figure out all right well what changed [TS]

  one of the ways you could figure it out [TS]

  for example if your time machine is they [TS]

  could say something change in this [TS]

  directory all right well I've got a [TS]

  backup copy of that directory why don't [TS]

  I read that the entire content of that [TS]

  directory and compare it to the current [TS]

  contents and find the differences this [TS]

  is something similar to what and user [TS]

  programs do when they get a notification [TS]

  it's up to the program to decide like if [TS]

  you're interested in changes in this [TS]

  directory start by reading the contents [TS]

  of the directory and then listen and [TS]

  will tell you if anything changed and [TS]

  when they tell you something change [TS]

  reread the contents of the directory and [TS]

  compare it to the copy you had in memory [TS]

  and see what changed then you can tell [TS]

  oh this file is new this file is [TS]

  modified you know that's kind of [TS]

  annoying and cumbersome or whatever [TS]

  but you can do that sort of at your [TS]

  leisure without angering the colonel [TS]

  who's sending a little chocolates down [TS]

  the assembly line and without being [TS]

  responsible or pulling chocolates off [TS]

  the assembly line fast enough to keep up [TS]

  with what's going on with file i/o now [TS]

  all this is a very roundabout way to [TS]

  talk about how ZFS tackles the same [TS]

  problem ZFS doesn't have to have [TS]

  something in the kernel that listens to [TS]

  file system changes and writes them out [TS]

  to a compress log file [TS]

  there is no continually growing compress [TS]

  log of all the things that have happened [TS]

  in a ZFS system and the thing that has [TS]

  to send the difference between one [TS]

  snapshot and another doesn't read a big [TS]

  log of the things that happen the file [TS]

  system itself is essentially a log of [TS]

  all the things that it happened because [TS]

  it doesn't overwrite data it only writes [TS]

  data to new location so it's basically [TS]

  got a long stream of new data just being [TS]

  added and added and added and then being [TS]

  reaped off the back when it's no longer [TS]

  in use so the thing that has to find the [TS]

  the differences between one snapshot and [TS]

  another first of all it doesn't have to [TS]

  do with the file level because that's [TS]

  not how the file system works all the [TS]

  i/o is at a block level so if you change [TS]

  three bytes inside a two gigabyte file [TS]

  you're going to have it in ZFS those [TS]

  three bytes written to a new location [TS]

  the old three bytes in an old location [TS]

  and you'll know since last time I look [TS]

  at this or since from one snapshot the [TS]

  other one snapshot includes this this [TS]

  those three bytes in this block and the [TS]

  other snapshot includes those two bytes [TS]

  in this block and it can just send those [TS]

  bytes from one place to the other it [TS]

  doesn't have to send the entire [TS]

  two-qubit file it doesn't have to keep [TS]

  track of the fact that file change [TS]

  because when it sees it that this [TS]

  snapshot has one block for this file in [TS]

  snapshot has the other block for the [TS]

  file everything's check sounds it knows [TS]

  the contents of the of the blocks are [TS]

  different it knows which one came later [TS]

  it doesn't it's sort of like inherent in [TS]

  the structure of the file in itself what [TS]

  has happened who and what the [TS]

  differences are between these things [TS]

  it's not instantaneous to figure out [TS]

  what the block level dips are but it [TS]

  doesn't require scanning all of the [TS]

  source and all definition destination [TS]

  and comparing them that's one of the [TS]

  reasons why this type of strategy long [TS]

  ago is called log structured file format [TS]

  or does everything is appended to the [TS]

  end as a big linear stream of new things [TS]

  that have happened [TS]

  and snapshotting just really tells it [TS]

  which sets of these are relevant to each [TS]

  other so this is one of the reasons [TS]

  people were excited about ZFS you can [TS]

  imagine a time machine you rebuilt on [TS]

  top of something like ZFS your time [TS]

  machine backups would look like this [TS]

  first of all you could do Time Machine [TS]

  backups locally now people know about [TS]

  Lian know that it has local time machine [TS]

  backups too because this is important [TS]

  for laptop users which is increasingly [TS]

  pretty much a close approximation of Mac [TS]

  users because vast majority people by [TS]

  Macs are buying laptops and they tend to [TS]

  just have one disk so what do you do for [TS]

  backups well Apple since their straddle [TS]

  with H OS plus just add in another [TS]

  mechanism whereby anytime a file is [TS]

  saved it writes out it tries to write [TS]

  out the byte level dips between that [TS]

  file in this whole separate directory [TS]

  full of file there everything that [TS]

  happens in HFS+ is built on top of the [TS]

  filesystem rather being built into it so [TS]

  it's another bunch of files and another [TS]

  bunch of database files it's like a [TS]

  sequel Lite database and little chunks [TS]

  of files you know Selway someone saved [TS]

  the new version of this file well since [TS]

  there are laptop user and they're not [TS]

  connected to their time machine disk [TS]

  right now make a note of this change and [TS]

  figure out which bytes changed since [TS]

  last version of this file and write them [TS]

  out to the separate directory in a [TS]

  separate file it's all in an [TS]

  application-level type of thing and it's [TS]

  not particularly elegant or nice but [TS]

  this is something that they're trying to [TS]

  do to protect Mac users who are [TS]

  frequently away from a time machine [TS]

  drive well if you have ZFS you could [TS]

  take snapshots of the entire disk every [TS]

  five minutes or every time you save or [TS]

  any you know instantaneously at any time [TS]

  you wanted they wouldn't require any [TS]

  demons listening for things or any [TS]

  separate database files or any sequel [TS]

  Lite database of different changed [TS]

  ranges or any secret directory full of [TS]

  chunks of old files with directories [TS]

  named vector dates all this stuff all [TS]

  that stuff wouldn't have to exist it [TS]

  would be inherent in the file system [TS]

  itself so it should do a time machine [TS]

  back up you would take a snapshot and [TS]

  the reason you want to take a snapshot [TS]

  is because you want to do a backup as an [TS]

  ax point in time that's nothing that [TS]

  Time Machine can't do when you start [TS]

  doing your time machine backup you might [TS]

  realize as you're using a computer that [TS]

  by the time you finish doing your time [TS]

  machine back up the disk is changed so [TS]

  what what is the time machine backup [TS]

  right is it a backup as of the time [TS]

  it started or as at the time it ended [TS]

  it's kind of a melange of all of them [TS]

  combined like if if you're editing a [TS]

  file and you see the little time machine [TS]

  spinning thing you know it's backing up [TS]

  right now and you hit save you have no [TS]

  idea what the version that of that thing [TS]

  got saved in fact what time machine will [TS]

  do is after it's done running it will [TS]

  quickly see all right while I was [TS]

  running it anything happen and it will [TS]

  run again and it only does that leg once [TS]

  or twice and tries to sort of catch up [TS]

  but it's not a point in time of anything [TS]

  it's just kind of a mixture of different [TS]

  of different things so if you had ZFS [TS]

  you would take a snapshot and say look [TS]

  I've got a completely consistent point [TS]

  in time snapshot instantaneously this is [TS]

  about the context of the disk was then [TS]

  you can make a clone a writable snapshot [TS]

  of that on the backup disk or you know [TS]

  in place wherever you want to do it and [TS]

  then you could send the block level [TS]

  diffs between the snapshot in the clone [TS]

  any time something changes so you would [TS]

  have on the backup disk a series a [TS]

  series of yeah they have to be right up [TS]

  because you have to send the block level [TS]

  disk you would have a series of clones [TS]

  that you send the block level dips to [TS]

  and to figure out what has changed this [TS]

  last time you don't have to read some [TS]

  big log of stuff or anything like that [TS]

  and if a two gigabyte file has changed [TS]

  you don't have to send the entire two [TS]

  gig file you can just change send the [TS]

  blocks that have been modified it would [TS]

  be vastly more efficient vastly more [TS]

  reliable and just the number of neat [TS]

  things you could do with a constant time [TS]

  snapshots and clones and block little [TS]

  easy block level dips is just tremendous [TS]

  you might still need a mechanism but [TS]

  through which applications can listen [TS]

  for file system events and things that [TS]

  are changed but then that that suddenly [TS]

  takes it off of the path of things like [TS]

  backups ah which can't possibly miss [TS]

  anything because they don't have to [TS]

  worry about anything it's inherent in [TS]

  the file system itself there's no way [TS]

  they could possibly miss any event [TS]

  because there are no events being [TS]

  written it's just that's how the file [TS]

  system is structured I'm one one more [TS]

  final thing I want to talk about it [TS]

  there's more things receive us usually [TS]

  the ZFS wikipedia pages get a lot of [TS]

  stuff deduplication this is sort of an [TS]

  Enterprise II feature that ZFS has [TS]

  because it's an enterprise a file system [TS]

  what this means is that so [TS]

  ZFS is basically managing these big [TS]

  blobs of data a single large file could [TS]

  be made up of lots of variable size [TS]

  blobs that all make up the entire piece [TS]

  of data and each one of these blobs is [TS]

  check sums of [TS]

  that's what ZFS does well once it has [TS]

  these check sums which are small [TS]

  compared to the data you know you could [TS]

  have I don't know how big the blocks are [TS]

  but like a megabyte 512 kilobytes or [TS]

  whatever the checksum itself is like you [TS]

  know 256 bits it's very small so given [TS]

  that you have all these check sums what [TS]

  you can do is when you say you write out [TS]

  a 2 megabyte file and then you write out [TS]

  another juvett file a week later that [TS]

  happens to have the exact same contents [TS]

  as that first 2 megabyte file why are [TS]

  you storing the exact same 2 megabytes [TS]

  in two different places on your disk [TS]

  since ZFS checksum both of those 2 mega [TS]

  byte blocks say they're in stored in big [TS]

  contiguous 2 megabyte block it knows [TS]

  that they're the same and what it can do [TS]

  is say well we don't need two copies of [TS]

  this 2 megabyte file somewhere on disk [TS]

  I know these the contents are the same [TS]

  because the checksum is matched why [TS]

  don't I just get rid of one of them and [TS]

  now both of those files are talking to [TS]

  the same 2 megabyte block on disk that's [TS]

  that's deduplication and it happens at a [TS]

  block level not at a file level so if [TS]

  you have a 3 gigabyte file and there [TS]

  like 1 megabyte regions within them that [TS]

  are identical any region that's [TS]

  identically when we pulled out so one [TS]

  example is say some file format has a [TS]

  long header on it it's always identical [TS]

  for all files it's just like our [TS]

  preamble or PO stamp or something and [TS]

  you have hundreds and hundreds of those [TS]

  types of files you only have to store [TS]

  that preamble or post amble in the file [TS]

  format once for all those hundreds of [TS]

  files so you can get disk space back by [TS]

  finding out which portions of the file [TS]

  are the same as portions of other files [TS]

  and only storing one of those portions [TS]

  this is something that happens sort of [TS]

  offline where it doesn't happen [TS]

  necessarily in real-time or whatever but [TS]

  it's a way that you can say look my disk [TS]

  is almost full there any way you can [TS]

  give me more space it will find any [TS]

  little piece of any file it's the same [TS]

  as any other piece of any other little [TS]

  file and check out one of them or check [TS]

  out all the mix-up for one ah and why [TS]

  does this works like what about when [TS]

  something needs to modify that again [TS]

  well it's copy and write anything you [TS]

  want about anything you're always [TS]

  writing to a new location this doesn't [TS]

  hurt you at all in terms of making your [TS]

  data frozen in place or unable to modify [TS]

  this is one of the consequences of [TS]

  having a file system or you just write [TS]

  things out and never overwrite data in [TS]

  place [TS]

  uh and it's part of the file system so [TS]

  if Apple had deduplication as part of [TS]

  their file system I I would really not [TS]

  like to see them try to add this HFS [TS]

  boss but if they had a file so isn't it [TS]

  supported this if you just started to [TS]

  get full one of the things the operating [TS]

  system could do is say ok let me just go [TS]

  look for let me just start deduplicating [TS]

  blocks and let me find some common [TS]

  regions of files and it doesn't have to [TS]

  scan every single file to find common [TS]

  regions it just has to look at the check [TS]

  sums which are very very small compared [TS]

  to the file so we're going to say you [TS]

  know if it stores all its check sums in [TS]

  kind of like a sorted table a checksum [TS]

  somewhere they can very quickly find [TS]

  check sums are the same and just start [TS]

  reaping data and marking it as free [TS]

  space because it only needs one copy so [TS]

  that's it for my neat features of ZFS [TS]

  that HFS+ doesn't have I hope I've [TS]

  wetted people's appetites for ZFS [TS]

  another cool new file system there's not [TS]

  much they can do about it except hope [TS]

  wait and hope they can be as [TS]

  dissatisfied as I am I put a link in the [TS]

  show notes two more links to Jeff bond [TS]

  wigs blog about these carpet topics a an [TS]

  old ACM queue interview from 2007 with [TS]

  Jeff bond wick and a link to the [TS]

  original like PowerPoint presentation [TS]

  for ZFS it's called ZFS the last word in [TS]

  file systems these are all classic [TS]

  pieces of literature from ZFS oh and by [TS]

  the way on that on the butter FS thing [TS]

  that I link to as well there are many [TS]

  links to the academic PDF so the [TS]

  academic paper is on which is based even [TS]

  if you're someone who doesn't ever click [TS]

  on links to PDFs of academic papers did [TS]

  you expect it to be just horrible and [TS]

  dry and impossible to read these are [TS]

  surprisingly readable to anybody with [TS]

  even a tiny bit of a CS background so I [TS]

  encourage you to actually click on the [TS]

  PDFs and read the the academic papers [TS]

  they're short they're not too [TS]

  complicated they're not filled with [TS]

  equations that you're not going to [TS]

  understand it's pretty basic stuff that [TS]

  each oh yeah I was running my iPad 3 [TS]

  predictions yes of course I've been [TS]

  going back and forth in this when I was [TS]

  much more sure like two weeks ago but [TS]

  now as we get closer I don't know it [TS]

  seems like there's there are a few [TS]

  points [TS]

  of that the rumors are addressing a few [TS]

  points the first point is the screen [TS]

  everybody seems to agree there will be a [TS]

  Retina display there's a second question [TS]

  which is will there continue to be a [TS]

  previous generation of iPad like the [TS]

  iPad 2 along with the new one there is [TS]

  the question of price points and and [TS]

  then finally I believe there's a [TS]

  question of naming conventions will this [TS]

  be the iPad 2s the iPad 3 the iPad HD [TS]

  there's even more question about yeah so [TS]

  those get paid once I'd love to hear all [TS]

  of them [TS]

  the only thing everyone agrees on is [TS]

  Retina display yeah I also agree on that [TS]

  yes [TS]

  I mean we've we've all known this is [TS]

  kind of like we all know that the the [TS]

  MacBook Pro without an optical drive is [TS]

  coming like we know it's coming yeah [TS]

  matter of time and now we know the time [TS]

  it's that's definitely come okay ah [TS]

  I am almost as certain that Apple will [TS]

  continue to sell an iPad with a non [TS]

  Retina display almost hundred percent [TS]

  certain because it just seems crazy to [TS]

  me that they wouldn't will it be the new [TS]

  iPad or will it be the existing iPad [TS]

  with without the Retina display like a [TS]

  two versions of the new one come out or [TS]

  the one that we currently have today as [TS]

  the iPad 2 continues to be sold [TS]

  I think they keep selling the old one [TS]

  because that I think that the economies [TS]

  of scale network well they already have [TS]

  the tooling the assembly lines the [TS]

  people you know everything ready to make [TS]

  those exact things and maybe you stamp [TS]

  something different on the back of it [TS]

  for branding if they wanted to change it [TS]

  but it's the same reason they sell still [TS]

  sell the iPhone 4 and the 3GS like once [TS]

  you do the investment in making those [TS]

  products it's much cheaper just to keep [TS]

  making them and if they you know if they [TS]

  want to do they could change something [TS]

  about the name but I don't even think [TS]

  they will I think it will be just be [TS]

  exactly identical it'll be the iPad 2 I [TS]

  don't think we'll even bother renaming [TS]

  it to be like the iPad 3 slow version [TS]

  you know because that would just be a [TS]

  matter of stamping something different [TS]

  on the back of the thing but I think [TS]

  they'll just continue to sell the old [TS]

  version the other one that I was really [TS]

  really sure about and now I'm kind of [TS]

  waffling but I'm still mostly sure is no [TS]

  LTE I was so sure there would be no LTE [TS]

  but ibex I'm reading too many rumor [TS]

  sites and it getting into my head I'm [TS]

  pretty certain olt that's I feel [TS]

  strongly about that I but now [TS]

  I have doubts doubts have been so about [TS]

  this rumor site so I should sure not [TS]

  read them should just stuck to my guns [TS]

  and say no you're still saying no LT [TS]

  yeah what does the price point at price [TS]

  is another thing that I used to be more [TS]

  certain of and now I'm doubting I I [TS]

  think I used to think and still kind of [TS]

  do that Apple at least shouldn't be [TS]

  afraid to sell a higher priced model to [TS]

  have one model available that is more [TS]

  expensive than any of the current models [TS]

  available i but now I think that Apple [TS]

  is afraid of that I don't think they [TS]

  should be I think they should extend in [TS]

  both directions I think they should be [TS]

  able to have high-end model for people [TS]

  who want it and if the high amount is [TS]

  like oh it's got 128 gigs of flash and [TS]

  the retina display and you know the LTE [TS]

  radio because I'm wrong on that or you [TS]

  know something like push out in the high [TS]

  end but as we get closer I think man [TS]

  Apple just really does not want to go in [TS]

  that direction the direction I think [TS]

  more people can agree on is they're [TS]

  going to try to push lower if that means [TS]

  a price drop on the iPad - if that means [TS]

  you know like whatever that means they [TS]

  want to keep pushing down that always [TS]

  want to push down push down with the [TS]

  iPods until they're so incredibly cheap [TS]

  they cost about as much as a case you [TS]

  get an iPod shuffle the same prices you [TS]

  can get like an iPhone good iPhone case [TS]

  all right pushing down down down I think [TS]

  the way they push down they're going to [TS]

  push down with the iPads the same way [TS]

  they would have with the iPhones which [TS]

  is selling previous models the real [TS]

  question is can Apple afford to sell an [TS]

  iPad with a Retina display at the same [TS]

  price points as the iPad 2 for the same [TS]

  other features so if you take an iPad 2 [TS]

  that has a certain amount of flash [TS]

  memory and it has Wi-Fi and it has 3G [TS]

  can you just swap out that display and [TS]

  sell it at the same price certainly albo [TS]

  can they've got room in their margins to [TS]

  sell it with that but if they were going [TS]

  to I thought I would have seen more of a [TS]

  warning about future margins going as a [TS]

  thing for Horus to talk about nothing [TS]

  but in the earnings call I would have [TS]

  thought that there would have been more [TS]

  warnings about our margins are going to [TS]

  be lower next quarter so just keep that [TS]

  in mind and if you ask them why they [TS]

  would say we don't talk about our [TS]

  announced products and component [TS]

  prices in blah blah blah like they would [TS]

  be evasive as they always are but if [TS]

  they're going to have the exact same [TS]

  price points as the iPad 2 and maintain [TS]

  their margins I don't think it's [TS]

  possible because the screen just plain [TS]

  cost more no matter what kind of [TS]

  sweetheart deal they got on these [TS]

  screens and I'm sure they got a good one [TS]

  it doesn't cost the same as the old [TS]

  screen their margins have to go down so [TS]

  given that I think they may do some [TS]

  fancy mumbo-jumbo by shuffling the other [TS]

  features of the things so that the price [TS]

  points remain the same but what you're [TS]

  getting for the current cost of a 32 gig [TS]

  Wi-Fi only iPad isn't is not exactly [TS]

  comparable to what you get with the same [TS]

  price point a Retina Display iPad this [TS]

  is all lots of waffling which is to say [TS]

  that it's clear that Apple wants to hit [TS]

  exactly the same price points and extend [TS]

  down that's the ideal they want exactly [TS]

  the same price as the iPod - iPad - but [TS]

  now the price range goes lower so if you [TS]

  can't afford even the cheapest previous [TS]

  iPad now maybe you can because the [TS]

  cheapest model is even cheaper and yeah [TS]

  maybe it's last year's model I think [TS]

  they should extend up but I think [TS]

  they're afraid to maybe if they [TS]

  introduced one with more with more flash [TS]

  like a 128 gig one they might extend up [TS]

  but so if I had if you had to pin me [TS]

  down I would say same price points and [TS]

  the price range extends downward but not [TS]

  upward what else do we have the name [TS]

  three versus 2's I'm not really [TS]

  interested in that because it's not [TS]

  going to change the product that's just [TS]

  as well you refuse to comment to s3 a [TS]

  check those are the ones that are [TS]

  floating around today it probably [TS]

  depends on if it has a new case ah the [TS]

  rumors are that the case is going to be [TS]

  slightly thicker if that's the case [TS]

  they're going to call three if the case [TS]

  is exactly the same that you get away [TS]

  with 2s and I think they might do that [TS]

  but I'm leaning towards three and a [TS]

  different back slightly different back [TS]

  camera I'm assuming the camera will be [TS]

  improved but I also would not be shocked [TS]

  if it wasn't about other people would be [TS]

  shocked if it was I know but other [TS]

  people would flip out this could be one [TS]

  of those ways they maintain the price [TS]

  point because that camera that [TS]

  piece-of-crap camera in the [TS]

  back of the iPad it's got to cost them [TS]

  two cents by now and so they said well [TS]

  how can we hit the price point that we [TS]

  want to hit and how can we not raise [TS]

  prices given that we're putting the [TS]

  super expensive screen in there well [TS]

  keep the same crappy camera ah do you [TS]

  think a lot of people use the iPad 2 [TS]

  cameras for the FaceTime video [TS]

  I bet the one in the front gets much [TS]

  more use than the one in the back yeah [TS]

  but you know we've all seen the people [TS]

  holding up the iPads to take pictures of [TS]

  stuff it happens everyone is like all [TS]

  that now they can put in the iPhone 4 [TS]

  camera in there if not the 4s camera [TS]

  you're right they could but will they [TS]

  Adam I'm I doubt that a lot the CPU a 6 [TS]

  verses a improved variant of the a5 I'm [TS]

  leaning pretty heavily towards the [TS]

  variant of the a5 which means a [TS]

  dual-core chip instead of quad-core I [TS]

  don't think the a6 is ready yet and I [TS]

  think the a5 with a better GPU and [TS]

  whatever in their hands as they need for [TS]

  the big honkin screen will be okay and [TS]

  some people have said that even the a5 [TS]

  could handle a Retina display as is now [TS]

  maybe just a few other things need to be [TS]

  tweaks so what else is left a 5x same [TS]

  price points HD no LTE possibly the same [TS]

  crappy camera will there be anything [TS]

  announced at the event such as a new or [TS]

  an actual new version of the Apple TV [TS]

  something different that supports 1080p [TS]

  better newer CPU inside I would really [TS]

  like it if I I'm leaning towards know [TS]

  nothing about Apple TV but if there is [TS]

  something about Apple TV I'm I'm [TS]

  thinking it's a new little black Apple [TS]

  TV box that just has like bumped specs [TS]

  basically it's not like an Apple [TS]

  television set it works the same as the [TS]

  current little black square but you know [TS]

  supports 1080p or whatever you know but [TS]

  nothing big revolutionary which would [TS]

  save the Apple television set Apple [TS]

  saves TV whatever pipe dream that we all [TS]

  have for an end-of-the-year event and [TS]

  just give us 2012 to mostly say if [TS]

  you're going to buy an Apple TV now you [TS]

  can get an apple TV 3 which looks [TS]

  exactly like the Apple TV 2 it's little [TS]

  black square [TS]

  maybe the biggest change they make is [TS]

  that the the remote is Bluetooth but I [TS]

  don't think they've changed that it [TS]

  would be the same so there's going to be [TS]

  no announcement or it's going to be a [TS]

  little dinky black Apple TV 3 type thing [TS]

  su degree what you agree with John [TS]

  Gruber who said laser focus on the iPad [TS]

  and and maybe something supportive in [TS]

  the Apple TV to go along with 1080p in [TS]

  the iPad or something like yeah why [TS]

  would you take away from the iPad event [TS]

  by by showing the new version of the [TS]

  Apple TV and the flip side is if there [TS]

  was like a really interesting TV product [TS]

  like whatever Apple plans to do in TV [TS]

  that's different than this little box [TS]

  they've been making that would have been [TS]

  evident on the invitation because that's [TS]

  not something you do is like oh you know [TS]

  oh by the way one more thing we have an [TS]

  entirely new crazy product that is just [TS]

  insane that's going to revolutionize [TS]

  tell that's what the event is about if [TS]

  they're going to ever come out for you [TS]

  know we're gonna revolutionize [TS]

  television that's not going to be an [TS]

  afterthought in the event I just don't [TS]

  see that so I'm thinking nothing about [TS]

  television and then people would just be [TS]

  sad like I say what the heck am i except [TS]

  for maybe itunes store selling HD stuff [TS]

  that seems reasonable to do because [TS]

  there's like I get this read an iPad [TS]

  yeah my games look great but I gotta [TS]

  watch the 720p movies on it and I get [TS]

  zoomed up that's kind of icky so 1080p [TS]

  content could have come at any time [TS]

  because max can play 1080p now so I [TS]

  don't you know whatever Apple is waiting [TS]

  on to unleash the 1080p video content [TS]

  that could be waiting on deals or [TS]

  re-encoding video or they gotta [TS]

  renegotiate something at all those deals [TS]

  is kinda like the Beatles it's like what [TS]

  was stopping the Beatles thing from [TS]

  happening [TS]

  just stupid negotiation so I'll add the [TS]

  last question I've heard a lot of people [TS]

  out there speculating what apps or what [TS]

  kind of apps will be demoed and featured [TS]

  to show off the new Retina display I've [TS]

  heard you know if you remember earlier [TS]

  this week Adobe released photo photo [TS]

  shop touch and perhaps I hadn't thought [TS]

  of this but a couple people on Twitter [TS]

  and said to me oh I think that app was [TS]

  leaked it shouldn't have come out that [TS]

  soon it was supposed to be held back it [TS]

  came out too soon something like that [TS]

  and that was supposed to be the big app [TS]

  that Apple demoed on stage perhaps [TS]

  that's true perhaps not question for you [TS]

  what what kind of app do you think that [TS]

  they'll use to highlight the new Retina [TS]

  display games [TS]

  so given they're gonna do games because [TS]

  that it combines everything they want to [TS]

  show they want to show the more power [TS]

  slightly more probable GPU they want to [TS]

  show that the graphics aren't Jaggi or [TS]

  anything drawing app is probably a good [TS]

  choice the real problem as Gruber [TS]

  pointed out is the Retina display is [TS]

  very difficult to demonstrate because [TS]

  you're probably showing it in some [TS]

  little movie and like YouTube or even [TS]

  Apple's own HD movies you can't see the [TS]

  difference of like someone filming a [TS]

  Retina display and then compressing that [TS]

  video you know I think he said you need [TS]

  a Retina display to appreciate a Retina [TS]

  display [TS]

  yeah and they can do it in slides [TS]

  because on slides they'll zoom in the [TS]

  slide they'll show the zoomed in picture [TS]

  of a sprite from a game looking all [TS]

  blocky and I'll show the nice smooth [TS]

  line it's very easy to demonstrate in a [TS]

  slide but when they have the actual iPad [TS]

  thing in their hand what are they going [TS]

  to demonstrate that I think this is [TS]

  actually a problem for for this iPad [TS]

  announcement because we all know that is [TS]

  going to be Retina display we all know [TS]

  it's impossible to demonstrate and it's [TS]

  like what applications is this Retina [TS]

  display make possible that previously [TS]

  we're impossible [TS]

  pretty much none it just makes a whole [TS]

  bunch of them better and we all [TS]

  understand that but then like what do [TS]

  you talk about for the rest of the time [TS]

  if it's just an a5 X you can say oh and [TS]

  the graphics are you know X amount [TS]

  faster if it's not quad-core you can't [TS]

  say on this a double number of cores but [TS]

  even that like I demonstrate speed ah if [TS]

  even if is LTE that you know I don't [TS]

  know that I guess I guess I could look [TS]

  back at what they demonstrated the iPad [TS]

  2 announcement there's not something [TS]

  kintyre lee knew that you couldn't do [TS]

  before that you can do now it just makes [TS]

  things better and faster it's kind of [TS]

  like a it's not a speed bump but it's [TS]

  kind of like when they do would do all [TS]

  new all new Powerbooks and what would be [TS]

  different about them well they'd have [TS]

  faster CPUs that hold more memory that [TS]

  this could be able to get bigger and [TS]

  maybe the screen is nicer in some way [TS]

  but that's it's not a speed bump that [TS]

  I'll then have a new case so this could [TS]

  have a new case too so I think their [TS]

  ability to do cool demos with the iPad [TS]

  is decreasing and they are they're going [TS]

  to be forced to demonstrate their market [TS]

  strength by saying look at all the cool [TS]

  applications we have so they're probably [TS]

  going to demo applications that if you [TS]

  think about it like well you could add [TS]

  that application on the iPad [TS]

  - and yeah it looks nice easy to write [TS]

  in a display but there's nothing about [TS]

  this new iPad that now makes this [TS]

  application possible it wasn't possibly [TS]

  for a good example is Microsoft Office [TS]

  Microsoft Office could exist in for the [TS]

  iPad 1 or the iPad 2 it happens to be if [TS]

  it happens to be available now that [TS]

  would be a good thing to demonstrate to [TS]

  say look we've got Microsoft Office on [TS]

  the iPad isn't that awesome and the same [TS]

  thing for games or any medical imaging [TS]

  application you want to show I don't [TS]

  really know what their pitch is going to [TS]

  be I just know that their job is getting [TS]

  harder it's much easier to show the iPad [TS]

  but the first time and the second one [TS]

  it's like thinner and his cameras and [TS]

  stuff but now it's not like going to [TS]

  keep adding geegaws and making new [TS]

  things possible on the iPad it's just [TS]

  going to be a demonstration of look at [TS]

  the neat new applications that are out [TS]

  for the iPad now a dock connector I [TS]

  think the doctors say is the same do you [TS]

  think that we will see a new dock [TS]

  connector sometime in the near future [TS]

  I really hope so maybe not this revision [TS]

  it almost sounds like we could do a [TS]

  whole show just talking about connectors [TS]

  like we did that time I wish I did [TS]

  oh i phone 5 is my bed for connector [TS]

  changes will it happen just for the [TS]

  iPhone or will it happen across the [TS]

  board album for the iPhone first that's [TS]

  my prediction okay no so it be with the [TS]

  5 and it will not some people have [TS]

  speculated will it be a u.s. mini USB or [TS]

  micro USB and you're saying no it will [TS]

  just as we have the previous NOC [TS]

  connector it will be its own new thing [TS]

  that won't be compatible with anything [TS]

  else they'll probably sell an adapter [TS]

  for it but it's not like they're going [TS]

  to take a Thunderbolt port there or some [TS]

  kind of other that it will be Apple's [TS]

  own new port that we've never seen [TS]

  before I think I would love to use [TS]

  Thunderbolt I don't think there's room [TS]

  inside an iPhone right the controller [TS]

  chips to support this just it don't fit [TS]

  it's not going to not going to make it [TS]

  and never mind that the connector itself [TS]

  would has to have all this chips in and [TS]

  everything that's also big and bulky so [TS]

  they would love that Thunderbolt and [TS]

  thunderbolt is probably actually [TS]

  feasible on the iPad but not on the [TS]

  phone I think the reason they're going [TS]

  to go to a new connector on the phone if [TS]

  it doesn't happen in the iPhone 5 then [TS]

  maybe the 6 or something is that they're [TS]

  going to keep making that damn phone [TS]

  thinner and at a certain point it be [TS]

  like that the phone at the thin end is [TS]

  like [TS]

  as thin is the 30-pin connector is and [TS]

  it starts to become your limiting factor [TS]

  and you know also I've seen you know the [TS]

  people taking screens the inside how [TS]

  much that how much room on the inside of [TS]

  the phone that connector takes up it [TS]

  starts to become an issue it's like this [TS]

  this stupid connector is dictating the [TS]

  design of our phone to a large extent [TS]

  because it's taking up room on the [TS]

  inside and we can't make the edges any [TS]

  thinner than this because it gets a [TS]

  little bit ridiculous the ipod touches [TS]

  close to the as thin as you can make it [TS]

  and still support that 30 pin connector [TS]

  in a reasonable manner so I think it's [TS]

  it's days are numbered and and one of [TS]

  the easy predictions is no replacement [TS]

  the connector is gone and there is no [TS]

  other connector right they'll use they [TS]

  use Wi-Fi for syncing and capacitive [TS]

  charging further further charging and [TS]

  that's it there you don't plug your [TS]

  phone into anything well you could for [TS]

  charging you can always do a little [TS]

  pinhole like charging you can get away [TS]

  with a really tiny hole you know we [TS]

  think they would be able to repurpose [TS]

  the way they've done with the shuffle [TS]

  where you can charge with the headphone [TS]

  jack yeah it's possible you don't charge [TS]

  for the headphone jack on the shuffle or [TS]

  you charge with the well I guess it's [TS]

  the USB connector through the headphone [TS]

  jack but yeah right but it goes it in [TS]

  other words there's no separate there's [TS]

  only one hole in the shuffle and they [TS]

  could do the same thing with the iPhone [TS]

  yeah that's that's future if you chose [TS]

  like iPhone 6 or 7 like if we still have [TS]

  this 30 min connector by the time we're [TS]

  talking about iPhone 6 this time so I [TS]

  was thinking that that entire connector [TS]

  goes away which is it's weird because [TS]

  that connector doesn't just sync and [TS]

  charge it does all sorts of stuff with [TS]

  peripherals but at a certain point when [TS]

  this thing becomes a certain in [TS]

  particular becomes a certain size how [TS]

  can you attach something to this skinny [TS]

  little card it's just basically like a [TS]

  little screen in your aunt how do you [TS]

  attach something to that through to a [TS]

  port that connects with it it becomes [TS]

  cumbersome and unwieldy and just weird [TS]

  you know it's what weird already to [TS]

  attach things to the bottom of your [TS]

  phone even it's like a battery pack or [TS]

  something so I want the I want the [TS]

  30-pin connector go away and if [TS]

  something has to replace that I want to [TS]

  be much smaller so I'm predicting iPhone [TS]

  5 but this this side new iPad I'm almost [TS]

  certain will have 30 pin connector on [TS]

  the bottom just like usual okay [TS]

  and I have no inside info just this [TS]

  disclaimer in case people think I [TS]

  actually know stuff I know nothing [TS]

  you'll see how wrong I am [TS]

  and when March 7th comes if you're [TS]

  writing down things that I predicted [TS]

  just check them off at it then that's it [TS]

  hundred minutes pretty good okay have a [TS]

  good week Joe you do now [TS]

  you [TS]