Sound Data Management Training

h2. Reliability

h3. Laptop Reliability

* 2011 PC World Laptop Reliability Survey from 63,000 readers:

** 22.6% had signifcant problems during the product's lifetime

** Of which...

*** 19% had OS problems ~1 in 25 of all laptops

*** 18% had HDD problems ~1 in 25 of all laptops

*** 10% PSU problems ~1 in 50 of all laptops

PC World 2011 - http://tinyurl.com/876qza5

h3. Hard Disk Failures

* Failure Trends In A Large Disk Drive Population

** Usenix conference on File and Storage Technologies 2007 (FAST '07)

** Eduardo Pinheiro & Wolf-Dietrich Weber, Google Inc.

* Data collected from over 100,000 disk drives at Google

* As part of repairs procedures:

** ~13% of disk drives replaced over 3 years

** ~20% of disk drives replaced over 4 years

Article: http://tinyurl.com/octz6b

[[Failure Trends In A Large Disk Drive Population|More info]]

h3. Cloud Failures

* Hazards of the Cloud: Data-Storage Service’s Crash Sets Back Researchers "Chronicle Of Higher Education, 12 May 2014":http://chronicle.com/blogs/wiredcampus/hazards-of-the-cloud-data-storage-services-crash-sets-back-researchers/52571

** "Dedoose":http://www.dedoose.com systems failed 

** Dedoose blog post 9 May 2014 : "Dedoose's Black Eye":http://blog.dedoose.com/2014/05/dedooses-black-eye-crash-and-recovery-efforts/

bq. In short, work done on one aspect of Dedoose led to the failure of another, cascading to pull down all of Dedoose.  The timing was particularly bad because it occurred in the midst of a full database encryption and backup.  This backup process, in turn, corrupted our entire storage system. 

** Crash Updates: 11 May 2014 "Crash Updates":http://blog.dedoose.com/2014/05/dedoose-crash-updates/

bq. The backup file of data through April 11th has been pieced back together, however it remains encrypted and corrupted.  We are running a variety of tools on the file to restore things to a state where we can merge the data back into the live database. 

** Recovery Efforts: 14 May 2014 "Data Recovery":http://blog.dedoose.com/2014/05/dedoose-update-significant-data-recovery-successful-how-to-save-a-project-locally/

bq. At this point, we are very happy to report that we have recovered data entered to Dedoose through March 30th.  We are still working on the details of how these data will be safely merged into the master database.

** Merging Recovered Data: 16 May 2014 "Data Merging":http://blog.dedoose.com/2014/05/dedoose-update-recovered-data-to-be-merged-tonight-and-some-reminders/

bq. The data that have been viewable on our staging environment (stage.dedoose.com/app) represent those that have been recovered for work added to Dedoose between March 2nd and March 30th.  These data will be merged back into the live database beginning tonight at 8pm PST.  It is necessary to shut down Dedoose services during this procedure which should last approximately 4 hours.