~10tb storage - Backup or Raid 6?

Discussion in 'Storage & Backup' started by Ladicius, Aug 13, 2012.

  1. Ladicius

    Ladicius (Taking a Break)

    Joined:
    May 5, 2011
    Messages:
    83
    Location:
    Dudley Park, WA
    Hi guys,

    I'm contemplating putting together a small file server (or an off the shelf NAS) to store approx 10tb or so of "linux iso's".

    What I can't decide is whether a full on backup is worth the extra cost over say a 6 or 8 drive raid 6 setup considering the data can be reacquired (rather slowly however).

    Should I just stop being cheap and cough up for two sets of drives/hardware or is raid 6 an acceptable risk in this case considering the nature of the data?

    Any thoughts?

    Cheers :thumbup:
     
  2. Jim G

    Jim G Member

    Joined:
    Jun 20, 2010
    Messages:
    1,008
    Location:
    Newcastle
    Raid will only really provide safety against one or two drives failing; not against accidental deletion of files, filesystem corruption, theft, house fire, etc. etc. - if it's valuable data, back it up. How long would it take you to recover it all (not download time, actual time you spend organising to get it all again)? How much is that time worth?
     
  3. rowan194

    rowan194 Member

    Joined:
    Jan 5, 2009
    Messages:
    2,044
    For the RAID only option: if your files don't change much, or you're mainly adding to the collection, consider using snapshots as an additional level of "protection"... an accidental delete can be restored in seconds from an earlier snapshot.

    If you're modifying or deleting a lot of data, be aware that the free space from a deleted file generally won't be released until you delete the snapshot(s) that have preserved it.
     
  4. Ghoatman

    Ghoatman Member

    Joined:
    Jan 2, 2004
    Messages:
    2,085
    Location:
    Sunshine Coast, 4551
    I've found that storing the type of data that you are, drive failures are what you'll come across. Losing 2tb of Gentoo ISO's is a bit heartbreaking. RAID will protect against that, which is the most likely scenario. Accidental deletion isn't that big of an issue (who deletes 500gb+ in one hit by accident?) If there was a fire/flood etc, you have bigger issues than losing that shit.:lol:
     
  5. OP
    OP
    Ladicius

    Ladicius (Taking a Break)

    Joined:
    May 5, 2011
    Messages:
    83
    Location:
    Dudley Park, WA
    That's an interesting point... I'm not sure how I overlooked that.

    I only considered download time, not the time it takes to find/sort it etc (I like my data neatly sorted, uniformly labeled etc).

    That makes the decision quite easy (stop being cheap :lol:)

    Cheers :thumbup:
     
  6. Jim G

    Jim G Member

    Joined:
    Jun 20, 2010
    Messages:
    1,008
    Location:
    Newcastle
    I had a relative ring the other day after they had deleted their entire NAS full of data by copying an equally-named directory with a small amount of files in it over the root directory of the NAS... I would assume mostly anyone with an OCAU account isn't at that level of accident but hey, it happens from time to time ;)
     
  7. dakiller

    dakiller (Oscillating & Impeding)

    Joined:
    Jun 27, 2001
    Messages:
    8,208
    Location:
    3844
    I don't see the need to personally backup ISO's yourself when they are backed up by everyone else as well. If you truly lost it all, then you might be sad for a bit but it isn't irreplaceable.

    Raid6 is more than enough for a case like this, you aren't going to accidentally delete the stuff and you can take a corrupt sector or 2 for files like that, and having a good raid card that does periodical consistency checks will keep your array in good order.
     
  8. davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,933
    My vote goes to stop being cheap :)
     
  9. DarkYendor

    DarkYendor Member

    Joined:
    Feb 25, 2008
    Messages:
    3,210
    Location:
    Perth
    +1. I'm about to build my new 'linux ISO' NAS, using SnapRAID for redundancy. http://snapraid.sourceforge.net
     
  10. antenna

    antenna Member

    Joined:
    Feb 26, 2007
    Messages:
    530
    Location:
    SA
    My thoughts are that a raid setup is an acceptable compromise for this sort of easily available data. Realistically only perhaps 20% of my "linux ISOs" will be viewe.. err used more than once.

    I have my server write all filenames to a txt file and mail it to an email account once a day so at least if I do lose everything I know what I need to reacquire (learned the hard way). It would take forever to get it all back but I wouldn't even try to get most of it.
     
  11. OP
    OP
    Ladicius

    Ladicius (Taking a Break)

    Joined:
    May 5, 2011
    Messages:
    83
    Location:
    Dudley Park, WA
    Ok, I think I've come up with a reasonable middle ground.


    • Raid 6 with a decent hardware controller for the bulk of the data. If it falls flat, it'd be annoying but life would go on :thumbup:
    • Backup the iso's that are starting to become hard to find/can't be found anymore.
    • Find a decent batch renamer or something to take care of the data sorting.

    Am I heading in the right direction?

    Cheers :thumbup:
     
  12. Jim G

    Jim G Member

    Joined:
    Jun 20, 2010
    Messages:
    1,008
    Location:
    Newcastle
    Why a hardware controller? ZFS is pretty darn easy to run these days (OI+Napp-it, ZFS-on-ubuntu etc.) and offers many benefits over hardware RAID or mdadm; failing that, there are plenty of other software RAID solutions which only require cheap HBAs rather than expensive hardware RAID cards and perform at least equally well for most home requirements.
     
  13. OP
    OP
    Ladicius

    Ladicius (Taking a Break)

    Joined:
    May 5, 2011
    Messages:
    83
    Location:
    Dudley Park, WA
    I was under the impression that hardware controllers were just better (easier to use, faster etc). From past experience, linux software raid seemed quite slow. Is ZFS faster?
     
  14. Jim G

    Jim G Member

    Joined:
    Jun 20, 2010
    Messages:
    1,008
    Location:
    Newcastle
    That was true a few years ago; these days even Intel admits that software raid is a viable solution. They used to be pretty big on hardware raid being superior.

    I've had mdadm arrays of 6 drives hitting 300+MB/s, same goes for zfs. Way more than enough to saturate a gigbit network connection several times over. Have a look at it again; the story has changed a fair bit.

    My experience with hardware controllers has been that if you have a controller failure you must often replace it with the same make/model, else your data isn't accessible. One advantage of using "dumb" HBAs and software RAID is that it doesn't generally matter whether you swap your drives around between the HBA, onboard ports or whatever - there's no controller adding its own info to the drives. One of our arrays has been across three systems and it was a simple case of export, disconnect, insert into new system, import, data there. AMD -> intel onboard -> intel + 3x HBA.. all worked just peachy.
     
    Last edited: Aug 13, 2012
  15. OP
    OP
    Ladicius

    Ladicius (Taking a Break)

    Joined:
    May 5, 2011
    Messages:
    83
    Location:
    Dudley Park, WA
    I think I will, cheers :thumbup:
     

Share This Page

Advertisement: