1. If you're receiving a message that you are banned from the Current Events or Politics forums, it's not you specifically: those forums have been hidden for all users. For more info, see here.
    Dismiss Notice

Automated backups and offsite storage

Discussion in 'Storage & Backup' started by Lazlo999, Feb 18, 2021.

  1. Lazlo999

    Lazlo999 Member

    Joined:
    Mar 28, 2019
    Messages:
    48
    Location:
    Brisbane
    Hi guys,

    This is my first thread been a long time stalker :) its probably already been asked and spoken about thousands of times but I felt like this may be worth a new thread. (correct me if I'm wrong)

    Ok so, I work for an electrical company we build switchboards mainly for the mining industry (somewhat sensitive data) and we need to backup our server periodically. Currently we use uranium backup (free) and it backs up to an external drive located at the server. (yes, I know not ideal hence this thread)

    What I'd like is automated backups made then pushed offsite (to my house) to be stored incase of fire or theft of the server so on. If we loose that we loose pretty much everything. The server contains all our work past and present along with our entire MYOB Exo database roughly 600GB of data in total ATM and growing.

    I had planned to run the backups like usual dumping them into an FTP folder that would be constantly connected the ftp client at my house then DL the newer files. Is this a stuff around? Would I be best to just get another external and physically take it home each week or day?
    Are there other alternatives to this idea?


    What I'm after is your opinions or answers to:
    1: What we should use for backups (hardware & software wise)
    2: Where should we keep this data? (cloud, home or elsewhere?)
    3: How we should get our data stored externally (best practice and how to get it there)
    5: Any other thoughts or ideas you may have as I am no sys admin or IT guru, just somewhat tech savvy and self taught lol

    Any help, tips ideas and thoughts you all may have would be much appreciated.


    Thank you all in advance!
     
  2. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,380
    Location:
    Brisbane
    Congratulations on the first step.

    Good Choice

    Bad Choice.

    Statistically - Ransomware/Cyber Attack/Incident is your biggest threat - but disconnected, periodic and immutable backups are key here.

    This is 1980's thinking.

    This is 1990's thinking.
    Heaps.


    1. Assuming Windows - the usual suspects are Veeam and Shadowprotect. If you're not virtualised - you can probably get away with stuff like Cloudberry and Backblaze pretty effectively. Key features you're looking for are:
    * Automated - This thing *must* run without intervention. The time you will need your backup is the time you weren't there to make it happen if it is in any way manual.
    * Report on Success or Fail - You need to know that it ran, and you need to know if it didn't run.
    * Clear path to recovery - How are you going to use this? Who has passwords/accounts to get at this backup? How is the restore going to work? You do not have a backup until you've tested it
    * Multiple Versions of backup - A key component of many cyber attacks/ransomware now is that they encrypt your data. This means you're likely to need to roll back a few backups to get a clean one. In large targetted attacks - they say 3 months is the average time an attacker has been in your network by the time you notice it. Ransomware in small business is a lot less than this typically - but definitely seen instances where people were being used for crypto mining for a while, then just got bored with that and ransom'd your shit back to you.

    2. 110% the Cloud.
    3. Ideally via the internet - anything else is manual.
    4. ????
    5. Who is going to execute your recovery? how are you going to verify your backups are usable? Do you have a local IT Guy/MSP? What do they use/recommend?
     
    Hive, fredhoon, Yehat and 2 others like this.
  3. j3ll0

    j3ll0 Member

    Joined:
    Jul 13, 2005
    Messages:
    4,799
    To give my minor emphasis to one point: The Recovery Time Objective (RTO - how long it takes to go from dead server to rebuilt available server) is going to shape the decision on any option you pursue. Having a backup in the cloud is awesome, but restoring 600GB over a 12\1 ADSL pipe is gonna be "A Bad Time" (I have no idea what internet you have access to). Consider a tiered solution that lets you keep a local copy.

    By the way, deciding on what the RTO is *ISN'T YOUR JOB*. Business leadership needs to tell you what it is.
     
    Lazlo999, fredhoon and NSanity like this.
  4. OP
    OP
    Lazlo999

    Lazlo999 Member

    Joined:
    Mar 28, 2019
    Messages:
    48
    Location:
    Brisbane
    Thanks :)

    may I ask why?
    Its not really like I'm going to be followed home and robbed for the data, my current BTC holdings would be worth more hahaha.
    secondly if I lost my home data id still have a copy at work which id easily be able to re-backup.


    Statistically - Ransomware/Cyber Attack/Incident is your biggest threat - but disconnected, periodic and immutable backups are key here.



    so what is 2020 or better yet 2030 thinking?


    Yes, all windows machines. virtualised? as in running virtual machine instances? if so, no....
    Use the back up recovery software, I guess??? :S
    I would hold the passwords maybe 1 other staff member who already has access to all the data anyhow.
    Yes, id keep multiple instances I have heaps of HDD's id be able to put into raid and make a couple of TB worth to store multiple copies.

    Any reputable companies you could recommend?

    Sorry, I removed this one.
    I'd essentially be "in charge" of this, I'm not sure how does one verify the data? does this get done via a program?
    Yes, he is ancient.... that's who set us up currently with uranium. I'm trying to move the company into the year 2000 or closer to the 90's then where we currently are at (the 70's) so for example VOIP and cloud storage or actual backups....
    id rather not use him as he charges a fortune to supply gear id be able to procure from umart at half the cost and setup myself, so if you do know of someone decent in Brisbane pm me :)

    Thank you very much for your reply! I appreciate the time you've put into your response for me :)
     
  5. Hive

    Hive Member

    Joined:
    Jul 8, 2010
    Messages:
    6,376
    Location:
    AvE
    And be sure to pull out the big red stamp (Make sure it's full of stamp fluid or whatever the hell it is stamps use to stamp marks on things) that says "Management accepts the risk" when they only approve the cheap option of rotated external drives.
     
    j3ll0 likes this.
  6. j3ll0

    j3ll0 Member

    Joined:
    Jul 13, 2005
    Messages:
    4,799
    This line right here is the one that makes me reticent to recommend anyone I have an ongoing relationship with. I mean this in the gentlest way possible, but it sounds like you have a massive sticker shock coming for you.

    It's ink, my friend. Ink.
     
    Hive and NSanity like this.
  7. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,380
    Location:
    Brisbane
    How many days will the business last without EXO?

    5?

    10?

    Work out what its worth to you.
     
  8. OP
    OP
    Lazlo999

    Lazlo999 Member

    Joined:
    Mar 28, 2019
    Messages:
    48
    Location:
    Brisbane
    Sticker shock? As in i'm afraid of the price? Well I'm not paying, just don't like to see people throw their money away. Yes, I understand that the data is possibly invaluable so saving on hardware or IT techs to get better data backup solution imo is worth it. To be fair, I understand he has to put on a markup but, I've seen quotes from him and double checked the prices I can get them for as a standard civilian off the street and its sometimes $2000 less. For example, 8k we were quoted for a server I could go get off the shelf at megabuy for 5700 :S that's roughly 35% markup... we would rather use the 2k on something else like equipment/tools for the staff that's all.

    Anyway the threads not about an IT bloke, don't wanna get off topic...

    To answer your previous post the connection is 40/10 business grade, your completely correct I had the same thoughts as too how we pull 600gb down when we need it and how long its going to take to upload. My thoughts were do I do that from home on my connection lol

    Zero, which is why I brought backing up properly to their attention. Also, why I'm here to learn a bit more for myself and to educate them (senior management).
    Ultimately I'm trying find out what you all think would be the optimal solution, before we just go with what we're told. A second, fourth or tenth opinion can't harm right?
     
  9. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,380
    Location:
    Brisbane
    So you're saying the business is worth zero? Cool - who cares.
     
  10. OP
    OP
    Lazlo999

    Lazlo999 Member

    Joined:
    Mar 28, 2019
    Messages:
    48
    Location:
    Brisbane
    no zero days sorry
     
  11. elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    44,852
    Location:
    Brisbane
    You mentioned tasks that (a) were manual, (b) used plaintext/insecure transport methods. Very much 20th century tools and methods.

    They result in what we call "Schrodinger's Backup" - i.e.: a backup that can not be proven complete, useful or reliable until the moment it's needed, and then you're rolling the dice.

    21st century tools and methods should:
    * Be automated
    * Be encrypted in transit (you may even face legislation/contract breaches and fines if you don't)
    * Offer comprehensive reporting on request
    * Offer comprehensive alerting when things go wrong
    * Be verifiable digitally that data transfer was successful (or if not, which bits were missed)
    * Offer proof that the data in both locations is accurate, and not modified or corrupted either maliciously or accidentally.
     
  12. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,380
    Location:
    Brisbane
    I only didn't mentioned encrypted transport because

    [​IMG]
     
  13. mooboyj

    mooboyj Member

    Joined:
    Sep 13, 2005
    Messages:
    1,053
    I'll bite.

    I've just setup two Synology NAS' using their inbuild software to backup our VMs and O365 environment (Google "Synology Active Backup").

    Both boxes are specced with enough resources to run our main health system and a DC without worry. I have them setup so the primary replicates to the secondary with a decent lag time ("incase").

    One NAS lives in our main office rack (we have many sites), and another lives at another site (our most secure location).

    Is this solution baller and perfect, no it isn't. In an ideal world I'd have gone Veeam and Avepoint with third party offsite backups. But I believe the solution I have is robust enough and testing restores has worked.

    Our setup cost 10K but we have higher end NAS' with large enterprise disks and I did the setup.
     
    Lazlo999 likes this.
  14. scips

    scips Member

    Joined:
    Apr 10, 2004
    Messages:
    533
    Location:
    Melb
    NSanity will hate on me for it, but with the level of exp...Just get a Datto and monthly sub, they will do full DR testing for you like twice a year as part of the sub, it can virtualize the VMs either on itself or in their cloud in a few minutes.
     
  15. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,380
    Location:
    Brisbane
    RIIIIIP all your money.

    Actually have a client complaining about it and its efficacy.

    Actually have evaluated it as an MSP - Its not very good. If you fit their use case, sure - you've just bought an expensive solution. if you don't - you're pretty typical.
     
    Last edited: Feb 23, 2021
  16. scips

    scips Member

    Joined:
    Apr 10, 2004
    Messages:
    533
    Location:
    Melb
    This seems like an ideal use case :p

    Very $, but very easymode, and when noone cares about shit (like OPs company) its just $.

    I did the same (evaluated as MSP) and ended up with 2 of them in places I've worked after, solely off knowing for antique onsite wintel environment with shitty reactive upper management its easy to offload the responsibility.

    ed: they scale like shit tho, I'd draw the limit around the 10-15 vm's mark
     
    Last edited: Feb 24, 2021
  17. OP
    OP
    Lazlo999

    Lazlo999 Member

    Joined:
    Mar 28, 2019
    Messages:
    48
    Location:
    Brisbane

    If nobody cared I wouldn't have started the thread....

    Thanks for the info and input from everyone, it was much appreciated I have learnt a bit. :)
     
  18. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    218
    For me the whole backup item is a two step method to preserve data does not matter what happens.

    First and most important step is your primary storage/ filer as this is the only place where all data are up to date. Backup is like bread, always from yesterday and any restore requires a lot of time (decide what to restore, how old, is backup data vaild, how long lasts a restore). Primary storage must guarantee validity (ex via checksums), should have crash protection (no corrupted filesystem on a crash during write. Copy on Write is a method), should offer readonly snap versioning (protected against ransomware, ex a snap per hour/current day, a snap per day/current month, a snap per month/current year). Solid hardware, an ups and a raid redundancy level that allows a failure of any two disks are the important points. To achieve this, look at newer filesystems like btrfs, ReFS or the king of all ZFS.

    Such a system keeps your data valid, allows a restore of former data states within seconds and is everything you need or want in 99,9% of all cases where a recovery is needed. I have around a dozen of ZFS filers and use them for more than 10 years. I never needed an external backup since as a restore from snaps was always possible and all what I needed (I use Windows "previous versions" on clients to access ZFS snaps from a filer).

    Second and most critical step is disaster backup to protect against amok hardware, flash, fire, theft or someone with access intentionally deletes/modifies data with a criminal intention. To protect against this I use three backup systems with additional snaps on three locations on the campus. As said I never needed but in case of a disaster I want to be protected. Encryption of backup data is an additional aspect to secure backups. With ZFS this can be done with a key per filesystem and backup/sync of encrypted data over lan.

    At home or without a second secure location, you can think of a removeable datapool or an (additional) cloud backup. For such I prefer Amazon S3 (compatible, minIO) providers due its simplicity and performance. As I cannot allow to store data on servers where others have read access, I always insist on client encrypted methods (I must own the key, nobody else). For an automated backup/sync of local data with client encryption to a remote S3 (minIO) server, I use rclone, often called the rsync for clouds, see my howto http://www.napp-it.org/doc/downloads/cloudsync.pdf

    I use OmniOS (Solaris fork) for ZFS and SMB/S3 but minIO and rclone are available on any Linux/Unix.
     
    Last edited: Mar 19, 2021
  19. mwil7034

    mwil7034 Member

    Joined:
    Jan 15, 2003
    Messages:
    614
    Location:
    Woy Woy
    Microsoft servers? Use Azure MABs to Blob Storage over IPsec tunnel

    This is coming from a 20yr linux dude

    You get monitoring, alerting, reports, on prem cache yadda yadda

    Monthly billing and you determine the retention policy etc

    It will handle linux, exchange, AD, SQL server, vmware etc as well but you probably don't care about that. The important thing is it will quiesce the db/filesystem and ensure consistent data

    Otherwise, I'd use veeam and cloud storage

    Hosting things yourself is just not worth the hassle. You have better things to do with your time...
     
  20. evilasdeath

    evilasdeath Member

    Joined:
    Jul 24, 2004
    Messages:
    5,005
    Honestly data is so cheap these days 600GB is nothing. I have maybe 1TB in backblaze B2 thats backed up from my server overnight (family photos) costs me a couple of bucks a month. It does versioning and on the off chance my wife accidently deleted all our wedding photos while re-arranging folders on the zfs share drive, it was a few mins to restore. Yes there are restore costs but thats also moot if you havr lost something important.

    Even backblaze $6 (i think thats price) whole machine backup is cheap enough really.

    You have dropbox, azure, office space as well that are easy enough to do really and while dont offer differing level of feature all far exceed using a home server or external hdd
     

Share This Page

Advertisement: