Overclockers Australia Forums

OCAU News - Wiki - QuickLinks - Pix - Sponsors  

Go Back   Overclockers Australia Forums > Specific Hardware Topics > Business & Enterprise Computing

Notices


Sign up for a free OCAU account and this ad will go away!
Search our forums with Google:
Reply
 
Thread Tools
Old 16th December 2016, 9:35 AM   #61
NSanity
Member
 
NSanity's Avatar
 
Join Date: Mar 2002
Location: Canberra
Posts: 15,813
Default

Quote:
Originally Posted by obi View Post
More "enterprisey" versions of such a system exists. If you're interested, read up about hierarchical storage management.

I've worked in a place that had the same thing, but with two tape libraries as backing.

Two copies of data on disk, two copies on tape. Two disk arrays, two libraries, two locations. Eventually disk hits a high water mark, data on disk is purged based on age/size/access/<insert parameters here>. Tape copies stay forever. Backups are handled via metadata, so you can pull a file from a tape using the metadata to get info/location.

As long as you can tune the system well, and understand the workloads, it is possible to run with such a system. Tricky to get right, and can be tricky for users to understand that they can't always get a file from the depths of the past immediately. Helps to tell them to get coffee
IBM TSM, backed by DB2 (and hilariously, TSM is what DB2 grew out of).
NSanity is offline   Reply With Quote

Join OCAU to remove this ad!
Old 16th December 2016, 9:39 AM   #62
GumbyNoTalent
Member
 
GumbyNoTalent's Avatar
 
Join Date: Jan 2003
Location: Briz Vegas
Posts: 3,550
Default

https://en.wikipedia.org/wiki/IBM_DB2

https://en.wikipedia.org/wiki/IBM_Ti...torage_Manager

Having worked IT in the 80's, DB2 1983 and TSM 1988, 2 separate products back then.
GumbyNoTalent is offline   Reply With Quote
Old 16th December 2016, 9:44 AM   #63
elvis
Old school old fool
 
elvis's Avatar
 
Join Date: Jun 2001
Location: Brisbane
Posts: 28,122
Default

Quote:
Originally Posted by Myne_h View Post
Right. So the issue is that HP and co haven't built something like it as a compliment to their spinning rust arrays. Cheers.
More or less, yeah. If it was something folks could buy and not build, there'd be more of that out in the wild.

And to be honest, that's how a lot of stuff starts. I've been building storage out of server-grade x86 Linux boxes for decades. Back then, big enterprisey folk told me I was nuts, and I had to go and buy vendor storage instead. Problem was it cost 10 times as much, and the businesses I worked for didn't have that cash.

Fast forward 20 years, and I'm running "enterprise" storage in the shape of an Oracle ZS3-2 array. What's inside it? A generic x86 server with lots of SATA storage, and a pretty web GUI over the top. Exactly the sort of thing I was building years ago. Between then and now, folks like FreeNAS offered similar things too, until the enterprise market caught up, and realised this was something they could offer people for the same sorts of profits but without the penalty of proprietary hardware costs.

Same thing goes for other tech. For example, Google weren't happy with the way large scale data storage at the database/application layer worked, so they made "Bigtable". It was their in-house code for ages, until they released the spec, and other people had a crack at building it. From Bigtable's design spawned the open source "Hadoop", maintained by the Apache foundation.

Who uses Hadoop today? Lots of people (I reckon you'd see it in 75% of Fortune500s). Heck, Microsoft even SELL it as a managed service!

The difference between 2016 and 1986 is that we're just more aware of these cycles now. Once upon a time if a company did something crazy in-house, nobody heard of it except for a few industry rumours at the annual trade show in Las Vegas. Today, it gets social media'ed to death, and everyone hears about it. The cycle is the same though - it goes from R&D project to in house tech to lots of folks tinkering to vendor offering.

With all of that said, specific to backups and archives, you really are hard pressed to find technology that out lasts tape for long term reliability. Burnable optical media suffers a lot of problems 7+ years into its life, whereas tape can last for 30+ without much effort. Likewise, finding things that can read 30 year old tapes is far less costly/difficult, and once again you've got lots of vendor support there as well for things like robot loaders and other bits that make actually using it faster/easier. I know it's all a bit of "chicken and egg" argument for that last point, but it honestly does matter when you've got federal legislation demanding you keep this data for a long time (something Facebook doesn't have).
__________________
Play old games with me!

Last edited by elvis; 16th December 2016 at 9:47 AM.
elvis is offline   Reply With Quote
Old 16th December 2016, 9:49 AM   #64
cvidler
Member
 
cvidler's Avatar
 
Join Date: Jun 2001
Location: Canberra
Posts: 10,344
Default

Quote:
Originally Posted by Myne_h View Post
I think you misunderstood. You seem to think I'm advocating a hdd cache, optical primary. I'm not. I'm advocating an optical mirror of the array. The data stored is too critical to trust to offline backup or any source which can be overwritten.

So:
Array gets mirrored to optical-dukebox.
Changes to files are essentially incrementally backed up to new disks.
One small HDD+optical would be the consolidated index of all disks.

If for some reason the primary array failed totally, the optical would be read-only and thus unaffected. It would be used to restore the entire array. Theoretically, if it was an attack of some form, you could narrow it down to the exact minute the corruption started because you cannot overwrite the optical. You can write new disks, but the old ones would be intact.




Right. So the issue is that HP and co haven't built something like it as a compliment to their spinning rust arrays. Cheers.
No I didn't misunderstand you, it's still a stupid idea for this purpose, no doubt it works well for Facebook. Because no one will care if your years old cat photos are lost.

blurays (any optical disc) are unreliable*. Tape is for backups. And you can buy WORM tapes


funnily they still have the write protect tab, not that it matters. the write once is enforced by the ROM in the tape and the drives.



*unreliable, small, slow and expensive for useful data sets. 100GB a disc @ 9MBps for $50 is rubbish capacity, performance and value. LTO7 tape holds 6TB each (up to 15TB with compression), and run at 300MBps, and are ~$100/tape at low volume, price rapidly drops as you buy in bulk.
__________________
We might eviscerate your arguments, but we won't hurt you. Honest! - Lucifers Mentor
⠠⠵
[#]
cvidler is offline   Reply With Quote
Old 16th December 2016, 9:54 AM   #65
elvis
Old school old fool
 
elvis's Avatar
 
Join Date: Jun 2001
Location: Brisbane
Posts: 28,122
Default

Quote:
Originally Posted by cvidler View Post
funnily they still have the write protect tab
Yeah, that often gives folks a laugh, but you really need it there for certain drives otherwise they freak out.

Plus from a manufacturing point of view, it's a lot easier/cheaper to just keep making them the same physical shape/size/properties.
__________________
Play old games with me!
elvis is offline   Reply With Quote
Old 16th December 2016, 10:05 AM   #66
NSanity
Member
 
NSanity's Avatar
 
Join Date: Mar 2002
Location: Canberra
Posts: 15,813
Default

Quote:
Originally Posted by GumbyNoTalent View Post
https://en.wikipedia.org/wiki/IBM_DB2

https://en.wikipedia.org/wiki/IBM_Ti...torage_Manager

Having worked IT in the 80's, DB2 1983 and TSM 1988, 2 separate products back then.
There you go. I was told this many years ago by an ex-IBM midrange guy, so have repeated it a few times.

Handy to be corrected
NSanity is offline   Reply With Quote
Old 16th December 2016, 10:10 AM   #67
GumbyNoTalent
Member
 
GumbyNoTalent's Avatar
 
Join Date: Jan 2003
Location: Briz Vegas
Posts: 3,550
Default

NP, I'm always being corrected as being 50+ the grey mush isn't as good at recall as it once was, but I fondly remember DB2 on a S/360, and not so fondly Tivoli anything when it first came out as Tivoli in the 90s.
GumbyNoTalent is offline   Reply With Quote
Old 16th December 2016, 10:49 AM   #68
PabloEscobar
Member
 
Join Date: Jan 2008
Posts: 8,901
Default

Quote:
Originally Posted by obi View Post
More "enterprisey" versions of such a system exists. If you're interested, read up about hierarchical storage management.

I've worked in a place that had the same thing, but with two tape libraries as backing.
Was this presented seamlessly to the Application layer?

We've got applications that bitch if we redirect folders, because it expects local disk access times, and gets slightly slow access times because network.

I'd hate to think what an off-the-shelf application would do if it had to wait for a robot to load up tapes/optical disks and read in data.

GoogBook et-al have the benefits of systems built specifically for the hardware they build/use. and the benefits of scale, to make it all worthwhile. the ATO dataset would just be a drop in the ocean.
PabloEscobar is online now   Reply With Quote
Old 16th December 2016, 11:19 AM   #69
elvis
Old school old fool
 
elvis's Avatar
 
Join Date: Jun 2001
Location: Brisbane
Posts: 28,122
Default

Quote:
Originally Posted by PabloEscobar View Post
Was this presented seamlessly to the Application layer?

We've got applications that bitch if we redirect folders, because it expects local disk access times, and gets slightly slow access times because network.

I'd hate to think what an off-the-shelf application would do if it had to wait for a robot to load up tapes/optical disks and read in data.

GoogBook et-al have the benefits of systems built specifically for the hardware they build/use. and the benefits of scale, to make it all worthwhile. the ATO dataset would just be a drop in the ocean.
Good ones will keep metadata online forever, so any recursive directory traversal, file size lookups, or other things will work as expected, and only when full file read access is requested will data be pulled back.

It also stops someone triggering a restore of PB of data with an accidental recursive dir/ls command.

Disk space wise, typical for our use case, metadata is one SI unit less than the matching data (i.e.: 1GB data = 1MB metdata ; 1TB data = 1GB metadata, 1PB data = 1TB metadata, etc).
__________________
Play old games with me!

Last edited by elvis; 16th December 2016 at 11:22 AM.
elvis is offline   Reply With Quote
Old 16th December 2016, 11:33 AM   #70
obi
Member
 
Join Date: Oct 2004
Posts: 132
Default

Quote:
Originally Posted by PabloEscobar View Post
Was this presented seamlessly to the Application layer?

We've got applications that bitch if we redirect folders, because it expects local disk access times, and gets slightly slow access times because network.

I'd hate to think what an off-the-shelf application would do if it had to wait for a robot to load up tapes/optical disks and read in data.

GoogBook et-al have the benefits of systems built specifically for the hardware they build/use. and the benefits of scale, to make it all worthwhile. the ATO dataset would just be a drop in the ocean.
It was seamless*.

Files would be available via SMB or NFS shares to the end user. It also helped that the sector we were working in weren't putting dinky databases on a network share and then crapping their pants.

*Seamless assuming a sane client. Occasionally some operating systems would choke, but we could also perform manual calls to bring the files back into cache.

EDIT: Also what elvis said. Metadata was always available for a file, so could be browsed without triggering a restore of entire folders.

Oracle HSM/Sun SAM-QFS for those interested.

Last edited by obi; 16th December 2016 at 11:35 AM.
obi is offline   Reply With Quote
Old 16th December 2016, 12:50 PM   #71
Luke212
Member
 
Join Date: Feb 2003
Location: Sydney
Posts: 8,771
Default

Quote:
Originally Posted by NSanity View Post
word on reddit is the number 1 reason 3PAR's fail is during firmware updates apparently.
its also the no. 1 reason cloud services fail. i have seen a provider lose all servers and all backups from a firmware update.

firmware is a form of inter-dependency. its like putting all your data one 1 drive.
__________________
Democracy's greatest trick was convincing man he was informed.
Luke212 is offline   Reply With Quote
Old 16th December 2016, 12:51 PM   #72
Daemon
Member
 
Daemon's Avatar
 
Join Date: Jun 2001
Location: qld.au
Posts: 4,783
Default

Quote:
Originally Posted by wintermute000 View Post
So what's your flavour? GlusterFS or Nutanix or VSAN?

If Ceph is stuffed why is it the 'standard' Openstack choice?
None of the above, I run something fairly industry specific (Virtuozzo) which will come into the enterprise market soon Production proven with companies running it into the multiple PB range.

Ceph is just a completely different beast compared to what most enterprises deal with. At scale, there's no issue as you have a team working with it 24/7. It's like trying to drive a F1 racecar, if you don't have a full team and the right experience, it's very easy to crash and burn. Get it right though and it's incredibly powerful.

Quote:
Originally Posted by elvis View Post
Micro-instance workloads for web applications are different to enterprise VM workloads, which are closer to traditional physical servers. Different requirements there.
Ironically enough, they're not so different. You can get greater densities with micro-services / small web based systems but they present the same overall (70% read, 30% write) type scenarios as a typical VM environment. It's really the management and network layers where there's differentiation.

Quote:
Originally Posted by Myne_h View Post
Silly question for the storage experts here.

Facebook has a bluray dukebox thing for near-line WORM storage.
Wouldn't it be ideal to have something similar as a mirror for massive arrays of generally only written once data like this?
Keep a separate disc +hdd for the consolidated index and you're done. No?
If you want to copy Facebook for archiving, then this is what you want: https://code.facebook.com/posts/1433...orage-system-/

DC's designed from the ground up, hardware designed from the ground up just for cold storage. Everything down to the controllers only spinning up one drive at a time has been factored in. This at least can be scaled down to a single server scenario and therefore useful to enterprise.

The Bluray stuff you'd want to have the issue of archives for over 1PB before it was even a consideration. Tape however is dead at a large scale and many big companies are using 10TB drives in cold storage configuration for long term archiving.
__________________
Fixing the internet... one cloud at a time.
Daemon is offline   Reply With Quote
Old 16th December 2016, 1:41 PM   #73
obi
Member
 
Join Date: Oct 2004
Posts: 132
Default

Quote:
Originally Posted by Daemon View Post
Tape however is dead at a large scale and many big companies are using 10TB drives in cold storage configuration for long term archiving.
I'm not sure everyone agrees with you.

Quote:
The current total tape archive capacity is ~100 PB
obi is offline   Reply With Quote
Old 16th December 2016, 1:47 PM   #74
elvis
Old school old fool
 
elvis's Avatar
 
Join Date: Jun 2001
Location: Brisbane
Posts: 28,122
Default

Quote:
Originally Posted by Daemon View Post
Ironically enough, they're not so different. You can get greater densities with micro-services / small web based systems but they present the same overall (70% read, 30% write) type scenarios as a typical VM environment. It's really the management and network layers where there's differentiation.
Specific to our use case, we were on clustered storage for a long time, but were ultimately forced off it because our workloads crippled it.

We still use it around the place for secondary workloads, but our primary stuff had to move to more "traditional" storage to get the performance we needed.

Quote:
Originally Posted by Daemon View Post
Tape however is dead at a large scale and many big companies are using 10TB drives in cold storage configuration for long term archiving.
"Long term" is a relative statement, and will mean different things to different people. I've worked at places where "long term" meant 12 months (small business, #yolo), and others where it meant 30+ years (large scale architecture, where you can be sued for defects in buildings many decades after the plans were finalised).
__________________
Play old games with me!

Last edited by elvis; 16th December 2016 at 1:49 PM.
elvis is offline   Reply With Quote
Old 16th December 2016, 4:50 PM   #75
Daemon
Member
 
Daemon's Avatar
 
Join Date: Jun 2001
Location: qld.au
Posts: 4,783
Default

Quote:
Originally Posted by obi View Post
I'm not sure everyone agrees with you.
If there was 100% consensus or only the one answer, then it wouldn't be here to be debated Sure it's not dead for everyone but anyone rolling new systems then you won't see tape being mentioned often.
Quote:
Originally Posted by elvis View Post
Specific to our use case, we were on clustered storage for a long time, but were ultimately forced off it because our workloads crippled it.

We still use it around the place for secondary workloads, but our primary stuff had to move to more "traditional" storage to get the performance we needed.
Not all clustered systems have tackled the performance side well, especially those who are focused on data integrity or use object storage as the base product. There are already systems which can achieve 10x the typical Ceph performance in the same tin, which shows how far Ceph and the likes have to go when it comes to increases.

Quote:
Originally Posted by elvis View Post
"Long term" is a relative statement, and will mean different things to different people. I've worked at places where "long term" meant 12 months (small business, #yolo), and others where it meant 30+ years (large scale architecture, where you can be sued for defects in buildings many decades after the plans were finalised).
The drives for this are very specific to cold storage / long term archiving, which is a big change from previous technologies. This is exactly why all of the big providers are using them, and though the magic of automation they can continuously upgrade and replace drives to further prolong lifespans well beyond 30 years.
__________________
Fixing the internet... one cloud at a time.
Daemon is offline   Reply With Quote
Reply

Bookmarks

Sign up for a free OCAU account and this ad will go away!

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time now is 11:26 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
OCAU is not responsible for the content of individual messages posted by others.
Other content copyright Overclockers Australia.
OCAU is hosted by Micron21!