Linux Raid6 / XFS performance options

Dragon_191 · 20th August 2009, 3:48 PM

I'm going to be rebuilding my fileserver in the next week or so and am planning on using Linux software RAID6 and the XFS filesystem for the main data store. I will be running SLES11.

What performance options should I set when creating the RAID6 array and filesystem? Files will be mainly movies and tv shows ranging from 1 - 20GB in size and around 1TB of ESX virtual machine files. I will start off with 9x 1.5TB drives in RAID6 and this will gradually get expanded to 15-20x 1.5TB disks.

Hardware will be the following:

Tyan S2895 Dual Opteron Motherboard
2x 275 Dual-Core Opteron CPUs
6GB RAM
2x Supermicro 8port LSI1068E SAS Controllers
2x WD 80GB SATA disks - RAID1 array for OS.
9x Samsung 1.5tb SATA disks - RAID6 array for DATA
Quad port Intel gigabit NIC
Norco 4020 4U 20bay hotswap case.

Drunkmunky · 20th August 2009, 10:33 PM

Not sure how helpful this is to you but I'm just about to rebuild my RAID5 fileserver with EXT3 instead of XFS as I keep losing files when the server loses power.... Apparantly XFS doesn't take kindly to power loss, wish I had have researched a bit better in the first place

Dragon_191 · 21st August 2009, 8:44 AM

I've been using XFS on my current fileserver with hardware RAID5 for the last couple of years without any problems, but the server is protected by UPS.

I've found XFS significantly faster than EXT3 when dealing with large files.

Mikey_D · 21st August 2009, 10:01 PM

Quote:

Originally Posted by Drunkmunky

Apparantly XFS doesn't take kindly to power loss..

Care to elaborate? Interested to know what happened, and if there's any workarounds.

Crinos · 21st August 2009, 10:31 PM

Quote:

Originally Posted by Mikey_D

Care to elaborate? Interested to know what happened, and if there's any workarounds.

Delayed allocation.... ext4 is the same. If you just yank the power, there's some cached data that hasn't been flushed to disk yet, and you risk losing it and/or corrupting open files.

I've just spent the week installing 70TB worth of XFS filesystems... it's definitely a fast, modern, enterprise-grade filesystem but you do have to be conscious of the fact that pulling the power out from under it can mess things up sometimes...

But realistically, if you have such shitty power reliability that it cuts out constantly, you should probably have a UPS anyway...

And if you just like making a habit of dropping the system on it's arse by deliberately pulling the power, you deserve to lose files

Drunkmunky · 21st August 2009, 11:50 PM

Well I got into a bit of a bind, the network card that was in there had shonky Linux drivers and sometimes it would just drop off the network and would require a hard reboot. No screen or keyboard attached so I had to do it the bad bad way

I dropped an Intel Pro 100 in there (I have a 4 port D-Link server NIC sitting on my desk but the drivers are shonky for that as well

) and everything is ok now but there are missing files, all the folders and shit are there but the files are just gone. I tried using the xfs_repair utility and whatever the other one is called and it kept spitting errors even when it was unmounted. I couldn't find any documentation on the tools themselves other than the man page so I just gave up, I don't think I lost anything important anyway just a few tv episodes.

So yes it is my fault but that's my story anyway

20th August 2009, 3:48 PM	#1
Dragon_191 Member Join Date: Aug 2001 Location: ACT Posts: 257	Linux Raid6 / XFS performance options I'm going to be rebuilding my fileserver in the next week or so and am planning on using Linux software RAID6 and the XFS filesystem for the main data store. I will be running SLES11. What performance options should I set when creating the RAID6 array and filesystem? Files will be mainly movies and tv shows ranging from 1 - 20GB in size and around 1TB of ESX virtual machine files. I will start off with 9x 1.5TB drives in RAID6 and this will gradually get expanded to 15-20x 1.5TB disks. Hardware will be the following: Tyan S2895 Dual Opteron Motherboard 2x 275 Dual-Core Opteron CPUs 6GB RAM 2x Supermicro 8port LSI1068E SAS Controllers 2x WD 80GB SATA disks - RAID1 array for OS. 9x Samsung 1.5tb SATA disks - RAID6 array for DATA Quad port Intel gigabit NIC Norco 4020 4U 20bay hotswap case.

20th August 2009, 10:33 PM	#2
Drunkmunky Member Join Date: Jun 2006 Location: Hobart Posts: 1,843	Not sure how helpful this is to you but I'm just about to rebuild my RAID5 fileserver with EXT3 instead of XFS as I keep losing files when the server loses power.... Apparantly XFS doesn't take kindly to power loss, wish I had have researched a bit better in the first place

21st August 2009, 8:44 AM	#3
Dragon_191 Member Join Date: Aug 2001 Location: ACT Posts: 257	I've been using XFS on my current fileserver with hardware RAID5 for the last couple of years without any problems, but the server is protected by UPS. I've found XFS significantly faster than EXT3 when dealing with large files.

21st August 2009, 11:50 PM	#6
Drunkmunky Member Join Date: Jun 2006 Location: Hobart Posts: 1,843	Well I got into a bit of a bind, the network card that was in there had shonky Linux drivers and sometimes it would just drop off the network and would require a hard reboot. No screen or keyboard attached so I had to do it the bad bad way I dropped an Intel Pro 100 in there (I have a 4 port D-Link server NIC sitting on my desk but the drivers are shonky for that as well ) and everything is ok now but there are missing files, all the folders and shit are there but the files are just gone. I tried using the xfs_repair utility and whatever the other one is called and it kept spitting errors even when it was unmounted. I couldn't find any documentation on the tools themselves other than the man page so I just gave up, I don't think I lost anything important anyway just a few tv episodes. So yes it is my fault but that's my story anyway