Overclockers Australia Forums

OCAU News - Wiki - QuickLinks - Pix - Sponsors  

Go Back   Overclockers Australia Forums > Specific Hardware Topics > Business & Enterprise Computing

Notices


Sign up for a free OCAU account and this ad will go away!
Search our forums with Google:
Reply
 
Thread Tools
Old 19th November 2012, 3:23 PM   #1
ewok85 Thread Starter
Member
 
ewok85's Avatar
 
Join Date: Jul 2002
Location: Tokyo, Japan
Posts: 7,953
Default Let's talk about XenServer

I've been rather sad at the lack of XenServer stuff on here, so I thought I'd create my own thread.

I've been stuck at work with not being limited in my choices for budgeting - I'd love a better version of VMware but instead to get the features I want (mainly live VM migration) I've turned to XenServer.

To get things started I built a storage box just using FreeNAS - quick and simple to do, just running off a USB stick. Got about a dozen 1TB drives mirrored in one big zpool, with two large zvols. Box has 6 gigabit connections to a switch dedicated to iSCSI usage.

Next step was to install XenServer - installing from USB is simple and quick enough. Then installed XenCenter on a Windows machine. Interface is nice and simple, seems easy enough to use without having to read endless documentation.

First hiccup was that I cannot use the nifty VMware->XenServer conversion tool, as I have a "free" edition of ESXi. Bummer Alternative is to either use their P2V tool, copy the VMDK files, or "Export" to OVF.

Spent my Saturday creating a few Windows VM's, copying files around and testing general performance - in general very good, no lag while using remote desktop and moving multiple files at 90mb/s.

Sunday was VM migration day - moved some Linux VM's to XenServer (voip, web test, web production, nagios/rt/sugarcrm) - took a little while but everything went smoothly. Once the old ESXi host was empty I shut it down and installed XenServer so I would have 2 hosts.

Then came my next problem - the old ESXi box only had dual onboard NICs - crack it open and its PCIX... off to ebay for some cards. Once they arrive (probably Wednesday) I can migrate the VMs, shut down the box, and install the card. I'll probably put one of my spare Intel dual-gig cards in the other box as well.

The real fun begins when I migrate my two best ESXi hosts which run mainly Windows guests - the domain controllers will be re-created from scratch, mainly because they are still 2003, but I'm not sure how to handle our SQL server... I'm hoping the P2V works!


Anyone else out there using XenServer? What do you like, what do you dislike? Any gotchas or issues I need to watch out for?
__________________
半ばは自己の幸せを、半ばは他人の幸せを
http://www.leonjp.com - Rants and info about living in Japan
http://forums.expatjapan.net - The Expat Japan Network!
ewok85 is offline   Reply With Quote

Join OCAU to remove this ad!
Old 19th November 2012, 3:29 PM   #2
NSanity
Member
 
NSanity's Avatar
 
Join Date: Mar 2002
Location: Bathurst, NSW
Posts: 9,633
Default

Why can't you use Hyper-V out of curiosity? Linux Guest support problems?
__________________
i'm in your noun, verbing your related noun.
NSanity is offline   Reply With Quote
Old 19th November 2012, 3:44 PM   #3
ewok85 Thread Starter
Member
 
ewok85's Avatar
 
Join Date: Jul 2002
Location: Tokyo, Japan
Posts: 7,953
Default

Quote:
Originally Posted by NSanity View Post
Why can't you use Hyper-V out of curiosity? Linux Guest support problems?
No reason why, it just wasn't first on my list of things to try. I'm keen to have a look at that as well.
__________________
半ばは自己の幸せを、半ばは他人の幸せを
http://www.leonjp.com - Rants and info about living in Japan
http://forums.expatjapan.net - The Expat Japan Network!
ewok85 is offline   Reply With Quote
Old 19th November 2012, 3:49 PM   #4
bsbozzy
Member
 
bsbozzy's Avatar
 
Join Date: Nov 2003
Location: Sydney
Posts: 3,718
Default

I manage a few XenServer boxes.

Pros:
-Central Management - Free, no Vcenter or SCVMM needed.
-Free live migration (you pay for auto migration)
-It is simple to use
-CLI access with most standard linux tools - good for troubleshooting
-General Performance is as good if not better than the others

Cons:
-Lack of multipath for Dell Equallogic SAN (i believe the HP lefthand stuff is the same, and a few others), ESXi does it out of the box. I have to rely on teamed nics for failover, and then use in guest ISCSi for performance (which is how I would do it anyway, but I would like multipathing for OS access). What I would like is proper multipathing support for more vendors (or just fix the stupidity and allow 3rd party driver installs, it is centos ffs...)
-Takes a version update to include support for newer OS
-Last few version updates have had issues (patches released fixed them)
-Alot of stuff has to be done via CLI and cannot be done in the gui, e.g. 4 -port nic bonds, LACP bonds are unsupported with version prior to 6.1

I could go on, dont have the time....


My customer who has all these boxes might yet be migrated to Win 2012 with Hyper V - All MS shop, will save them thousands on their SA renewals.
__________________
WTB: Any one have a VT SS Commodore die cast model in heron white? Please PM Me.
bsbozzy is offline   Reply With Quote
Old 19th November 2012, 5:37 PM   #5
weznagwama
Member
 
Join Date: Jun 2012
Posts: 268
Default

Used XenServer once (2 x host connected to openfiler NAS) and had continual issues with nic-bonding.

The free features were nice (especially the "increase xenapp speed" tickbox LOL) but for stability, couldnt go past ESXi. Just to many stability issues with it.

That was 5.6(?), so has probably increased alot since then.

Quote:
Originally Posted by bsbozzy View Post
I manage a few XenServer boxes.
...migrated to Win 2012 with Hyper V - All MS shop, will save them thousands on their SA renewals.
Sorry for off-topic, can you elaborate on that?
weznagwama is offline   Reply With Quote
Old 19th November 2012, 5:45 PM   #6
NSanity
Member
 
NSanity's Avatar
 
Join Date: Mar 2002
Location: Bathurst, NSW
Posts: 9,633
Default

Quote:
Originally Posted by weznagwama View Post
Sorry for off-topic, can you elaborate on that?
2012 standard includes 2 guest licences for 2 sockets. 2012 Datacenter includes unlimited guest licenses for 2 sockets.
__________________
i'm in your noun, verbing your related noun.
NSanity is offline   Reply With Quote
Old 19th November 2012, 8:38 PM   #7
Herballizard
Member
 
Herballizard's Avatar
 
Join Date: Oct 2002
Location: QLD
Posts: 1,377
Default

Xenserver here as well, using solaris express + napp-it

NFS and CIFS

Running openxenmanger via macports on osx
Herballizard is online now   Reply With Quote
Old 20th November 2012, 10:30 AM   #8
mjunek
Member
 
mjunek's Avatar
 
Join Date: Apr 2003
Location: Ashfield, Sydney
Posts: 671
Default

We are using ESXi internally for alot of things, however the new licensing model introduced with V5 and removed with V5.1 really threw me off. So sticking with ESXi 4.1 for the moment.

Currently looking and testing XenServer, mostly due to cost.

Looking at purchasing 4 servers & 2x FC SANs, a pair+san for Melb and a pair+san for Sydney. Licensing on VMware for the lot would cost me around 3k/server + 8k for vCenter server- = 20k software, for 29k worth of kit! So that's a big increase.
XenServer, on the otherhand, is just under 1k per box, and includes the feature set I need. Can't argue with the cost.

As I'm the only one in the company who really looks after our VM environment, switching to another product wouldnt be an issue from a staff perspective, as I will need to train them on either one.

The management interface feels very "Linuxy" whereas ESXi feels very "Windowsy" - by that, there are lots of little options in Xen, the interface doesn't feel as clean as VMware. All tasks can be performed without issue however.

I think for my purposes it's neither here nor there. I like the idea of ext3 backend filesystem, would make failure recovery alot easier than VMFS.

That's my ramble of thoughts for the moment.
__________________
---> Skymaster <---
Successful Trade List
Main Rig: i7-3820 | Gigabyte GA-X79-UP4 | 64GB | 2x 7870 OC 2GB | 2x 512GB SSD | 4x 256GB SSD | LSI MegaRAID 9261-8i | 2TB 7200R
mjunek is offline   Reply With Quote
Old 20th November 2012, 12:19 PM   #9
username_taken
Member
 
Join Date: Oct 2004
Location: Austin, TX
Posts: 1,352
Default

Quote:
Originally Posted by mjunek View Post
Looking at purchasing 4 servers & 2x FC SANs, a pair+san for Melb and a pair+san for Sydney. Licensing on VMware for the lot would cost me around 3k/server + 8k for vCenter server- = 20k software, for 29k worth of kit! So that's a big increase.
XenServer, on the otherhand, is just under 1k per box, and includes the feature set I need. Can't argue with the cost.
what's your reason for going FC? In my experience with good 10gbps switching gear there's no reason you couldn't run the whole stack on iSCSI or NFS and skip the hardware/management overhead of fiber channel... and this is coming from a guy who used to be all about the FC.
username_taken is offline   Reply With Quote
Old 20th November 2012, 12:25 PM   #10
mjunek
Member
 
mjunek's Avatar
 
Join Date: Apr 2003
Location: Ashfield, Sydney
Posts: 671
Default

Quote:
Originally Posted by username_taken View Post
what's your reason for going FC? In my experience with good 10gbps switching gear there's no reason you couldn't run the whole stack on iSCSI or NFS and skip the hardware/management overhead of fiber channel... and this is coming from a guy who used to be all about the FC.
Mainly because I'd have to put in the 10Gb infrastructure to support it. So the hardware / management side still comes back to me, just in a different form.
__________________
---> Skymaster <---
Successful Trade List
Main Rig: i7-3820 | Gigabyte GA-X79-UP4 | 64GB | 2x 7870 OC 2GB | 2x 512GB SSD | 4x 256GB SSD | LSI MegaRAID 9261-8i | 2TB 7200R
mjunek is offline   Reply With Quote
Old 20th November 2012, 12:38 PM   #11
pH@tTm@N
Member
 
pH@tTm@N's Avatar
 
Join Date: Jun 2001
Location: BRISBANE
Posts: 1,856
Default

MS 2012 Hyper-V looks like buckets of win. Keen to take a look at it in anger and save heaps compares to vmware.
__________________
"And I know that assholes grow on trees but I'm here to trim the leaves" Mike Patton, Peeping Tom, Dont even trip
pH@tTm@N is offline   Reply With Quote
Old 20th November 2012, 1:43 PM   #12
albeeny
Member
 
Join Date: Feb 2002
Location: Syd
Posts: 103
Default

Has anyone had good results running XenServer over 10Gbps iSCSI? I'd be interested to here your story.

We've been running XenServer in production for a few years now (5.5,5.6 and 6.0). 2 pools with 4 nodes each. SAN devices are lowend Dell MD3000i and MD3200i. 4x 1GB iSCSI multipath'ed. Can't complain too much. Very few issues. Software iSCSI Multipath set-ups can be tedious, but once it's in place it's very stable. Our IO loads aren't massive, mostly Windows Application servers, DCs, Exch2003, a couple of SQL2008 nodes...
I've found the Citrix support to be very very good. The support chaps I've spoken to are from the Sydney office and they'll do everything in their power to get best results for your solution.

I like:
- The price
- The Support
- The new StorageMotion and LACP bonding for VM traffic in the latest version 6.1
- The XenConsole is easy as, once you let go of the vSphere Client way of thinking.

I don't like:
- Hardware compatibility list (last time I looked there was bugger all SAN options on the XenServer Compatible hardware list)
- version 6.0 did terrible things to software iSCSI performance.
- lack of integration features with SANS, vmware gets all the attention here
- When you call a Server or San manufacturer (*cough* de11 *cough* 3mc) looking for support information about XenServer, they have know idea what you're talking about.
albeeny is offline   Reply With Quote
Old 20th November 2012, 2:02 PM   #13
Daemon
Member
 
Daemon's Avatar
 
Join Date: Jun 2001
Location: bne.qld.au
Posts: 3,657
Default

Quote:
Originally Posted by bsbozzy View Post
-Lack of multipath for Dell Equallogic SAN (i believe the HP lefthand stuff is the same, and a few others), ESXi does it out of the box. I have to rely on teamed nics for failover, and then use in guest ISCSi for performance (which is how I would do it anyway, but I would like multipathing for OS access). What I would like is proper multipathing support for more vendors (or just fix the stupidity and allow 3rd party driver installs, it is centos ffs...)
Agree, the downsites to XenServer is the support for the Equallogic SAN (which is a great product) and the driver support (quick fix = don't use Broadcom).
Quote:
Originally Posted by mjunek View Post
Mainly because I'd have to put in the 10Gb infrastructure to support it. So the hardware / management side still comes back to me, just in a different form.
Do you have existing FC infrastructure? Also, don't discount plain old 1Gb just because it doesn't sound as good. As anyone in the storage world knows, latency is the critical factor. I manage an Equallogic SAN with some dedicated and Xen nodes connected via a dedicated 1Gb network. The only time we max the connection out is during backups and reboots yet the system is still very quick as all the access requests are sub 5ms (99.9% of the time). That said, the cost of 10Gb gear has dropped dramatically in the last 2 years and will continue to do so.

Hyper-V 2012 looks interesting but it's still quite costly if you're not 100% MS shop. SPLA licensing for a dual processor box will still cost roughly 3k per year to enable unlimited virtuals. Add in System Center to provide the centralised management and it's quite easy to see why Hyper-V is "free".
__________________
Fixing the internet... one cloud at a time.
Daemon is offline   Reply With Quote
Old 20th November 2012, 2:32 PM   #14
bsbozzy
Member
 
bsbozzy's Avatar
 
Join Date: Nov 2003
Location: Sydney
Posts: 3,718
Default

Quote:
Originally Posted by albeeny View Post
Has anyone had good results running XenServer over 10Gbps iSCSI? I'd be interested to here your story.

We've been running XenServer in production for a few years now (5.5,5.6 and 6.0). 2 pools with 4 nodes each. SAN devices are lowend Dell MD3000i and MD3200i. 4x 1GB iSCSI multipath'ed. Can't complain too much. Very few issues. Software iSCSI Multipath set-ups can be tedious, but once it's in place it's very stable. Our IO loads aren't massive, mostly Windows Application servers, DCs, Exch2003, a couple of SQL2008 nodes...
I've found the Citrix support to be very very good. The support chaps I've spoken to are from the Sydney office and they'll do everything in their power to get best results for your solution.

I like:
- The price
- The Support
- The new StorageMotion and LACP bonding for VM traffic in the latest version 6.1
- The XenConsole is easy as, once you let go of the vSphere Client way of thinking.

I don't like:
- Hardware compatibility list (last time I looked there was bugger all SAN options on the XenServer Compatible hardware list)
- version 6.0 did terrible things to software iSCSI performance.
- lack of integration features with SANS, vmware gets all the attention here
- When you call a Server or San manufacturer (*cough* de11 *cough* 3mc) looking for support information about XenServer, they have know idea what you're talking about.
I installed a 10Gb Equallogic SAN this year, SSD/HDD hybrid that a Xen pool connects to. What would you like to know?
__________________
WTB: Any one have a VT SS Commodore die cast model in heron white? Please PM Me.
bsbozzy is offline   Reply With Quote
Old 20th November 2012, 2:53 PM   #15
albeeny
Member
 
Join Date: Feb 2002
Location: Syd
Posts: 103
Default

Quote:
Originally Posted by bsbozzy View Post
I installed a 10Gb Equallogic SAN this year... What would you like to know?
I've read lots of forum threads where people were disappointed with the speed they were getting from 10g iscsi on Xenserver.
I'd like to know
- what switches you use
- what NICs you use
- What version of Xen
- Did you use Software iscsi for VM disk storage?
- Multipath on?
- Jumbo Frames on?
- Switch port Flow control on?
- What you used to benchmark your results
- ...Benchmarck results
- Did you align your NTFS sectors and stripes???
- did you notice a distinct lack of MB/s when writing large sequential files?

*gasp*

Last edited by albeeny; 20th November 2012 at 2:56 PM.
albeeny is offline   Reply With Quote
Reply

Bookmarks

Sign up for a free OCAU account and this ad will go away!

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time now is 10:08 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd. -
OCAU is not responsible for the content of individual messages posted by others.
Other content copyright Overclockers Australia.
OCAU is hosted by Internode!