Let's talk about XenServer

Discussion in 'Business & Enterprise Computing' started by ewok85, Nov 19, 2012.

  1. ewok85

    ewok85 Member

    Joined:
    Jul 4, 2002
    Messages:
    8,074
    Location:
    Tokyo, Japan
    I've been rather sad at the lack of XenServer stuff on here, so I thought I'd create my own thread.

    I've been stuck at work with not being limited in my choices for budgeting - I'd love a better version of VMware but instead to get the features I want (mainly live VM migration) I've turned to XenServer.

    To get things started I built a storage box just using FreeNAS - quick and simple to do, just running off a USB stick. Got about a dozen 1TB drives mirrored in one big zpool, with two large zvols. Box has 6 gigabit connections to a switch dedicated to iSCSI usage.

    Next step was to install XenServer - installing from USB is simple and quick enough. Then installed XenCenter on a Windows machine. Interface is nice and simple, seems easy enough to use without having to read endless documentation.

    First hiccup was that I cannot use the nifty VMware->XenServer conversion tool, as I have a "free" edition of ESXi. Bummer :( Alternative is to either use their P2V tool, copy the VMDK files, or "Export" to OVF.

    Spent my Saturday creating a few Windows VM's, copying files around and testing general performance - in general very good, no lag while using remote desktop and moving multiple files at 90mb/s.

    Sunday was VM migration day - moved some Linux VM's to XenServer (voip, web test, web production, nagios/rt/sugarcrm) - took a little while but everything went smoothly. Once the old ESXi host was empty I shut it down and installed XenServer so I would have 2 hosts.

    Then came my next problem - the old ESXi box only had dual onboard NICs - crack it open and its PCIX... off to ebay for some cards. Once they arrive (probably Wednesday) I can migrate the VMs, shut down the box, and install the card. I'll probably put one of my spare Intel dual-gig cards in the other box as well.

    The real fun begins when I migrate my two best ESXi hosts which run mainly Windows guests - the domain controllers will be re-created from scratch, mainly because they are still 2003, but I'm not sure how to handle our SQL server... I'm hoping the P2V works!


    Anyone else out there using XenServer? What do you like, what do you dislike? Any gotchas or issues I need to watch out for?
     
  2. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    16,190
    Location:
    Canberra
    Why can't you use Hyper-V out of curiosity? Linux Guest support problems?
     
  3. ewok85

    ewok85 Member

    Joined:
    Jul 4, 2002
    Messages:
    8,074
    Location:
    Tokyo, Japan
    No reason why, it just wasn't first on my list of things to try. I'm keen to have a look at that as well.
     
  4. bsbozzy

    bsbozzy Member

    Joined:
    Nov 11, 2003
    Messages:
    3,913
    Location:
    Sydney
    I manage a few XenServer boxes.

    Pros:
    -Central Management - Free, no Vcenter or SCVMM needed.
    -Free live migration (you pay for auto migration)
    -It is simple to use
    -CLI access with most standard linux tools - good for troubleshooting
    -General Performance is as good if not better than the others

    Cons:
    -Lack of multipath for Dell Equallogic SAN (i believe the HP lefthand stuff is the same, and a few others), ESXi does it out of the box. I have to rely on teamed nics for failover, and then use in guest ISCSi for performance (which is how I would do it anyway, but I would like multipathing for OS access). What I would like is proper multipathing support for more vendors (or just fix the stupidity and allow 3rd party driver installs, it is centos ffs...)
    -Takes a version update to include support for newer OS
    -Last few version updates have had issues (patches released fixed them)
    -Alot of stuff has to be done via CLI and cannot be done in the gui, e.g. 4 -port nic bonds, LACP bonds are unsupported with version prior to 6.1

    I could go on, dont have the time....


    My customer who has all these boxes might yet be migrated to Win 2012 with Hyper V - All MS shop, will save them thousands on their SA renewals.
     
  5. weznagwama

    weznagwama Member

    Joined:
    Jun 25, 2012
    Messages:
    263
    Used XenServer once (2 x host connected to openfiler NAS) and had continual issues with nic-bonding.

    The free features were nice (especially the "increase xenapp speed" tickbox LOL) but for stability, couldnt go past ESXi. Just to many stability issues with it.

    That was 5.6(?), so has probably increased alot since then.

    Sorry for off-topic, can you elaborate on that?
     
  6. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    16,190
    Location:
    Canberra
    2012 standard includes 2 guest licences for 2 sockets. 2012 Datacenter includes unlimited guest licenses for 2 sockets.
     
  7. Herballizard

    Herballizard Member

    Joined:
    Oct 9, 2002
    Messages:
    1,533
    Location:
    .
    Xenserver here as well, using solaris express + napp-it

    NFS and CIFS

    Running openxenmanger via macports on osx
     
  8. mjunek

    mjunek Member

    Joined:
    Apr 1, 2003
    Messages:
    795
    Location:
    Ashfield, Sydney
    We are using ESXi internally for alot of things, however the new licensing model introduced with V5 and removed with V5.1 really threw me off. So sticking with ESXi 4.1 for the moment.

    Currently looking and testing XenServer, mostly due to cost.

    Looking at purchasing 4 servers & 2x FC SANs, a pair+san for Melb and a pair+san for Sydney. Licensing on VMware for the lot would cost me around 3k/server + 8k for vCenter server- = 20k software, for 29k worth of kit! So that's a big increase.
    XenServer, on the otherhand, is just under 1k per box, and includes the feature set I need. Can't argue with the cost.

    As I'm the only one in the company who really looks after our VM environment, switching to another product wouldnt be an issue from a staff perspective, as I will need to train them on either one.

    The management interface feels very "Linuxy" whereas ESXi feels very "Windowsy" - by that, there are lots of little options in Xen, the interface doesn't feel as clean as VMware. All tasks can be performed without issue however.

    I think for my purposes it's neither here nor there. I like the idea of ext3 backend filesystem, would make failure recovery alot easier than VMFS.

    That's my ramble of thoughts for the moment.
     
  9. username_taken

    username_taken Member

    Joined:
    Oct 19, 2004
    Messages:
    1,352
    Location:
    Austin, TX
    what's your reason for going FC? In my experience with good 10gbps switching gear there's no reason you couldn't run the whole stack on iSCSI or NFS and skip the hardware/management overhead of fiber channel... and this is coming from a guy who used to be all about the FC.
     
  10. mjunek

    mjunek Member

    Joined:
    Apr 1, 2003
    Messages:
    795
    Location:
    Ashfield, Sydney
    Mainly because I'd have to put in the 10Gb infrastructure to support it. So the hardware / management side still comes back to me, just in a different form.
     
  11. pH@tTm@N

    pH@tTm@N Member

    Joined:
    Jun 27, 2001
    Messages:
    1,950
    Location:
    BRISBANE
    MS 2012 Hyper-V looks like buckets of win. Keen to take a look at it in anger and save heaps compares to vmware.
     
  12. albeeny

    albeeny Member

    Joined:
    Feb 25, 2002
    Messages:
    119
    Location:
    Syd
    Has anyone had good results running XenServer over 10Gbps iSCSI? I'd be interested to here your story.

    We've been running XenServer in production for a few years now (5.5,5.6 and 6.0). 2 pools with 4 nodes each. SAN devices are lowend Dell MD3000i and MD3200i. 4x 1GB iSCSI multipath'ed. Can't complain too much. Very few issues. Software iSCSI Multipath set-ups can be tedious, but once it's in place it's very stable. Our IO loads aren't massive, mostly Windows Application servers, DCs, Exch2003, a couple of SQL2008 nodes...
    I've found the Citrix support to be very very good. The support chaps I've spoken to are from the Sydney office and they'll do everything in their power to get best results for your solution.

    I like:
    - The price
    - The Support
    - The new StorageMotion and LACP bonding for VM traffic in the latest version 6.1
    - The XenConsole is easy as, once you let go of the vSphere Client way of thinking.

    I don't like:
    - Hardware compatibility list (last time I looked there was bugger all SAN options on the XenServer Compatible hardware list)
    - version 6.0 did terrible things to software iSCSI performance.
    - lack of integration features with SANS, vmware gets all the attention here
    - When you call a Server or San manufacturer (*cough* de11 *cough* 3mc) looking for support information about XenServer, they have know idea what you're talking about.
     
  13. Daemon

    Daemon Member

    Joined:
    Jun 27, 2001
    Messages:
    4,955
    Location:
    qld.au
    Agree, the downsites to XenServer is the support for the Equallogic SAN (which is a great product) and the driver support (quick fix = don't use Broadcom).
    Do you have existing FC infrastructure? Also, don't discount plain old 1Gb just because it doesn't sound as good. As anyone in the storage world knows, latency is the critical factor. I manage an Equallogic SAN with some dedicated and Xen nodes connected via a dedicated 1Gb network. The only time we max the connection out is during backups and reboots yet the system is still very quick as all the access requests are sub 5ms (99.9% of the time). That said, the cost of 10Gb gear has dropped dramatically in the last 2 years and will continue to do so.

    Hyper-V 2012 looks interesting but it's still quite costly if you're not 100% MS shop. SPLA licensing for a dual processor box will still cost roughly 3k per year to enable unlimited virtuals. Add in System Center to provide the centralised management and it's quite easy to see why Hyper-V is "free".
     
  14. bsbozzy

    bsbozzy Member

    Joined:
    Nov 11, 2003
    Messages:
    3,913
    Location:
    Sydney
    I installed a 10Gb Equallogic SAN this year, SSD/HDD hybrid that a Xen pool connects to. What would you like to know?
     
  15. albeeny

    albeeny Member

    Joined:
    Feb 25, 2002
    Messages:
    119
    Location:
    Syd
    I've read lots of forum threads where people were disappointed with the speed they were getting from 10g iscsi on Xenserver.
    I'd like to know
    - what switches you use
    - what NICs you use
    - What version of Xen
    - Did you use Software iscsi for VM disk storage?
    - Multipath on?
    - Jumbo Frames on?
    - Switch port Flow control on?
    - What you used to benchmark your results
    - ...Benchmarck results
    - Did you align your NTFS sectors and stripes???
    - did you notice a distinct lack of MB/s when writing large sequential files?

    *gasp* :)
     
    Last edited: Nov 20, 2012
  16. thedrover

    thedrover Member

    Joined:
    May 29, 2012
    Messages:
    61
    Been using XenServer for just under a year now. No complaints.

    Running 3 Dell R710 hosts redundantly connected to 8TB Dell SAN via SAS.
     
  17. bsbozzy

    bsbozzy Member

    Joined:
    Nov 11, 2003
    Messages:
    3,913
    Location:
    Sydney
    -2 Dell 8024F Stacked
    -Intel x520 DA2
    -5.6 SP2
    -Yes - only for OS, anything that requires fast storage for data, e.g. SQL, Exchange, etc, I added the storage nics to the VMs and connected to the san directly
    -no equalogic XenServer multipathing support, yet xenserver does other shit with equalogic....doesnt really matter as there is only 1 active 10Gb port anyway. The 2 Interfaces on Xen are bonded so it will failover if 1 link dies
    -mtu 9000, flow control on (on by default on the dell switches)
    -I did bench it at the time and found it to only use 200-300MBs for one server, multiple servers at the same time would get that speed each. I think increased the speed after talking with a tech, but I cant remember.
    -No need, aligned itself.
    -Yes as above


    I also have an older eql san using 4 x 1GB nics in the xenservers, however this is just for mass storage for file servers, backups, etc.
     
  18. Herballizard

    Herballizard Member

    Joined:
    Oct 9, 2002
    Messages:
    1,533
    Location:
    .
    Anyone using xenserver and infiniband?
     
  19. Ressurection

    Ressurection Member

    Joined:
    Nov 12, 2004
    Messages:
    117
    Location:
    Canberra
    I had 12 months of hell with vSphere and Infiniband, would not want to even contemplate the seven layers of hell to be experienced with XenServer and Infiniband.

    I have a rather large client who uses XenServer but based on the constant minor failings it has in comparison to Hyper-V or vSphere I would not be using it if it was not cost prohibitive to change vendor.
     
  20. 3t3rna1

    3t3rna1 Member

    Joined:
    Dec 24, 2001
    Messages:
    1,452
    Location:
    Perth
    IIRC you can install the OFED stack on it with some hacking. It could work but chances are you will be unsupported by everyone. If you want to run Infiniband I would stick with RHEL/SL/CentOS as your hypervisor where OFED is officially supported.
     

Share This Page