The enormous Virtualisation help thread

Discussion in 'Other Operating Systems' started by elvis, May 11, 2011.

  1. elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    Virtualisation topics are coming up a lot in other threads (no wonder, seeing as it's dominating a lot of IT in general at the moment). While positive, they invariably dominate the threads and take them off topic.

    So this thread is aimed at keeping it all in one place. Bring all your questions into this thread and we'll nut out your nastiest virtualisation problems.

    I'll invariably try to change everyone over to KVM, but don't let that stop you posting questions about other hypervisors and virtualisation systems. (Extra points for the first person to say they're running bochs in production :)).

    Hijacking from another thread:

    As mentioned, outside of the realm of KVM (or other hypervisors in general), but minimum disk sizes are important to know for virtualisation, as generally you're trying to keep things as small as possible.

    With a quality OS, you should have access to things like LVM (Logical Volume Manager) that allow you to span your OS easily over extra drives should you wish to add them later, or at the very least decent tools like resize2fs that allow you to repartition and resize your install easily.

    Some legacy OSes won't allow this, so it's important to pick the right size. Speaking for myself, 10GB is a minimum disk size I'll give to WinXP. For Windows 7 64bit, Microsoft recommend 20GB. I'd put down 30GB for some room to grow. For Win2K8 R2 64bit Server, Microsoft recommend 32GB minimum. I don't roll them out under 50GB on the C: drive.

    It ultimately all boils down to what you do with the VM when built, of course. With that said, for "appliance" style Linux boxes, I frequently get away with as little as 2GB for some installs. I average 5-10GB for most others. If I have a large app or heaps of static content to server, that always goes on a second virtual disk not attached to the main system directories or disks.

    Also, I'm in the process of building a 2-node "mini cluster" KVM setup for a friend, running GlusterFS to mirror disk images between nodes, and allow live migration and a relatively good level of failure recovery between the nodes. It isn't true HA (although VMWare call this HA, and they call HA "Fault Tolerance", but they're idiots), but it's nice and easy to maintain for a 2-node setup.

    Once done, I'll share some of the install notes here if other people want to experiment.
     
    Last edited by a moderator: Sep 7, 2012
  2. oli

    oli Member

    Joined:
    Jun 29, 2002
    Messages:
    7,266
    Location:
    The Internet
    That was more useful than the RHEL docs.

    Thanks
     
  3. hollstar

    hollstar Member

    Joined:
    Dec 14, 2002
    Messages:
    440
    Location:
    Brisbane
    All very valid points and for the most part, I'd agree.
    Again I'll reiterate, I'm very green when it comes to Linux so I'm feeling my way! :)

    Following on from the thread in question, when I was initially toying with this, I was merely playing with a Windows XP Professional SP1 disc I had spare in the interest of testing. Once I was comfortable with this, I'd set up a number of basic virtual environments so I had something to test on when providing phone support to clients, or having to use Windows specific software.

    In lieu of the fact an install of XPP SP1 is less than 1GB, I thought 2GB would be enough for the sake of testing however it complained of a lack of free space whilst reporting (it seemed anyway) the correct "USED" space? I found it really strange and found no mention of it during my search at that time hence 6GB, seemed to become the magically sweet spot?

    I think I also used the qcow2 format which (correct me if I'm wrong) doesn't assign a fixed file size to the image file depending on what you set up. If I'm right, mine DID assign a size regardless and wasn't flexible at all.

    I guess that's why I say I found it all a little difficult and confusing! I will however be toying with this all again shortly so I'll hang in there...
     
  4. Gecko

    Gecko Member

    Joined:
    Jul 3, 2004
    Messages:
    2,715
    Location:
    Sydney
    Any recommendations for a "nice" interface into KVM? VMware lives on in my environment due to the simple point and click interface :(

    Interested to read your docs when done - I've done it a few times with Xen and DRBD - and am looking at doing it with KVM and Gluster
     
  5. wellonchompy

    wellonchompy Member

    Joined:
    May 29, 2002
    Messages:
    747
    Location:
    Sydney
    1. Install Fedora 14
    2. # yum install virt-manager
    3. ...
    4. Profit
     
  6. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    No, that's not how qcow2 works.

    qcow2 offers a "sparse file" format (not by default, you need to do that when you create it via a flag). When you do chose sparse, that means it doesn't fill the file with data when you create it. So if you do an "ls -lah" on the file, you'll only see how much data is actually used. But sparse files need a maximum size that they can grow to, which they can never grow bigger than.

    Also, Windows XP (and most operating systems, in fact) need a maximum size that they can see. They can't deal with an incrementally and eternally expanding disk.

    So even though the file representing the hard disk is sparse (it starts small, and grows as real data fills it), the virtual operating system above only sees the full size, and sees it as a static device.

    Sparse files for virtualisation can be useful, but more for the host administration side of things. For starters, you can over allocate your disk (you can give guests more storage than you actually have, which you can then grow into over time, although it has obvious risks attached). Secondly, sparse files are nice to back up, as they don't waste the unused space of the virtual file system as stuff you have to back up. One final minor advantage is that they're quick to create. Creating a full file takes time, particularly on slow disk and if it's a big file.

    From the guest side, sparse files don't offer any real extras.

    FWIW, you can also create sparse files with dd using the "seek" parameter. Try the following:

    #To create a 100MB file full of blank space - this will take a while:
    dd if=/dev/zero of=test.img bs=1M count=100
    ls -lah test.img
    du -sh test.img

    #To create a sparse file that can grow to a maximum of 100MB - this will be instant:
    dd if=/dev/zero of=test-sparse.img bs=1M count=0 seek=100
    ls -lah test-sparse.img
    du -sh test-sparse.img

    Notice the difference between outputs in both.

    From the other thread:

     
    Last edited by a moderator: Sep 7, 2012
  7. joyufat

    joyufat Member

    Joined:
    Jun 27, 2001
    Messages:
    1,014
    Location:
    Moral High Ground
    When I last tried Covirt last year I kept getting all sorts of Python errors, and also JS errors in the web UI. It looked neat when it worked, equivalent to the ESXi client interface, although I didn't like the fact that some features like VLAN tagging require payment.

    Also Convirt isn't directly compatible with libvirt (i.e. it's not just a GUI on top of libvirt), and with KVM I get the feeling that libvirt is going to become dominant eventually. This is the same problem that Proxmox has, although whether it really is a problem is up to you.
     
  8. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    VLAN tagging is done at the OS layer (create vlan interface attached to physical interface, create bridge attached to vlan interface, connect guest to bridge).

    Simple stuff - I do this pretty much as standard most places I go, straight from the command line. You do it once per physical host, and then never touch it again.

    Can you expand on that? It appears to talk lib-virt from everything I've seen.

    I'll have to do a full test on the KVM/Gluster rollout I mention above.

    It's dominant now. lib-virt interfaces are the preffered way to manage KVM. Manual KVM/QEMU command lines are discouraged.
     
    Last edited: May 12, 2011
  9. Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,373
    Location:
    CFS
    Great thread! You've won me over i'll be using KVM when I finally get a working server again (hardware issues :\)
     
  10. Gecko

    Gecko Member

    Joined:
    Jul 3, 2004
    Messages:
    2,715
    Location:
    Sydney
    Convirt looks like it is worth a go - I'll have a play around with it

    Virt-manager is OK as well (use it personally) - would prefer something that can be accessed from Windows clients (without X forwarding) though
     
  11. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    I'm kind of surprised that a free Windows client hasn't popped up yet. What's really bizzare is that RedHat's enterprise KVM management tool actually started life as Windows-only (back in the Qumranet days), although now they're pushing their web based solution, which is built on their oVirt tool.

    oVirt itself was looking great. But development seems to have gone underground. It's available as part of RHEV, and despite being GPL and open source, nobody else has bothered to bundle it up as a ready to go binary.

    For all of Ubuntu's cloud love, they too are missing a decent web interface (let alone one with proper enterprise features, like full cluster management and awareness of which VMs are booted where).
     
  12. joyufat

    joyufat Member

    Joined:
    Jun 27, 2001
    Messages:
    1,014
    Location:
    Moral High Ground
    But it means there are two interfaces for managing things, which I hate. If I'm in CLI anyway I might as well do everything by CLI.

    What I mean is that for each VM host you need to install a Convirt agent, instead of Convirt just using the libvirt API like many of the other management tools do. Although admittedly I don't really know what the convirture-tools package contains, so for all I know it may just be installing a version of libvirt :D

    Edit: just to add, I'm really interested in Cloud.com CloudStack. It looks very interesting, just waiting to some spare servers so I can try it out
     
    Last edited: May 12, 2011
  13. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    The VLAN stuff should be a "once only" setup. It should be done at the host install phase, and not for day to day management of guests.

    Once set up, you have a bridge per VLAN that you can attach to. From the client side of things (deploying a new VM), the correct vlan/bridge combination should appear in the dropdown list ready to go. No need to use a CLI if you don't want to.

    Lower level host VLAN management is not the job of lib-virt, virt-manager nor KVM.
     
  14. vmstash

    vmstash Member

    Joined:
    May 12, 2011
    Messages:
    29
    I'd like to mention ProxMox for anyone that would like to play with Linux Virtualisation. It's a great distro that has worked for me quite well. :thumbup:
     
  15. 3t3rna1

    3t3rna1 Member

    Joined:
    Dec 24, 2001
    Messages:
    1,452
    Location:
    Perth
    I ran into this problem the other day. I needed to provision another VLAN because we ran out of pre-defined ones.

    Not knowing that it would cut all the networking to my active VM's I... -
    Added new VLAN's and bridges to /etc/network/interfaces
    ran /etc/init.d/networking restart

    This worked but killed the networking to any running VM's. They now needed to be powered off and on again, massive PITA.

    Is there a better way to 'hot add' a VLAN so I wouldn't have to migrate VM's off a host?

    I was just going to make up a massive list of VLAN's in the interfaces file so they are already predefined but it seems like a bit of a brute force way of getting it done.

    Also, if anyone isn't aware of the Sheepdog project. It's getting close to being a very awesome storage layer for KVM - http://www.osrg.net/sheepdog/
     
  16. Swathe

    Swathe (Banned or Deleted)

    Joined:
    Mar 23, 2007
    Messages:
    2,509
    Location:
    Rockhampton
    It does do a good job, clustering with it is quite simple too.
     
  17. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    Probably obvious to you now, but this tears down the entire OS networking stack (including physical devices and bridges) and builds it back up again. This is not something you do on a production machine!

    man vconfig

    "vconfig" is the program that can add and remove VLANs on the fly, and "brctl" is the command to add and remove bridges on the fly.

    If you want to add a VLAN dynamically, use these tools. If you want the VLAN to stick around after reboot, add the config to your particular config files. Out of interest, all the Linux startup scripts do is parse the network config files, and run "ifconfig" (or "ip"), "vconfig" and "brctl" commands based on what the config files say.

    A problem that has been bugging me for a while: low entropy on virtual hosts causing guest entropy pools to starve, which means any crypto inside the guests (SSH, OpenSSL, https, whatever) slow right down. This is a problem for a lot of hypervisors (I've seen it big time in busy VMWare hosts).

    I've just installed "randomsound". It was in Ubuntu's universe repository. It reads data from your sound card and feeds the kernel entropy pool when it gets below a certain value (obviously you need a sound card for this, so it's not a working option for some big enterprise boxes, but this has saved my workstation).

    1) apt-get install randomsound
    2) edit /etc/default/randomsound and add startup option "-m 2048" (set the minimum pool size to 2048).
    3) Use "alsamixer" to unmute and turn up all recording volumes, amplifiers and boosters to max
    4) Restart the daemon.

    End result: no more entropy starving. Particularly with a lot of Windows guests using crypto, I've noticed things don't slow down nearly as badly any more. Hooray!

    Now I need to go and reboot all of my home and small business servers and turn the audio devices back on in BIOS. :lol:

    [edit] No sound card? No worries. Try this one instead:
    http://www.issihosts.com/haveged/
     
    Last edited by a moderator: Sep 7, 2012
  18. s3kemo

    s3kemo Member

    Joined:
    May 13, 2003
    Messages:
    5,889
    Location:
    in a house
    I've got one for you Elvis!

    How can I change the hardware IDs of KVM to emulate VMWare? The desired result is the virtual machine thinks it's on VMWare but is actually on KVM, so things like the Hardware Vendor, BIOS version, etc can be changed to a string of my choice.

    I can achieve this with Virtualbox (tested on Windows) pretty easily using the VBoxManage commands but no idea how to achieve this in KVM. My google fu is not helping me either!

    EDIT: would also appreciate some insight how I can google KVM without it meaning the other KVM (keyboard video mouse)... it drives me nuts :lol:
     
    Last edited: May 12, 2011
  19. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    37,043
    Location:
    Brisbane
    A quick google leads me to a post on the KVM dev list dated April 28, 2011 which seems to indicate the functionality doesn't yet exist in KVM. :(

    It is possible to do by hacking source, but not through config. It's certainly a feature I think has a valid use case (for the example you're using, and many others). Hopefully it'll get added in down the track.
     
  20. wellonchompy

    wellonchompy Member

    Joined:
    May 29, 2002
    Messages:
    747
    Location:
    Sydney
    You can mess with what qemu says is in the hardware. This particular script takes the ACPI tables that tell Windows 7 it's on hardware with an OEM license, but I'm sure it can be modified to say nearly anything: https://github.com/ghuntley/seaslic

    Admittedly, it does mean you need to patch the BIOS image that you give to a particular guest, but it is possible to modify some fake hardware values.
     

Share This Page

Advertisement: