The enormous Virtualisation help thread

Discussion in 'Other Operating Systems' started by elvis, May 11, 2011.

  1. mesaoz

    mesaoz Member

    Joined:
    Jan 15, 2015
    Messages:
    4,626
    Location:
    Twitch / Brisbane
    Can you not leverage integrated graphics for the host?
     
  2. Ck21

    Ck21 Member

    Joined:
    Jan 2, 2011
    Messages:
    577
    Location:
    SA
    you can if your cpu has integrated graphics, but its not guaranteed from what ive read (its dependent on cpu arch, the more recent the Cpu, the more likely it will work)
     
  3. stevenx

    stevenx Member

    Joined:
    May 2, 2002
    Messages:
    2,267
    Location:
    Fai oh see fo.
    Has anyone played around with Bhyve?

    I was looking at deploying FreeNAS inside a KVM VM with PCI passthrough of my SAS controller, but it looks like I can run a bunch of Linux VM's on FreeNAS running Bhyve. This eliminates any possible problems associated with virtualizing FreeNAS, but I'm worried about the features and maturity of Bhyve. The VM's will be in production, but we're a small outfit and our type of production is different to large enterprise in that it doesn't really matter if things go down for an hour or five.

    I've been using Ovirt for a long time and I like it, but I'm not using it for anything close to what it's capable of in terms of running a cloud type setup with multiple hosts (I run it on a single host).

    Bhyve on FreeNAS will allow me to consolidate some hardware and it looks like it'll run FreeBSD, Scientific Linux and Ubuntu quite happily - and that's all I need it to do. I guess I'll give it a go for non-critical tasks and report back. Stay tuned :)
     
    Last edited: Aug 28, 2017
  4. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    35,924
    Location:
    Brisbane
    I haven't, but I've been casually following its progress. It's been around a while now (introduced in FreeBSD 10.0, with 10.3 and 11.1 currently the stable releases).

    By all accounts it covers off all the basic things you'd want from a virtualisation system, and supports libvirt so it can be managed by all the standard open source tools.

    BSD already has its excellent jail system, which should solve any need for BSD-on-BSD. But if you're wanting to virtualise other stuff on BSD and don't have crazy requirements, it seems bhyve is perfectly capable.
     
  5. hollstar

    hollstar Member

    Joined:
    Dec 14, 2002
    Messages:
    438
    Location:
    Brisbane
    Can anyone report on their real world experience with GPU pass through under KVM, specifically stuff that is OpenGL capable with Windows guests?
     
  6. kronikabis

    kronikabis Member

    Joined:
    Jul 27, 2006
    Messages:
    392
    Location:
    Illawarra
    GPU pass-through presents the GPU to the guest OS as if it were plugged into a physical computer (there is a bit happening in between but yeah)
    All 3D OpenGL etc all works as it would.

    You will need a dedicated graphics card per guest/s running at the same time to do it.
     
  7. Statitica

    Statitica Member

    Joined:
    Dec 29, 2009
    Messages:
    3,184
    Location:
    Lae, Morobe, PNG
    My real world experience is: "Your host does not support GPU passthrough".
     
  8. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    17,551
    Location:
    Canberra
    You probably need VT-dx or whatever.
     
  9. stevenx

    stevenx Member

    Joined:
    May 2, 2002
    Messages:
    2,267
    Location:
    Fai oh see fo.
    Thanks for your input - much appreciated, as always. To be perfectly honest, I could get away with running jails for 90% of the tasks I want to perform, but it would be nice to have the capability to spin up Linux VM's for whatever reason. I caught myself spinning up a FreeBSD VM on Bhyve for some tests the other day, which was a bit of a "WTF am I doing this for?" moment :lol:

    I reckon I need to spend some more time with FreeBSD again before I can make a more informed decision. It was my preference over Linux for many years but I decided to move to CentOS and SL for a few different reasons which aren't so important these days, and aside from pfSense and FreeNAS I'm a bit out of the loop. I didn't even know about Bhyve until recently - I think the last FreeBSD installation I used was 6.4!
     
  10. Statitica

    Statitica Member

    Joined:
    Dec 29, 2009
    Messages:
    3,184
    Location:
    Lae, Morobe, PNG
    Nah, I need a host which doesn't have a discrete gpu.
     
  11. Myne_h

    Myne_h Member

    Joined:
    Feb 27, 2002
    Messages:
    8,409
    Eh? Descrete gpus are exactly what you need.

    You might have to run the host headless if you only have one though. Makes it harder.

    If you have an igpu and descrete, that's the easiest. Run host on igpu, pass the hardware through. I got it working as a test with ovirt without even reinstalling the windows guest (ran directly from the hdd) .

    Booted, ran. Then I broke the host config and couldn't be bothered any more.

    If you do what I did, be prepared to fixmbr. That was annoying with only a shitty netbook to dl the iso.
     
    Last edited: Sep 1, 2017
  12. Statitica

    Statitica Member

    Joined:
    Dec 29, 2009
    Messages:
    3,184
    Location:
    Lae, Morobe, PNG
    From what I've read, nVidia optimus chips, like my 920M, don't work well with passthrough, not least of all because of drivers.

    There's also the minor detail that the machine I would do it on, is one on which I don't want to exclude the "power" of the nVidia chip, as it gets used regularly.
     
  13. Myne_h

    Myne_h Member

    Joined:
    Feb 27, 2002
    Messages:
    8,409
    Lappy?

    Oh dear. Yeah, it'll be easier on a PC.
     
  14. Statitica

    Statitica Member

    Joined:
    Dec 29, 2009
    Messages:
    3,184
    Location:
    Lae, Morobe, PNG
    Yeah.

    Eventually, if I ever end up moving back to Australia, or if I decide to get more stuff shipped over to PNG, I will likely get this happening on a dedicated rig. Unless, of course, they make a major breakthrough in passing through the GPU without having to blacklist it first.
     
  15. stevenx

    stevenx Member

    Joined:
    May 2, 2002
    Messages:
    2,267
    Location:
    Fai oh see fo.
    So, whatever issue I had with the performance of VM's under Bhyve hasn't reared its head since I first noticed it. I don't know if the problem was disk IO, network IO, or what. Maybe I just hadn't allocated enough memory to the VM for the tasks it had to do when first set up. I haven't investigated too closely because the problem hasn't repeated.

    Anyway, it looks like FreeNAS 11.1, which should be released late this month or early next month, will include the ability to attach a certain network interface to a VM. In my case, I want some VM's to have a tap interface for internal LAN use, and others to have a tap interface based on another interface so the VM's can have public IP's. It's doable now, bit awkward to set up. Not that I'm scared of the command line, but FreeNAS configuration works best from the GUI...
     
  16. sic_vl2000

    sic_vl2000 Member

    Joined:
    Dec 13, 2004
    Messages:
    858
    Hey, I'm trying to update ESXi 6.0.0 to 6.0 Upgrade 2 from CLI but when I attempt it I get the following error:

    Failed to remediate the host: (None, "Failed to save Bootcfg to file /altbootbank/boot.cfg: [Errno 30] Read-only file system: '/altbootbank/boot.cfg.tmp'")

    I have ESXi currently running from a USB, is this why its complaining about the read-only file system? I'm fairly new to upgrading ESXi so any assistance would be greatly appreciated.
     
  17. sic_vl2000

    sic_vl2000 Member

    Joined:
    Dec 13, 2004
    Messages:
    858
    Nevermind I sorted it, ended up grabbing the HP iso and upgraded.
     
  18. stevenx

    stevenx Member

    Joined:
    May 2, 2002
    Messages:
    2,267
    Location:
    Fai oh see fo.
    I've noticed some interesting behaviour with Bhyve. I'm running Scientific Linux 7 on FreeNAS/Bhyve and also on CentOS/KVM. The two virtualization hosts are connected to the same switch on our network.

    The SL7 instances on KVM fetch DHCP leases just fine and behave as normal. On Bhyve, the SL7 instances occasionally lose their DHCP lease, leaving the VM uncontactable and with high system loads. Logging in via VNC and ifup/ifdown fixes the problem, as does configuring static addressing.

    I'm not sure if it's a problem with my pfSense DHCP server, Bhyve and the Virtio network device I've assigned to the VM, or a configuration issue with SL7. But it's interesting, and something I haven't encountered before.
     
  19. SiliconAngel

    SiliconAngel Member

    Joined:
    Jun 27, 2001
    Messages:
    619
    Location:
    Perth, Western Australia
    Hmm, I think I'm in the correct thread... I'm setting up pfSense on my Hyper-V host, and I want to use SR-IOV. Yes, I know pfSense works fine with abstracted virtual switches, but this is 2017 and SR-IOV has been a standard for a pretty long time now - it is more efficient, and because pfSense is getting two dedicated hardware ports for it anyway, I figure giving it the lowest level access possible is for the best.

    The problem is, I can't find anything, anywhere, that comes even remotely close to discussing this, let alone implementing it - neither pfSense nor even FreeBSD. Apparently FreeBSD had SR-IOV baked in years ago, but I can't find more than off-hand references - again, nothing I can actually work with.

    So does anyone have any suggestions for how to go about this? Or am I just spitting in the wind here?
     
  20. xlot

    xlot Member

    Joined:
    Dec 26, 2005
    Messages:
    431
    Location:
    Northern Beaches, NSW
    I've been running virtualised pfSense and FreeNAS for a few years with pci passthrough of dual network (pfSense) and LSI HBA (freenas), but not on Hyper-V; I use Xen on centos. My pfSense box uses a dual NIC so I can physically separate external from internal networks through the host.

    In Xen, IOV is largely useless as you can't live migrate vm's on it, and the xen netback driver for virtual bridge interface between VM's allows >10gbps throughput. I get native performance from my ZFS VM to other VM's with volumes mounted as iSCSI (windows) or nfs (linux), although it does burn up a few CPU cores on the hypervisor.

    Having said which I don't know much about Hyper-V. There is Xen pci backend support for vt-d devices baked into freeBSD 10 (i.e. current versions of pfSense / FreeNAS) so it all just works, which probably isn't the case for Hyper-V?
     

Share This Page