OpenSolaris/Solaris 11 Express/ESXi : BYO Home NAS for Media and backup images etc

Discussion in 'Storage & Backup' started by davros123, Dec 14, 2009.

  1. chook

    chook Member

    Joined:
    Apr 9, 2002
    Messages:
    3,789
    Yes, these new disks have the WWN on them but the old ones didn't. While doing the identification I made a list as you suggest in there for next time. Unfortunately I only have 6 bays and they are all full which is why I had to go this way instead of adding an extra one and then removing the old one.

    Good to know that I don't have to scrub because the resilver is about eight hours to complete. I would have done it if necessary because better safe than sorry.

    I was only using iostat as an inventory tool. The disk had reported too many errors. This wasn't the first time it had done that and given they are about eight years old I decided it was time to bite the bullet and got some replacements that are larger anyway. Once I do the entire pool I will expand it.

    I can't get Smartmontools to install because I am on Solaris 11.3 and it does the dummy spit about the certificate when I try to install it. Once I have the pool sorted out I am going to do a reinstall of Solaris 11.4 and clean it all up. This was my first Solaris install and I have learned a lot along the way which has been fine but I have been putting this off for a long time and now the time has come.
     
  2. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
  3. grs1961

    grs1961 Member

    Joined:
    Jan 21, 2005
    Messages:
    524
    Location:
    Melbourne
  4. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    OmniOS stable 151036 is out
    https://github.com/omniosorg/omnios-build/blob/r151036/doc/ReleaseNotes.md

    Some improvements:

    SMB 3.1.1 (ZFS/kernelbased SMB server)
    Improvements on Linux/LX zones
    Improvements on Bhyve virtualizer
    ZFS can use hardware acceleration for Raid-Z
    Improved support for Mellanox ConnectX-4/5/6 , i40e and bge nic devices

    gcc10 is now default compiler
     
    Last edited: Nov 3, 2020
  5. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    OpenIndiana 2020.10

    OpenIndiana, one of the main Illumos distributions beside OmniOS, NexentaStor and SmartOS and more or less the successor of OpenSolaris with a server and desktop edition is here with a new snapshot

    new among others: SMB 3.11, support for Bhyve, FreeRDP, Remmina, Mate 1.24 and gcc10
    https://www.openindiana.org/2020/11/01/openindiana-hipster-2020-10-is-here/
     
  6. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
  7. revhed

    revhed Member

    Joined:
    Aug 22, 2001
    Messages:
    1,529
    Location:
    Darwin
    Hi Gea,
    What zpool versions does OpenIndiana support?
    I'm running zpool v34 on a Solaris 11.3 rig, and it looks like I have to do a re-install to get to Solaris 11.4. I am thinking about switching to OpenIndiana, but don't want to.rebuild my arrays. Is.the only way of getting the latest zfs features to still use a Solaris host?
     
    GoofyHSK likes this.
  8. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    The distributions based on the free Solaris fork Illumos (NexentaStor, OpenIndiana, OmniOS, SmartOS etc) are all on pool v5000 with features. As only pool version 28/5 is compatible between Oracle Solaris with native ZFS and Open-ZFS you must backup your data (or move to a pool v28/5), then import the pool v28/5 or create a new pool and restore data to switch to the free Solaris forks.

    While Oracle Solaris with native ZFS was still the fastest ZFS server in my tests, the important ZFS features are now also in Illumos and Open-ZFS like ZFS encryption, sequential/sorted resilvering, removeable vdev, SMB 3.1.1 (ZFS/kernelbased SMB server) and more.

    For private use to must face an additional problem. As Solaris is like Windows 10 now frozen on version 11.4 with rolling updates (ex 11.4.SRU24), you have no chance of a new free version every 2 years. Without a paid subscription you are not able to get these updates what means that an original Solaris 11.4 from two years back has more and more compatibility problems.

    If this is a storage server only, I would prefer OmniOS due the stable and long term stable editions (optionally with paid commercial support). If you want a true successor of OpenSolaris due the desktop and a server edition, OpenIndiana that is based on pure ongoing Illumos is it.
     
    Last edited: Nov 30, 2020
  9. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,006
    Over 10 years later - she's still rocking :)
    Now running Gea's nappit and 8TB drives and 10GB fiber but otherwise same same. Now that is a good ROI.
    upload_2020-11-30_22-56-43.png
     
  10. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    ZFS encryption performance on OmniOS and Intel Xeon silver vs AMD Epyc 7302

    After weeks of waiting I got a new SuperMicro BTO system with the H12SSL-C mainboard and a 16/32 Core 7302 and 128GB RAM, H12SSL-C | Motherboards | Super Micro Computer, Inc.

    I made some tests with a disk pool, an NVMe pool and an Optane pool with and without encryption as this is what requires performance. While sync write seems a performance problem with encryption, the other results are promising. As you can load such a system with 24 NVMe system performance may need a jump.

    I have also tried a virtualized OmniOS on ESXi 7.0U1 but found NVMe pass-through problems that need some more work.
    12G SAS in pass-through mode ex with WD SS530 SAS SSDs are troublefree and nearly as fast as NVMe.

    First impression
    https://napp-it.org/doc/downloads/epyc_performance.pdf
     
  11. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    Last edited: Dec 16, 2020
  12. HobartTas

    HobartTas Member

    Joined:
    Jun 22, 2006
    Messages:
    1,067
    The last common format was v28 (using filesystem v5) so the only way to "migrate" would be to create a second zpool v28 (fs v5), then copy the data from v34 > v28 and export the v28 pool and then import than into Openindiana and from there upgrade the pool to whatever 5xxx version they are currently at. Same advice applies if you want to go from Openindiana back to Solaris. P.S. Sorry just noticed that Gea has already stated this.
     
    Last edited: Dec 16, 2020
  13. Axims

    Axims Member

    Joined:
    Jan 28, 2003
    Messages:
    13
    G'day Gea, thanks for the analysis on ZFS encryption. I've been waiting for this feature to become stable for a long time. I am very close to setting up a disk based async pool with encryption. Looking at your document TLDR, I'm seeing a minor performance decrease (as expected) but otherwise quite impressive numbers.
     
  14. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    Important OmniOS Security Update r36m, r34am, r30cm - Security / Features/ Bugfixes

    Support for secure RPC for the Netlogon protocol between OmniOS systems and Microsoft Active Directory servers is added to all OmniOS versions under support. This is required to fully mitigate CVE-2020-1472, and will shortly be enforced by Windows domain controllers.

    If you use Windows Active Directory you should at least evaluate.
    https://omniosce.org/releasenotes.html
     
  15. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    Emergency security update due a very critical bug in sudo

    https://www.helpnetsecurity.com/2021/01/27/cve-2021-3156/

    https://blog.qualys.com/vulnerabili...p-based-buffer-overflow-in-sudo-baron-samedit

    https://illumos.topicbox.com/groups...ecurity-update-r36m-r34am-r30cm-baron-samedit

    I suggest all to update sudo as soon as possible (update OS if no fix is available)
    For OmniOS under support (151030-151036) a fix is available now.

    To install the update enter the following
    # pkg update sudo omnios-userland entire
     
    Last edited: Feb 1, 2021
  16. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,006
    Hi gea , my white box esxi server has been great for the last 10 years - but lately a few things have started to glitch so I am itching to get a new beast with more ram and a few more cores than the aging E3-1230.
    • Supermicro X9SCM-F Mobo, 4x pcie, C204, IPMI and iKVM (http://www.supermicro.com/products/m...04/X9SCM-F.cfm)
    • 32GB ECC (4 x 8GB ECC Kingston KVR1333D3E9SK2)
    • 10G Intel 82599 2x SFP+ Fiber card - will move this to the new mobo.
    • 2 x IBM M1015 in IT (will not need these with onboard 3008)
    I saw your post and was wondering if youd recommend the H12SSL-C mainboard with a 16/32 Core 7302 and 128GB RAM?
    I'd likely continue running esxi 6.7 with 8 drives in passthrough to the solarish/nappit server and other sata's for storing vms etc.

    My "norco" has sata backplane and so would need the relevant cables.
    Can you confirm (and pref link to relevant cables) that go from
    1) the H12SSL-C from both the 3008 and
    2) the onboard sata connector to the sata on my norco backplane?)

    Happy to take alt. suggestions as I have been out of the game for a bit.
    It'll be running a 10-15 VM's mix of windows and linux and may enable me to consolidate from my two esxi servers at the moment - ram out of compute and memory :)
     
  17. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    The SuperMicro H12SSL-C (https://www.supermicro.com/en/products/motherboard/H12SSL-C or the variant with 10G onboard) is my planned board to replace older ESXi or storage systems instead with Xeon systems that I have formerly used in the same pricerange. Main advantage is the huge number of pci-e slots and lanes that allows up to 24 NVMe each connected with full 4x performance and the high number of cores available at a quite decent price.

    To connect Sata disks or backplanes with Sata connector to the onboard SAS or Sata connector you need a cable like https://store.supermicro.com/superm...s-with-sideband-75cm-cable-cbl-sast-0699.html

    If the backplane has a miniSAS Connector 8087 you would need (for SAS 3008 only)
    https://www.delock.de/produkte/947_Mini-SAS-HD-SFF-8643/83389/merkmale.html

    If the backplane has 8087 and you want to connect to onboard Sata, you usually need a "reverse" breakout cable. I have not yet seen one from SFF 8643 connectors used for onboard Sata.

    Onboard Sata ports on this board are organized in 2 disks per pci device. It should be possible to use some devices for ESXi and pass-through others, see https://forums.servethehome.com/index.php?threads/supermicro-h12ssl-and-sas-storage-options.31272/
     
    Last edited: Feb 14, 2021
    davros123 likes this.
  18. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,006
  19. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,006
    Went with a H12SSL-i (no onboard 3008 or 10G ethernet) as I have 10G fibre and HBA's already.[​IMG]
    Just waiting on the EPYC 7302P....due next week.
    [​IMG]
     
  20. gea

    gea Member

    Joined:
    May 22, 2011
    Messages:
    211
    OmniOS 151038, Stable and Long-Term-Supported (LTS) Release, TBC of May 2021
    https://github.com/omniosorg/omnios-build/blob/master/doc/ReleaseNotes.md

    There are some important new features available like persistent l2arc, SMB improvements, improvements around Bhyve/LX or improved support for newer hardware ex AMD Zen, Intel X710 or newer chipsets. If you intend a fast switch, you can evaluate the new features in OmniOS 151037 bloody that can be updated in may to 151038.
     
    davros123 likes this.

Share This Page

Advertisement: