1. OCAU Merchandise is available! Check out our 20th Anniversary Mugs, Classic Logo Shirts and much more! Discussion in this thread.
    Dismiss Notice

FreeBSD + ZFS + Web-Interface ready for testing

Discussion in 'Storage & Backup' started by sub.mesa, Jun 23, 2010.

  1. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Hello there!

    On another forum, i began work on building a web-interface to ZFS, by far the most advanced filesystem available and usable at this moment.

    If you (still) don't know what ZFS is or does, go read up at Wikipedia or my FAQ:
    http://en.wikipedia.org/wiki/ZFS
    http://submesa.com/data/zfs/faq

    ZFS is too cool for home users to ignore, who often still store their files with conventional RAIDs and some of those have terrible reliability; especially windows software based RAIDs using proprietary drivers, also known as "FakeRAID", has caused many people to lose valuable data. Since i enjoy storing my data on ZFS so much, i'm inclined to help you guys out getting this to actually run on one of your systems.

    Here's a screenshot to keep things in perspective:
    [​IMG]

    Edit: And here's how it looks at the moment:
    [​IMG]

    To try this out, you could use two methods:
    1. The manual installation of FreeBSD using my FreeBSD install/setup guide (advanced)
    2. Use the pre-installed binary image and write it to a small HDD or USB pendrive of at least 614MB large, following the Mesa install instructions (easier)

    If any of you love running ZFS, but prefer a web-interface, and don't want to use the unstable/experimental ZFS version currently implemented in FreeNAS, then please test my interface! Please bear in mind this is a pre-release, not intended to be used for real usage.

    This is a community project, meaning its idea was raised on another forum (HardOCP) where the idea developed into a first prototype release, and now a prerelease based on FreeBSD 8.1-RC1. Since my project affects basically everyone interested in modern data storage using a NAS fileserver, i feel it is appropriate to broaden my horizon to other concentrations of home 'power-users' - like on this forum perhaps.

    Many parts of the interface are also still incomplete. But that's where your feedback comes into play: tell me what you need to make this a good solution to you, and if i'm able to i'll implement those suggestions in future releases. It already has a remote update feature so you can update the scripts and discover new functionality added.

    Cheers!
     
    Last edited: Aug 1, 2010
  2. sreg0r

    sreg0r Member

    Joined:
    Jul 9, 2001
    Messages:
    1,146
    Location:
    Melbourne
    been following your progress on the [h] and this looks like a really promising project. If I can find some spare time I'll definitely try this out in a VM. Perhaps it's good enough to tear me away from opensolaris?
     
  3. OP
    OP
    sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Well OpenSolaris should be fine too, but since i already knew FreeBSD and likely FreeBSD is a better home for ZFS in the long run, i decided to build around FreeBSD instead.

    Some advantages of BSD:
    - truly opensource, more so than linux or basically anything; its as close to public domain as possible
    - the coolest technologies either come from BSD or have been ported to BSD; OpenBSD's pf firewall, Sun's ZFS, Sun's D-Trace, geom I/O framework, threading scheduler (which was used to improve Linux kernel and Firefox projects) and so on.
    - extended ZFS support (booting from RAID-Z; not possible under OpenSolaris)
    - better hardware support
    - geom label allows you to assign human readable labels (i.e. "disk3") to disks, allowing you to know which disk you're working on; if you ever connect them totally different they will still be identified by the same name.
    - portstree is useful
    - possibly a better community

    One significant down-side is that FreeBSD has no kernel-level CIFS driver like OpenSolaris does. That means you have to use Samba, the worst open-source project that became popular but is rotten in and out.

    If possible, i suggest using NFS, iSCSI, FTP or basically anything over CIFS. I encountered too many problems with CIFS to be usable; though it works fine for other people apparently.
     
  4. fredhoon

    fredhoon Member

    Joined:
    Jun 27, 2003
    Messages:
    3,200
    Location:
    Brisbane
    I'm keen to give this a bash as well (eventually). I have a need to upgrade the home file server and will explorer options in ZFS, unRAID or mdraid. I was going to test the waters with opensolaris, however I’ve always wanted to play with open/freeBSD, but haven't the knowledge or the time. This 'easy way in' might convince me to put some effort in the BSD direction.

    As a side note i'm also keen to check out native ZFS support in linux that Saba linked to recently. That said, unRAID would be a better match for my usage, however I prefer the flexibility & (beginner level) support of a common linux distribution.
     
  5. Statts

    Statts RIP

    Joined:
    Aug 29, 2004
    Messages:
    2,504
    Location:
    Radelaide, SA
    Subscribed!
     
  6. barbz

    barbz Member

    Joined:
    Dec 12, 2006
    Messages:
    272
    Location:
    Brisbane
    Link to download your pre-release file is broken.

    Keen to test it out tho!

    Ta
    Paul
     
  7. OP
    OP
    sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Oops! Was an out-of-memory error in my scripts. Could you try again?
     
  8. thebranded

    thebranded Member

    Joined:
    Nov 30, 2006
    Messages:
    2,244
    Location:
    Sydney
    looks good, been meaning to upgrade my freenas system
     
  9. ford ftw

    ford ftw Member

    Joined:
    Jun 25, 2008
    Messages:
    1,449
    Has this got ISCSI support?
     
  10. OP
    OP
    sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Yes, it has pre-installed:
    - Samba
    - NFS
    - iSCSI-target (istgt)
    - rsync
    - pureftpd

    But currently the GUI does not allow you to share data using the GUI; that's one of the last few core functionality missing at this point. But the software is pre-installed; all it needs is an extended GUI that actually uses it.

    You can set up the iSCSI for yourself, using my iSCSI on FreeBSD guide:
    http://submesa.com/data/bsd/istgt

    Remember, the advantage of my "Mesa" Web-GUI is that it is true FreeBSD 8.1RC1; not some stripped down version. It has everything a full (minimal) FreeBSD install has; meaning you can do anything normally possible with FreeBSD, unlike FreeNAS which basically is untweakable.

    As my GUI progresses, it will be able to do more things and thus the user would need the CLI less and less, though i feel CLI still cannot be completely replaced by GUI neither does it have to be.
     
    Last edited: Jun 29, 2010
  11. flain

    flain Member

    Joined:
    Oct 5, 2005
    Messages:
    3,181
    Location:
    Sydney
    which pool version does it support up to? i'm running opensolaris snv134 but would like to try this. I have a feeling tho that the freebsd implementaiton of ZFS still runs an old pool version?
     
  12. bRuZdEGO

    bRuZdEGO Member

    Joined:
    May 19, 2004
    Messages:
    35
    Location:
    Sydney
  13. DavidRa

    DavidRa Member

    Joined:
    Jun 8, 2002
    Messages:
    3,102
    Location:
    NSW Central Coast
    FreeBSD is still at v13 or v14 (depending on which tree you're tracking). v24 is being ported, I believe, but is not there yet (I want v22+ for dedupe of VMs, basically).

    I've had some minor stability problems when using FreeBSD ZFS on older kernels (FreeNAS is 7.3 still) - whether by iSCSI or Samba, I can exhaust kernel memory (panic!) by writing half a TB to the volume.
     
  14. DrGeforce3

    DrGeforce3 Member

    Joined:
    Mar 8, 2002
    Messages:
    15
    Location:
    Adelaide, Australia
    You must use the amd64 builds as ZFS is built for 64bit.
     
  15. DrGeforce3

    DrGeforce3 Member

    Joined:
    Mar 8, 2002
    Messages:
    15
    Location:
    Adelaide, Australia
    Feature Request

    ZFS Snapshot Management would be a very nice feature to add to the web interface.

    Also a search interface for searching files in snapshots could also be useful.
     
  16. bleckers

    bleckers Member

    Joined:
    Sep 19, 2005
    Messages:
    284
    Location:
    St Kilda
    Would you be looking to port this over to OpenSolaris?
     
  17. OP
    OP
    sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Currently in Mesa web-storage i use FreeBSD 8.1RC1, which uses ZFS version 14. This version of ZFS is stable, and not labeled as 'experimental'.

    Earlier versions (everything before version 13) was considered to be an experimental feature, and printed a warning at the console when the ZFS module was loaded. It also prints those warnings if you're on a 32-bit system or have too few memory. You need an amd64 machine with more than 4GB memory for no warning messages to be printed. 4GB or lower memory would already start disabling prefetching as you need more RAM to use that effectively.

    People who want deduplication (basically to save space by not storing two identical files twice but instead only once) - would either have to run unstable ZFS or wait until it has stabilized.

    Currently, the highest stable ZFS version is 14. Anything higher is not production quality, and deduplication still had bugs a few weeks back; you do NOT want to run this on your system holding your precious data without backup. Even if you have a full backup you have to thread carefully.

    Should be possible, yes.
    But certainly not on the short term. OpenSolaris already has a web-interface, and i'm just a lot more familiar with FreeBSD. Also FreeBSD can label a disk much better, and has extended ZFS support. For example, on FreeBSD you can boot directly from your RAID-Z or RAID-Z2; while with (Open)Solaris you need to have a separate system disk; as Solaris cannot boot from ZFS RAID-Z arrays.

    FreeBSD should also have better hardware support, and unlike OpenSolaris, i consider it to be truely opensource and free software. That may be important as no commercial interests can influence development and support of ZFS in FreeBSD. I also have a lot of trust in the FreeBSD developers; the best i've seen in OpenSource-world.

    In other words, i see FreeBSD as the platform to develop for regarding ZFS, even though it may have ZFS v24 a little later than OpenSolaris would have.
     
  18. Statts

    Statts RIP

    Joined:
    Aug 29, 2004
    Messages:
    2,504
    Location:
    Radelaide, SA
    How long do you imagine until this project is at a 'usable' state?
     
  19. Snuffy22

    Snuffy22 Member

    Joined:
    Sep 1, 2004
    Messages:
    652
    Location:
    Brisbane
    looks interesting, subscribed :)
    i shall wait till the cifs features are fully implemented :)
     
  20. grs1961

    grs1961 Member

    Joined:
    Jan 21, 2005
    Messages:
    548
    Location:
    Melbourne
    Strange (and silly) restriction.

    ZFS is in both the 32- and 64-bit Solaris and OpenSolaris kernels.

    (Of course, it runs much better in 64-bit than 32-bit, but still...)
     

Share This Page

Advertisement: