DIY pfSense! Hello again OCAU! Sorry for the delay, life got in the way, and my wife decided to break her foot. On we go! As from my other threads, and my post’s around the place, and more than a few PM’s, I’m a bit of a pfSense fan. So I thought I would finally post a review/overview/rambling thoughts on some new hardware I built, and also a more thorough review of the current state of pfSense (as of 08/03/2012 the stable release is 2.0.2). This also will be (unavoidably) a bit of a How-To, only because I don’t know any other way to explain an installation without detailing how it’s done. A bit of background first, I work in a small government site (<100 people) with a very tight IT budget. I’m not a networking or security person (except out of necessity), however as most of you know (and many of you do!), IT on small sites get to do a bit of everything. That said I have several open source systems in place now, from my iSCSI server (OpenFiler), PBX (Asterisk+FreePBX), monitoring (Opsview/Nagios), and document distribution (ownCloud). Anything that I can build or do for free (or very little dollars) is appreciated. Originally we replaced a Fortigate 100A with a ClearBox 300 and ClearOS (which was an absolute disaster, and not in the scope of this post), and then I replaced the OS on the Clearbox hardware with pfSense, and couldn’t be happier. So a few weeks ago we had a major hardware failure on our ClearOS box, so we thought we would replace it, with something a bit more “standard”. Click to view full size! Meet my new firewall. I call it “Standard Overkill” Onto some pictures and explanations of what I’m trying to achieve. (Ignore the silver POS PSU I had, was just to test while i waited for my real PSU’s to arrive) Click to view full size! Click to view full size! Hardware first! As I said before I’m going for a more standard build, with ATX hardware and off the shelf components. This reduces cost, as it’s basically a PC, with very easy repairability. And yes, why build one.. Click to view full size! .. when you can build two for twice the price? So the idea of this is to have a HA firewall/UTM solution. Cases are PCI Case IPC-RK-238B models, so they take a Micro-ATX motherboard, standard ATX power supplies, and low profile cards, as well as up to 2 3.5 inch drives and 2 5.25 inch optical drives. Click to view full size! They are solid cases, I love the ATX PSU space, and they are nicely powder-coated. Not tool-less of course, and the way you mount drives is very fiddly. They do have rubber grommets on the base to reduce noise. The 5.25 cage is very painful to remove, you need to undo 4 screws, 2 on the bottom of the case, and 2 inside the case, then you slide the cage out. Other than that, there are a few sharp edges, but I didn’t cut myself during the build. Fans are dead quiet, great for them being in my office. For $200, (no PSU) they are OK value, but not great. Now the guts of the machine: Click to view full size! Click to view full size! I needed a M/B that was MicroATX, as well as having an Intel NIC and 3 PCI Express slots. The only one I could find with this G1 Killer M3 from Gigabyte. It has all solid state caps, a Z77 chipset, as well as being bright green. You also get a few sheets of stickers, as well as a poster as part of the package. They went up in the server room, just for a laugh. This is also my first experience with UEFI at a desktop level, (I have many IBM servers with UEFI) and I like Gigabytes implementation. It’s very easy to navigate, though I completely ignored the overclocking section of the BIOS. Only things I set were Always on, and boot from SATA only. Click to view full size! Processor is an Intel i5-3470S, the (slightly) lower-power version, with a 65W TDP. Superb performance and not that expensive. RAM is 8GB of Kingston’s Hyper-X Blu series, only because it had tight timings and was on the approved RAM list. Not much else to say about it. Click to view full size! Now I’m using SSD for storage, and I’m well aware that Squid and SSD’s don’t mix, however I don’t mind throwing the SSD out after year for the performance gain in cache speed. From what I have read, the Samsung 840 pro’s have the best garbage collection in that price range, and since pfSense's FreeBSD version (8.1 RELEASE) doesn’t support TRIM yet, (it will!) that what I needed. 120GB is more than enough, the pf install is only a few MB, and the cache will be only a few GB. Squid will put everything in RAM if I tell it to anyway. I’m also NOT using mirroring, because software on FreeBSD (gmirror) is useless, the on-board is fake RAID (and just a useless as software raid) and it’s just as easy to have a “cold” spare in the box, and replace as needed. In a HA environment, I can take one offline with no drama. Click to view full size! Network cards are Intel I350-T2 dual port low-profile PCIe, giving 7 ports in total. I don’t need TCP offload or anything fancy, just interfaces. These also have native FreeBSD support. The PSU is a an FSP Silencer 400W unit, 80+ Gold certified as well as being fanless. It scored very well in the JohnnyGuru review, and I won’t be pushing it very hard at all, maybe drawing 100w at most. So put it all together: Click to view full size! Click to view full size! Click to view full size! The LCD is from LCDmodkit, it’s just a standard HD44880 4x20 characters, with a USB controller board on the back (in Red on Black). They work very well under pfSense, I’ve bought many from him over the years. You can output almost anything to it, we will talk more about that below. Lastly, I hooked up an internal USB header, so I can have a small USB stick in there. Not to boot off, although you can with a version of pfSense on NanoBSD, just for config backups. I have a cron job that runs once a week to back up the configs to the USB. So that’s it for the hardware, pretty standard stuff, on purpose. This only took a few hours to build, and I took my time, and tested them overnight. Click to view full size! Click to view full size! Click to view full size! Software time! Installation is a snap, the only gotcha here is that FreeBSD didn’t recognise the SATA3 ports, but plug the drive into a SATA2 and you are good to go. I just chose defaults on the install, using all the drive and default swap space. After that just choose a WAN and LAN port, then you are ready to configure from the front-end. As an aside here, this is FAST. Takes seconds to build and 30 seconds to reboot, from the time you press the reboot button. There is actually a nice wizard to take you through the initial setup if you like, otherwise we should jump right into configuring. There are some initial setting you should look at, particularly in the general and advanced settings, for ssh, and password protecting the console (definitely!) and a few others. Click to view full size! Click to view full size! Click to view full size! Then we can assign interfaces as such. Click to view full size! Finally the rules! Click to view full size! Click to view full size! Click to view full size! Now we can look at some of the really nice aspects of pf, making it a proper UTM, with content filtering, antivirus, and the such. Firstly, you “build” your copy of pfSense, with packages that are published from pf and contributors: Click to view full size! Click to view full size! Most of the obvious ones are there, like snort and squid. There is HVAP, which you can parent with squid. Then as an extension from squid, you can use dansguardian, or squidguard, however a subscription to rules will cost you money, but it’s really a nominal amount. With 2.0.2 PPTP is disabled by default, and there is a warning message if you try and enable it, due to the compromise of the MS-CHAP protocol. OpenVPN is a great alternative, and there is now an iPad app to support it. The windows client isn’t very pretty, but it’s very easy to deploy and use, and you can sync it to your Active Directory or any other LDAP. The LCD module is configured here, though you might have to drop to a shell to run dmesg, and then edit the config file to suit. in the case of these boxes, the LCD found itself on ugen1.3. Then you can output whatever you want to the display and rotate it every few seconds. Click to view full size! Click to view full size! I wanted this to be more about the software then the hardware, since the hardware is all of-the-shelf stuff. I wanted to demonstrate and impress upon people that for commodity stuff like a firewall, VPN and Internet filtering, there really isn’t a need to pay for it. Even for larger sites. To give you a breakdown of costs: (Rough guide, includes GST) Case: $200 CPU: $220 Motherboard: $212 RAM: $71 PSU: $150 SSDs: $ 328 NICs: $450 So total hardware (each) is : $1631. Software = $0 Maintenance = $0 Support = $0 Just for reference last time I got quoted to a XTM505 was $2400 for the hardware, plus NBD replacement $600, and the “Pro” upgrade (more clients basically) $220. so call it $3600, and I can’t fix, upgrade, or manage it outside a windows client. Or use a secure VPN solution without paying more for their client. And now we talk about the elephant in the room, support. For me, and for most SME’s that I’ve dealt with, this is a bit of a non-issue, as I’m the first support call anyway. If I can’t fix it, well I’d have to wait for support anyway. In an N+1 environment, the chances are small that both would fail. I understand that this sort of arrangement isn’t for everyone, but it’s an acceptable risk for me. There is no support outside of me and my team, and you can get corporate support from pfSense for $500 USD a year, with 5 hours remote support. In the several years I’ve used pfSense, I’ve never had a failure via a software fault, besides my own stupidity with gmirror. I suppose this sort of thing isn’t for everyone, however it really works for me. SO my final thoughts. This has been a very worthwhile exercise, and you don’t need to do what I’ve done, However I want this hardware to last a few years at least. I’ve seen sites similar to mine run on much less hardware, even an embedded Atom system would be OK. This flies along, and it really, really easy to backup and restore, even to replace with any sort of hardware you may have lying around. The real seller of this sort of setup for me is the close-to-zero (power and depreciation notwithstanding) ongoing costs. The facts that it performs as well as it does is a bonus. AS always, I welcome questions or comments, and if you want to know anything more, just let me know below or in a PM. Though I would prefer to keep any discussion public. Cheers, Mario. PS. I bought some other toys, there will be another thread in a bit.