Recently transitioned from Ubiquit stuff to a OPNSense setup. It was such a good decision. The firewall rules make much more sense. Better functions than the dream machine series. You can also get a lot more for the same price. Ubiquti hardware is very under spec for the money you pay for. Highly recommend this guide to setup your own. It’s very dense and more verbose than you need so skip the irrelevant sections.
I switched to pfSense from EdgeRouter a few years back, and find the firewall rules make _less_ sense. The reason is likely that I understand IPTables pretty well, where as the approach used in pfSense seems "abstract" in comparison.
I'd certainly recommend grabbing something like a Protectli box (if power draw is a concern) or building a small server with NICs to install OPNSense on over the Ubiquiti stuff.
For me, the router graveyard was getting out of hand, buying everything from mid-range to high-end consumer routers only to have them left behind software-security wise within 3 years, I needed something open.
The promise of the EdgeRouter range was the hardware offload and Debian based OS, but Ubiquiti has fallen out of favour with me in that space, their software has gotten worse rather than better over time.
I went with Protectli because of the ability to use Coreboot meaning I could get as much of the stack Open Source as possible. Unfortunately, at the time I set it up, OPNSense didn't work for my requirements and I had to use pfSense; pfSense is now too falling out of favour, but I don't have the time to swap over a fairly large home network without pissing off the family.
I've had great success with an old Dell Optiplex computer running xcp-ng with PCI passthrough of the NIC to pfSense. I was considering a protectli box, but for $300 on ebay + an old NIC I had laying around I got something much more powerful (Intel I5-8600 w/ 16GB of RAM) than even the most expensive protectli offering, which was ~$1500 last time I priced it out. I calculated out the power draw and the Optiplex ends up with only a cost of about an extra $1.10 on my power bill per month at high usage, which was the deciding factor.
As a bonus I have xcp-ng spin up a ubuntu server with Portainer running things like pihole / unifi controller / plex / a terraria server. I also decided to randomly throw in a cheap nvidia GPU for some extra oomph. Highly recommend it for tinkerers, and the only reason I would spend more for the protectli is if the fan noise of a mini PC might be a nuisance.
I recently switched to RouterOS. The learning curve was a bit high to me. I'm still tinkering with it but I got my main things working - vlans, default internet access out via vpn, one port (internet accessible server) routed without vpn.
I learned a lot in the last 3 weeks it also helped me understand networking a lot more. Sure I had to do a lot of trial and error or figure out why things don't work but in the end it was worth it for me.
I came from 30€ OpenWRT routers with only 100mbit links which is why I upgraded.
The hAP ax³ were 140€ a piece (I use two) and the one 2.5Gb PoE port is actually nice since it powers the second router. It was a pricey upgrade compared to what was there before. I tried to buy another OpenWRT router but RouterOS seems to offer 10 times more compared to what OpenWRT can do.
I've looked at a lot of things even one of the more looked at super cheap thin client for this stuff (Fujitsu Futro S920).
At home I can't really justify a dedicated firewall.
May I ask which VPN you are using? I have tried several and they are always blocked somewhere. I tried: Mullvad, IVPN, ProtonVPN.
So far the only one which is never blocked is cloudflare’s WARP but I suspect it’s because it’s newish (the VPN functionality of WARP is at least, originally it was mostly about DNS). But it has severe performance problems when running on a router as far as I can tell, I suspect it has to do with MTU, but I couldn’t solve it so far.
Next I was going to Google One’s VPN, but I don’t think it’s a candidate for me, I want to use a VPN for privacy, so google is probably not the best choice.
The sites which are blocked are mostly enterprise login pages. As I work from home, it’s a no go for me. Secondly, I need to connect via VPN to some company network a couple of times a day and the performance of “VPN over VPN” is currently catastophic, it’s just unusable. Not sure if it’s also related to MTU…
RouterOS has a steeper learning curve, but I find it very comforting that it is predictable/boring and I run it on hardware that is easily replacable. I have a cheap Mikrotik 1U unit.
I'm the other way, I find IPTables less intuitive and featureful than PF. The way Netfilter/IPTables works under the hood seems much more over-complicated[1] than it needs to be (which is how I feel about a lot of Linux system stuff these days). The architecture of PF[2] (and thus how the rules get processed) just makes more sense to me.
pfSense/OPNsense is just a stripped-down FreeBSD under the hood. If there are things that you want it to do that you can't with the base install, you can look into installing FreeBSD packages.[3][4] I personally just use vanilla OpenBSD for most edge router/firewall tasks and then if I need mesh wifi or some modern gear that is not well supported, I just delegate the BSD box to run a strong PF firewall (which is what pfSense uses) in front of it all. Ubiquiti mesh wifi and RTSP stream play really nicely with it.
It's so easy to overcomplicate a routing setup which is probably why these off-the-shelf solutions are so popular, but I agree, I prefer to have less of a magic box approach to routing/firewalls whenever possible.
I use exactly the same thing after I got tired of pfSense: straight OpenBSD, PF, unbound, and a couple of other native obsd things (starting to play with OpenBGPD for AS level filtering) on NUC-alikes (currently liking Protectli and Seeed). No pretty web interface, but the documentation is so good it was pretty trivial to set up. This has worked really well for several years now.
Note: like vogon said above, it's easy to overcomplicate things like this. If you go this route, start with the most basic configuration that works, and only go looking for 'tuning' or 'optimizing' if you run into a specific issue and can
measure before-and-after improvements. Obsd does a very good job of working correctly out of the box, and if you follow the advice of some 'popular' PF tuning sites, you'll end up with tons of stuff you probably don't need, probably will interact in weird ways and will get you laughed at when you go looking for help. KISS.
One caveat: Don't expect to go to the obsd mailing lists or IRC channels for beginner support.
I definitely believe PF/Netfilter is the way forward, I just don't have the experience with it yet.
I'd love to build a BSD based router/firewall in a declarative manner/source controlled configuration on top of a vanilla OS, I just don't have the time (specifically network downtime for the home, family) to play with it.
I like graphs and charts, but otherwise not a massive fine of GUIs that hide functionality and complexity, would rather know/understand exactly what's going on at the CLI.
Thanks for the links, no time like the present to learn more!
I get that OPNSense can do this, but do you need a switch with a capability to make sense of this?
I'm considering moving from Unifi USG to OPNSense and have two Cat6a runs from one end of the house to the other (through the loft and it's not possible to add more runs without building/decorating work). Presently the two cables do WAN and LAN, but I've been curious about putting something closer to the modem and to somehow use both cables for the LAN.
LAGG looks like it can do this and isn't something I knew about for the home.
Would I need a special switch on the other end? I've currently got Ubiquiti switches but as I'm already looking at binning the USG I'm fairly open to reconsidering a lot of the network.
PS: The reason to abandon the USG is heat issues. Packet loss when the ambient room temperature exceeds 30'c, and serious packet loss when the room temp is 35'c. This is no longer rare, and the USG is only rated to ambient temp of 40'c and there are many Reddit threads of people ripping the case apart and fitting fans. I'd rather just have stable internet with better hardware.
You need a switch that can handle LAGG but I don’t think you need a particularly expensive or fancy switch. I run a TP-Link jetstream switch that supports LACP and was only $110 (same price as a 8 port lite ubiquti switch). It has 8 port PoE+ AND 2 port SFP. You have to run a omada controller (similar to a unifi controller) but you can buy a box (OC200) or just run a Docker image which is what I do. I find tbe TP-link APs much better performing at a reasonable price too. Ubiquti with their no free shipping and price increases has gone completely out of hand since the pandemic. Made me switch out all their hardware because of it. Ubiquti hardware was simply not worth their price and OPNsense + omada had blown my Ubiquti setup out of the water.
Alternatively, you can pick up a used switch from Cisco, Dell, etc. that have excellent support for LAGG and other useful things for significantly less than the TP-Link (as long as you don't need 10G). I think the last 2960Gs I picked up were less than US$30/pc. Down side is 1) they use more power, 2) bigger switches (24- or 48-ports) are much louder, and 3) not going to see any new OS updates for most of them. So you have lots of options.
I shall check those out; thanks for the tip. I will say I checked and it looks like my elderly Dell switches have gotten an update in the 2020s, so perhaps it's just Cisco who said "nope...we're done with IOS 12".
I think there are multiple ways to set up a lagg, but the one I've tried and works fine is LACP. This requires an equipment that can handle this at the other end, though. I'm not familiar with Ubiquiti kit, so you'd have to check if it's supported.
if you want to have two links grouped together so that a single transfer uses both links together (2gbps) then the switch has to support it. default is for two separate transfers to each get one gbit, which is good enough for most applications.
packet loss at summer temps is indicative of faulty hardware (maybe just the thermal paste or other parts of the heat management)
> packet loss at summer temps is indicative of faulty hardware (maybe just the thermal paste or other parts of the heat management)
there is no heat management in the Ubnt USG. no thermal paste on the hot network ports, no cooling design beyond passive cooling (little air holes in the side of the case) that doesn't work when it's laid flat (need to vertically mount to encourage airflow).
I've been looking at this as well. I'm currently running my gigabit fiber connection through a GPON-ethernet media converter and from there over Cat 6 to a ubiquiti Edgerouter-X. It does okay with hardware offloading enabled for things like ipsec and NAT, but it's taxing the CPU and I'd like to move to something with a little more headroom.
Got any recommendations for something that can route beyond gigabit, when NATing and DPIing and other things?
> Got any recommendations for something that can route beyond gigabit, when NATing and DPIing and other things?
goto recommendation for the last decade is a used small formfactor enterprise desktop or a laptop. the former allows for a pcie nic and more performance while the latter usually requires a usb-eth but has a builtin keyboard and screen for debugging.
just go for something x86, avoid the ultra low-end cpu's and your usually good for soho stuff.
You should checkout the new R862 mini pcs. You can get 3X 2.5 NIC and 2 SFP port with 10G support. The homenetworkguy did a great review on it. But overall, 10G works but only with nothing turned on. With IPS/IDS and proper MTU, you can get about 3 Gbps. But all this for sub $400 in a computer about the size of your hand is insane to me.
I think you mean R86S, I wasn’t able to find something relevant by R862. They’re pretty interesting. I am sort of looking to replace my R210ii and this fits the bill but downgrading cpu performance (for a great deal of efficiency gain) kind of feels bad.
Aha yeah, I’m on mobile so it was a typo. I currently run a fanless mini pc with N100 and 4X 2.5G which has been great. If you don’t need SFP and can live with LAGG + 2.5G, you can probably get pretty similar real world speeds compared to a 10G SFP line.
I’ve been thinking about swapping out my UDMP for opnsense, but keeping the ubiquiti APs, they’re the best I’ve ever known and I think they play reasonably well with non-ubiquiti stuff. Thanks for the link to the guide!
OPNsense is the core router platform I default to for all my network infrastructure (work devops env, homelab, vpn to family members etc). Its feature packed and ROCK solid. I almost always run it in a virtual machine so i can live migrate it between hosts and have no downtime. The cluster / high availability works great and ensures no loss of connectivity during upgrades. OPNsense is a true hidden gem in the open source world.
I'm curious if you have any suggestions re configuration management? At the moment i'm just point and click configuring but i'd really like to move towards using source control, edit some config file and push configuration to it.
Do you (or anyone else reading) have any suggestions?
There is a git plugin that pushes and commits every change automatically. But be carefully where you push it as the config file can contain sensitive stuff! I have my own Gitea behind the firewall.
Has there been a positive inflection in its quality recently? I tried it a while back but pretty quickly it got Unbound's config XML in a state which wouldn't allow the daemon to run. I had to get in and fix it up by hand to get it going again. Wasn't impressed with the quality it was showing me, switched to pfSense and haven't had any similar issues, so I haven't felt the need to look at OPNsense again.
I did a bare metal migration and it was pretty painless. Had to reinstall some packages, but it was as simple as just hitting the + on the package manager.
Yes! The single file xml config export is super easy to move an install between systems (physical and virtual). There is even a plugin to manage the changes with git!
Every time I've given up on an OPNsense instance and re-installed it, importing a config backup, half the config didn't import. Maybe things are better these days.
I used to find it rock solid, but around two years ago reliability tanked. I found myself regularly having issues with interfaces (a genuine Intel server-grade multi-port NIC) flip-flopping. About a year ago, I started having random issues with traffic no longer routing, out of the blue. Lately both issues seem to have gone away.
Right now the software update function dies half the time I try to run a check, with a long sqlite query string / error being dumped to the console. This has been going on for at least the last couple of months worth of releases.
About a year or two after install, reboots and power-offs stopped working. The system just hangs instead after printing out a message about USB, and I cannot figure out for the life of me what's wrong. It's a standard Dell SFF PC, nothing exotic, and had been working fine until a major release broke it. FreeBSD's documentation about ACPI is impenetrable, so I can't figure out what's going on.
Startups and reboots used to be lightning quick, with maybe a minute or less between the bootloader kicking off loading the kernel and interfaces/routing/firewall up and it giving its happy chime. These days the system spins its wheels for ages doing...something, not sure what.
I find the project pretty outdated and behind the times. The UI purposefully obtuse with terrible organization and field names and a lot of missing help text to keep their support/consulting biz strong.
They're really far behind on features. There's no application blocking, monitoring/diag is rudimentary, it has extremely limited backup functions (Google Drive and that's it, I believe), and even the DNS blocklist functionality is extremely rudimentary, with only a fixed list of really trashy, unreliably lists available to pick from (one of the groups they pull lists from has demonstrated extensive issues with QA, routinely including things like certificate validation servers in their blacklists.) They've also gone out of their way to make the Adguard Home plugin annoying and confusing to get working if you want to configure it as a proxy to unbound, which is needed if you want DHCP hostname records to work (speaking of which, DHCP leases are needlessly obtuse to mange.)
Their release process is wildly unsuitable for production network equipment. A 'major' release is immediately EOL'd as soon as the next major release comes out. Running 20.1 and need to stay on it because 20.2 breaks something or you want to wait for the dust to settle? Too bad. There's no security releases for older major revisions. And it wouldn't be so bad if each major release was followed by a number of "oops we fucked up...." point-point releases because their QA isn't very good.
The devs are sticks in the mud, too - mostly "franco." They bitched and moaned up a storm for YEARS about wireguard being "insecure" despite no evidence to back their claims, citing that as the reason for refusing numerous requests for integration, and even refusing code contributions from the community for it. They eventually caved. The wireguard plugin is still pretty meh and difficult to navigate unless you know wireguard well.
ARM support? Zero interest in even assisting community efforts, which have gotten impressively far with it, especially now that ARM support in FreeBSD got appreciably better in the last release or two. I suspect it's because they see it as a threat to their (grossly overpriced) hardware offerings.
The list goes on.
They forked pfSense (a good thing, the pfSense devs were being massive dicks) but seem to now be largely on "cruise control" and leveraging community goodwill.
> About a year or two after install, reboots and power-offs stopped working. The system just hangs instead after printing out a message about USB, and I cannot figure out for the life of me what's wrong.
My initial read here would be that this is where serial port / console redirection is happening. There should be settings in the BiOS you can look at however I don't know how limited PC's are in their options / functionality.
>I find the project pretty outdated and behind the times. The UI purposefully obtuse with terrible organization and field names and a lot of missing help text to keep their support/consulting biz strong.
pfSense was the same way as well as most projects if you understand the underlying configurations. You can find people saying the same about ubiquiti's interface in this thread as well. In my experience the GUI is to capture the 80% of mostly default configurations.
What other firewall/routing software have you looked at in comparison to opnsense? I'm interested in what other features they have? The API interface and IDS functionality was one of the draws for me.
>Their release process is wildly unsuitable for production network equipment. A 'major' release is immediately EOL'd as soon as the next major release comes out. Running 20.1 and need to stay on it because 20.2 breaks something or you want to wait for the dust to settle? Too bad. There's no security releases for older major revisions.
I mean are you saying this as a paying customer? Free always has its risks and costs.
> The devs are sticks in the mud, too - mostly "franco."
> wireguard being "insecure"
> ARM support? Zero interest in even assisting community efforts
I observe this with projects overtime and this usually just adds to the bloat and disorganization because everyone is looking for "their" one-stop solution. I think its useful to consider things from other view points and complexities you may not have insight into. Not that I have any specific insight into this project however, there are other companies that make a lot money off networking gear, firewalls, etc and provide what you are asking for but the price isn't free. I assume most open source projects are "best-effort" unless they have a formal revenue stream or foundation behind them and even then I wouldn't expect any claim to expect features or support (not saying you are, just generally).
You could be right but thats a big gap between "could likely do well selling..." and actually having a market and reliable customers.
What if thats not their business model? What if it requires hiring or finding a dedicated ARM developer? What about security? If a zero day comes out now what you have (3) different architectures to support and test. Even if they offered it as a supported solution would most of the people complaining fund the work through support. Probably not, because they expect the software to just "work" for them, for free.
I looked through the forum on one of the first ARM[1] posts and as expected (2) pages in and it becomes a tech support thread for people who want to try the latest but be handed the answers. For a project where they document[2] the development workflow, architecture, and environment its a bit difficult to understand the complaints when its opensource. Clone the repo and get to work.
The pfsense people are, frankly, fucking clowns. They bought the domain opnsense.com and used it to badmouth the opnsense project[0]. Do not give pfsense time, attention, or money.
I remember reading this nearer the time, but I never clicked through to see just what was on the site; having just looked at the cache, I'm appalled. That stuff goes beyond petty, to the point that whoever is behind that should not only NOT be doing business with the grown ups, they should be in a facility seriously re-evaluating their life choices. What a disgrace.
Well, you don’t have to wonder, the owner of the domain was revealed by court action to be Jamie Thompson, one of the two founder of Netgate, which sells the commercial version of pfSense. Surprise…
When trying to choose between the two, it struck me how unacceptable the behaviour of people on the pfsense project are on platforms like reddit; this was enough to tip the scales permanently. Toxicity and arrogance were prevalent, along with actively trying to squash discussions about non-Netgate hardware (the pfsense brand), while touting the project as open source.
not to dispute your points but i rekon this goes along dealing with semi-noobs and prosumers that don't necessarily bring any business but demand attention for the better part of two decades.
PfSense has some pretty bad problems relating to interfaces disappearing temporarily and services dying from it. I would go as far as saying 2.6.0 is unreliable.
With PCEngines shutting down it’s almost impossible to find reliable, cost effective hardware. I hope pfSense gets back on track because their hardware seems ok.
I recently moved my main router from a Netgate device running pfSense to a Deciso device running OPNsense, and couldn't be happier with the change. The Deciso hardware is pricey, but you get what you pay for. Very well built, silent, and very powerful hardware. No issues whatsoever.
They also like to tell you why you're wrong for wanting wireguard (so it's for your own good they don't support it) instead of letting you make up your own mind. No thanks.
As an alternative, I've been watching VyOS with great interest and it seems like they are finally going to release their controller and LocalUI interface this year, which is exciting. It seems to have a similar architecture as a Ubiquiti controller.
Yeah I’m waiting for a good open source Linux-based firewall, it’s way overdue. Linux has surpassed FreeBSD in terms of networking, at least for the home/SMB router use case. I’m talking about Cake, eBPF, etc.
I’m currently running Mikrotik x86 on a NUC, but their hardware support on x86 isn’t great. I’d rather switch to Vyos, but I’m too lazy to learn their CLI.
Semi-related: OpenWRT is also genuinely not bad on x86, but it’s slightly too simplified compared to say Mikrotik or VyOS. It also runs great in a VM.
As I said above, it's stuff like Cake, which is an advanced QoS scheduler that helps eliminate bufferbloat (very relevant for home networks). Additionally, eBPF and XDP will soon be integrated into standard firewalls, which should speed them up dramatically, allowing for crazy high speed (or power efficient) routing on commodity hardware.
The documentation about the build process is awesome, thanks!
And btw:
> Everyone can build an LTS release image from the stable branch too. For 1.2.x, the branch is named “crux”. The image built from the branch is equivalent to the latest official LTS image.
The Ubiquiti controller is fine if you're managing one site, but beyond that it's pretty poorly designed IMO.
By far the biggest problem is that they don't give enough consideration to recover-ability of offsite devices. If something causes a device to disconnect from the controller their solution is to SSH in and re-adopt it. That doesn't work once you're dealing with hundreds of devices across dozens of sites.
A good example of where that becomes problematic is to look at the controller hostname override setting they have. You could change it to 'unifi.invalid' and it would happily push that change out to a thousand devices and leave them in a state where you'd need to be hands on with every device to recover. If you can do it on purpose, they can do it accidentally with a buggy update, so, IMHO, there's always a risk that an update could break things pretty badly.
That's not a hypothetical either. They (purposely) pushed an update that did something like that when they started supporting HTTPS for the inform URL several years ago.
The second issue I have with it is more of a design issue. Sites should be somewhat sharded and I should be able to update the controller version on a per-site basis. I think that does a lot to reduce the risk of an update breaking things.
I also dislike the default settings and prompts for auto-updates. I like the scheduled updates and think they're great, but the push to "update everything daily at 3:00 AM" is too much. I have a controller with 100+ sites and need to schedule updates to ensure any breakage is fixable via manual intervention up to the point of physically visiting a site. Edit: To clarify this, I'm sure I've been prompted to enable 3:00 AM auto-updates on the newer controller versions and accidentally clicking "yes" would be a huge headache for me.
The "rolling update" was also pretty trashy when I tried it. That was years ago, but I think it simply updated APs sequentially and happily continued if the previous one didn't come back up. How hard is it really to implement a rolling update that stops and waits for intervention if even one device doesn't come back online?
And the UI. I can't even use it without setting my browser to 80% zoom and it gets worse every time they push out an update. Everything is stuck into tiny little scrollable boxes. I have multiple 27" monitors and get stuck scrolling around in a 1" x 2" box that can't display more than 2 or 3 lines of config. Why?
And then TP-Link copied them with Omahda. It's almost funny. I wonder if they even realize they're copying off the dumbest kid in the class. Lol.
Those are all very valid points you raise. I’ve also faced the recoverability problem when something goes awry. I’ve also noticed their ISP line has difficulty being reachable via API when they lose internet connectivity, even if you are local to the network.
That said, is there anything better? Mikrotik, while more configurable, is so much worse and feels straight out of the 90s design-wise.
Beyond those options I’m not aware of anything remotely better.
The design is old fashioned but it really doesn't matter. Imagine if they decided to revamp their UI as a mobile friendly monstrosity with large and padded components everywhere. It would likely be terrible. The compact look they have right now makes it possible to display a lot of information in a small area.
How well does OPNsense deal with bufferbloat in a home networking situation? It appears to implement fq_codel for traffic shaping, but not the newer cake algorithm. Test: https://www.waveform.com/tools/bufferbloat
Debacle with wireguard? Opnsense has wireguard easily available. Also, it's just base wireguard, so you don't have to go through any extra steps of trying to understand / trust other additions on top of it, which is very nice IMO.
I'm referring to the kernel implementation of it, unless you weren't. But I think you likely are, considering it is now in the mainline FreeBSD kernel. But this took more than 2 years after it was mainlined into the Linux kernel[1], and the delay was largely was because of what happened regarding it's initial implementation[2]. That's the debacle I'm talking about.
I've just set up my first OPNsense box (using one of the 4-port 2.5 gig AliExpress tiny PCs recommended by ServeTheHome) and like it a lot.
A few rough edges in the UI, but I got the basic routing functionality running within minutes, and got Wireguard going with the help of a guide from Home Network Guy pretty soon after.
Since then I've bought a second box and will be setting it up soon, too!
I personally use a router box from teklager.se and they use open source schematics and bios which are actually updated. Without having done a comparison I am assuming they are a more expensive option though. It has been small, silent and stable since I bought it pre-installed with OPNSense though, so I am happy with it.
Why would you want BIOS updates? It’s not like firmware degrades over time. Frequency of zero day RCE fixes should not be taken as indication of better security.
Can wholeheartedly recommend OPNsense as an opensource firewall of choice. I run it at home, and we use it on a couple of Dell R210ii servers (4c/8GB RAM/dedicated Intel NIC) to run LAN gaming events ramping up to 1,200 active devices.
Migrated from pfsense because OPNsense actually had a usable API, so we can do things like add people into the Captive Portal programatically from our event check-in system.
I've heard enough good things about OPNsense that I would probably pick it these days for a new installation. But I've been using pfSense happily for enough years that I'm not sure it's worth the hassle to switch, especially since I'm using Netgate hardware.
Has anyone attempted to run OPNsense on Netgate hardware?
The great thing about running OPNSense in a VM on a regular x86 box is it's transferability.
I'm currently upgrading my box to add an NVMe M2 SSD, so I'm moving the VM to a spare box, doing the upgrade, then moving it back. The VM itself knows nothing, no reinstall and reconfigure, minimal downtime. Easy af.
Was using pfSense for a few years until a couple of years ago, upgraded to OPNSense. The plan is to keep OPNSense for much longer than a handful hardware cycles.
If you want to take it even further a fully virtualized Opnsense with Proxmox is amazing. Your router can float between cluster nodes and each VLAN becomes a virtual interface in the hypervisor. What still blows my mind is how I can migrate the instance to a second server and bring the original server down for maintenance without my users noticing a thing.
As long as your Proxmox Cluster is backed by shared storage (ceph, gluster etc.) and HA is configured for your opnSense VM you can just shutdown the Node. Works flawlessly with pfSense (PVE backed by ceph).
I use managed switches for this, L2 units are fine if you let opnsense do all the inter-VLAN routing. All network devices go into the switches which are connected to the servers. As CptKriechstrom mentioned my PON or modem is connected to a switch and is tagged into a VLAN, which enters opnsense through that specific virtual interface.
I use (EdgeMAX) managed switches, so that shouldn't be an issue, I understand the concept now of what I'd need to do but I'm going to have to find a tutorial or something as I wouldn't know how exactly to set that up.
Ok, It's slowly coming together. So I assume I'd then put the virtual WANs of each cluster node on the same VLAN and whichever is currently hosting the router would chat with the PON to give me a WAN link.
This site is the worst mobile web experience I’ve ever had. It automatically scrolls itself both up and down while you are in the middle of the page reading.
I am actually interested in OPNsense but I can’t actually find out if it meets my needs or if I can contribute because the website sucks so much.
NixOS is a Linux distro based on the nix package manager. There is a freebsd package for nix, I am 90% sure. NixOS on FreeBSD seems nonsensical, like saying Ubuntu on FreeBSD.
That's what I was thinking. The only reason that I thought Nix on FreeBSD as even remotely possessive is that, as I remember things, it's possible to run the Nix package manager on non-Nix systems.
You can run nix-the-package-manager on FreeBSD, so you can build stuff. However I think it would take a lot of novel and elegant boilerplate to have nix manage network config on a BSD system.
NixOS is a Linux distribution managed by nix-the-package-manager.
I run OPNsense pretty much stock + unbound + adguard home. Does anyone have a security hardening guide? I feel like I must be missing a few key things to make it self-hosted ready?
I used to run several pfsense routers, tried various forks including opnsense and they were fine but ate quite a bit of memory and didn't have great driver support. If you are like me and just need a router solution for a homelab or small office I would recommend the x86 variant of openwrt. It uses a trickle of memory and CPU and will route anything you ask it too with or without a ton of filters and sniffers. I've never looked back.
I had an issue with OPNsense on a NUC with a USB Ethernet adapter where it would just like stop working. A reboot fixed it every time. I didn’t look much into the exact cause, I went back to DD-WRT because it’s what I know and it just works.
What kind of hardware do you use to install OPNsense? Please don't suggest to buy some old Dell optiplex from ebay. Hardware that can beat any commercial vendor, is better.
Alternatively the Protectli’s https://protectli.com/vault-4-port/ . Protectli gets you support and coreboot, next day Amazon shipping but more expensive and older hardware.
These are great devices for running Opensense bare metal or virtualised in Proxmox. I use Proxmox on a protectli so can run unifi controller and ad guard home in lxc containers with Opensense in a VM. I’d buy a cwwk if buying again, I just didn’t know about them at the time.
I went with an AMD Ryzen 7 5800X and I run OPNsense virtualized for a 25gbit uplink with 10gbit hosts. No traffic inspection or anything fancy however. [1]
Old thinxlients are great for this. I'm running my setup on a Fujitsu Futro. They can be had for <50 euros and you can buy an additional network card if you need more ports. They are quite energy efficient (~5W) and have enough grunt for a home system.
I just got a Protectli Vault FW4B to run it on. I wanted something small and fanless. Didn't even realize Celeron's were still made. Been running it 2 years now with no problems. Even that hardware (8GB RAM / 32GB msata) is overkill for home network.
Protectli was nice to buy from, but I do think there's a lot of similar options.
The jest part about Protectli is the Coreboot and they provide instructions to compile your own, so it even feels open.
That said, fully configured with RAM and storage mine cost roughly £600.
Given electricity prices in the UK, and my requirement for an as-open-as-possible software stack, it was well worth it. If it outlasts the EdgeRouter it replaced (7yrs before power circuitry died) I'll be very happy.
That said, my main concern, as a reply to you pointed out, is that I'll want more than Gigabit in that time; my street is getting 10Gb/10Gb in the next couple of years.
Most of the boxes sold here support 2.5G. No mention of Coreboot, unfortunately,
but could be worth asking if they can install it as they already offer OpnSense or other firewall software installation as a service.
Installing it isn't really enough, you want updates. Protectli provide instructions to compile it yourself, so even if they stopped providing binary releases, you could DIY it; well worth the premium for the security conscious in my opinion.
I own a DEC3840 [1] appliance and I'm happy with it so far. I was also on the search for a proper commercial vendor after running it in a VM for years.
I run mine on HP’s version of an optiplex with a 2nd hand pcie 1Gb quad nic, but if you wanted nicer “real” hardware with a proven track record, an HP microserver gen10 or gen10+ can be had on eBay for very little. A gen10+ should even come with iLo, which is nice.
I use an old mac mini which has onboard gigabit ethernet (lan side) and an apple thunderbolt to gigabit ethernet dongle for wan. I am using 900/300 fibre and it has been rock solid.
No need to buy expensive stuff for this task.
This really isn't viable in somewhere like the UK.
At our current electricity prices, if that machine ran at 100W, you'd be spending the equivalent of what you paid for it EVERY month in electricity.
I've always seen Americans, who typically have more space and cheaper energy and fuel suggest people grab 1U servers for $40 etc, and respond just like you did when people ask them _not_ to.
My car also does 50mpg, and I still pay more pay mile for fuel.
In Europe, we need to spend more to buy efficient tech because the running costs require it.
It's perfectly viable in the UK. The Optiplex 7050 is available for under a hundred pounds and the PSU tops out at 65W. It idles under eight watts. This is comparable with any other router capable of handling gigabit traffic.
I think the misunderstanding is the assumption about form factor. The 7050 series is available in a case the size of a paperback book.
I'm willing to accept your point if you or they, can confirm that they are in fact talking about an optiplex that idles at 8W, which I doubt, for 43 dollars.
Despite DELLs willingness to slap the same series number on everything from an ATX tower to a "paperback book", they are far from the same thing.
I'm willing to accept I may be wrong, so let's see.
First page, the one on the right. I don't see why you'd doubt an 8-watt idle, since it's basically a laptop processor and chipset. Please note that these models are available with both 35W and 65W TDP processors; I'm talking about the (generally cheaper) 35W models, of course.
I doubt the original poster is talking about the small "book" sides micro form factor at ~$40.
I expect that one to cost at least double, if not more (a quick eBay search outs the micro form factor at closer to a £200 average), and the tower form factor, with it's 240W PSU is likely to be drawing much more than ~8W in a typical configuration.
My point stands; if you want to spend £35 on an x86 machine, youll pay in electricity costs, spend ~£200+ and overall youll likely save money overall if it lasts a decent period.
It probably doesn't run at 100W, although I'd be quite curious to know whether someone actually measured the consumption, since I have the HP version of this.
The reason why I don't expect it to draw as much is because I have an 8-core xeon with 2x10k RPM + 4*7200 RPM drives, a dedicated raid card, integrated BMC, and it reports ~100W power draw when booting up. When sitting around doing nothing but with the drives spinning, it reports a draw of about 80W.
edit: The BMC alone draws 11W, judging by the reported power consumption when the server is off, but plugged in.
The main issue I see with that option is space and power, the micro-form factor (NUC-sized) clone systems that people are discussing in here are quite nice for that. Functionally an old Dell tower should be fine.
Sure, but those tend to be much more expensive, so depending on the actual cost of electricity in your area, it may take a long time for you to break even.
However, if you want to have a really small appliance-like pc, then yeah, the NUC-sized ones are much better.
The low-power AMD based systems everyone recommended a couple years ago were indeed great, until the ebay resellers realized what was going on and started slapping "opnsense pfsense AES-NI" etc on their listings and doubled their prices. The systems went from selling at $40-50 to over $100.
Also, the AMD processor in those systems is getting extremely long in the tooth.
Pfsense is, in turn, a fork of m0n0wall. It's also important to note that pretty much all of the functionality of all aforementioned firewalls is part of FreeBSD or from packages installed behind the scenes. The configuration UX is the most visible differentiator but they also have different strategies when it comes to pushing updates and testing/validation.
But the actual heavy lifting of these router/firewall systems is done by code that ships with FreeBSD.
[1] https://homenetworkguy.com/how-to/set-up-a-fully-functioning...