My x86 router journey

  • Thread starter Thread starter varkey
  • Start date Start date
  • Replies Replies 125
  • Views Views 21,790
I had recently acquired a mini pc with 4 intel NICs to be used as an x86 router / firewall. More details around the process of acquiring it is in another thread - Important - All international "Gift" packages now liable for customs duty @ 41.2% | Online Shopping

So I got the mSATA SSD module (and a keyboard 😛) and I was trying to set up things over the last 1-2 days. At first I tried OPNsense, but to be honest, I didn't find it whole thing as refined or polished as pfSense. I couldn't even get my BSNL and Kerala Vision connections up for some reason. Maybe it is bit of unique or non-supported, I do not know, but the same config works on first try in OpenWRT without issues.

The Kerala Vision and BSNL connections come over the same fiber, goes into my Huawei ONT in transparent bridge mode (well, it doesn't have any other mode 😉) and then into the router, on the same port. Then VLANs are setup for each (1830 for BSNL Voice, 702 for BSNL Internet and 140 for Kerala Vision). On OPNsense, the the voice interface came up fine cause it was just a static IP config and didn't need mac spoofing. However for BSNL and Kerala Vision, as they do mac binding, I needed to spoof the mac.

But this doesn't appear to be supported natively OPNsense, and my PPPoE interfaces never even came up. Maybe once or twice, the BSNL interface came up but got disconnected in a short time. The logs show PPPoE timeout and retrying. 🤷‍♂️
I tried what I can and gave up.

Now on pfSense, the UI is more clean and polished. The layout also seemed better than OPNsense. I did the same config on pfSense and here they were pretty direct and mac spoofing on VLAN interfaces is not supported and must be set on the parent interface (this is the feature request Feature #1337: VLANs with different MAC address than parent interface - pfSense - pfSense bugtracker). But the issue is, I need two different macs when talking to Kerala Vision and BSNL (as that's what is originally registered on their end and I didn't want to get it reset).

I anyway I found a work-around in pfSense forum (MAC address spoofing on VLAN's and impressions from a second-try user) where you need to create bridge interface with just the VLAN interface as a member, and on the bridge interface you can spoof the mac. So BSNL Internet was setup on the regular VLAN interface and Kerala Vision on the bridge interface.

Finally, I was able to get my WAN interfaces up, now the next problem. The unbound DNS resolver kept on getting restarted. Log entries like below were being logged repeatedly. I searched online but couldn't find a solution that worked for me. Every 4 seconds or so the resolver was getting restarted which breaks everything as DNS fails.

Code: 
Mar 22 13:51:25 mercury unbound: [54761:0] info: start of service (unbound 1.9.1).

Anyway I found the problem, it was the DHCP client for the BSNL IPv6 which was constantly getting a no addresses response. And for each such response, unbound gets restarted, I have no clue why 🤷‍♂️ Mostly related to this bug Bug #5413: Incorrect Handling of Unbound Resolver [service restarts, cache loss, DNS service interruption] - pfSense - pfSense bugtracker where unbound is restarted for each change rather than using unbound-control for such cases.

Code: 
Mar 22 12:55:33 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:55:41 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:55:47 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:55:54 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:56:00 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:56:06 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:56:13 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:56:19 mercury dhcp6c[97095]: status code for NA-0: no addresses
Mar 22 12:56:25 mercury dhcp6c[97095]: status code for NA-0: no addresses

Setting this to true fixed the problem, as it is probably BSNL not responding.

pfsense-ipv6-dhcp.webp

So far things are working, I am still playing with how the fail over and load balancing should work. Overall, OpenWRT seems more mature for my use cases? I mean so many issues in getting something like this working which I could get it running in OpenWRT with no such challenges.

dashboard-pfsense-edited.webp


@C3PO @vishalrao @vignesh_venkatesan @achaudhary997 @JB700
 
Click to expand...
Until FreeBSD includes WG in the kernel we won't see WG in pfSense. Anybody wanting to use WG should use a VPN provider for eg. Mullvad, PIA, NordVPN among several others that have implemented it via their apps and in the servers. But a large number are in the beta testing phase. So expect a roll out only a 1-4 months from now. I am not too enthusiastic about beta software, and remember WG is still in testing phase, when used on a router, I don't mind a desktop application through. For a router I prefer the more mature albeit slower OpenVPN. That said I removed the OVPN configuration (I had a ExpressVPN subs, but now have others) from my pfSense box and only run VPN through a W10 app and when required for e.g. whilst torrenting. Gives me better control over what goes through a VPN and what does not. Also, if privacy is paramount then avoid WG because from what I read it doesn't work without logs making it unsuitable for VPN services that tout privacy as one of their USPs.
 
I switched to OpenWRT itself, pfSense wasn't giving me that good feeling. It certainly has a lot of features but lack of WireGuard, ZeroTier support made me lean towards OpenWRT. OpnSense has support for these, but I didn't like OpnSense much either.

With WireGuard, I get to max out to the link to around 250-300 Mbps over the IPv6 tunnel (I think it's limited by the backhaul capacity of my LCO or the last mile PON port limit)





If anyone's interested the x86 EFI image of OpenWRT is available here -- x86: add EFI images and make iso images EFI bootable by uxgood · Pull Request #1968 · openwrt/openwrt
 

Openwrt is good, but pfsense is way better if you have enough horsepower. Snort/pfblockerng/BGP routing are awesome in pfsense.

Not sure what is the deal with wire guard. Ipsec with aesni works great.
 
Last edited:
WireGuard is more efficient, so for a given hardware WireGuard gives a higher throughput. Also, it's extremely easy to set it up.

For my current use cases I felt OpenWRT works better. Pfsense sure is great, but I guess I don't really need all the features in pfsense but also require some which aren't available get. Maybe I'd re-evaluate after a while. 😅
 
can anyone explain what is WireGuard, OpenWRT, pfsense? and for what it is used?

Cause I am new to this tech stack and I feel excited to learn 😬
 
1. WireGuard is a VPN Protocol that is very lightweight and performs very well even on weak devices unlike much heavier OpenVPN. This is why many VPN Providers are offering WireGuard as an option since its usually much faster and consumes less battery on battery powered devices. WireGuard is also less prone to disconnections compared to OpenVPN. SBCs like in Raspberry Pi can give very good speeds on WireGuard whereas OpenVPN will struggle with encryption/decryption especially without AES-NI (found on newer CPUs).

2. OpenWRT is a Linux based firewall/routing software that is much more customizable compared to default consumer router software.
It's not as fancy as PFSense but it is very lightweight and can be installed on many regular routers. It unlocks many stuff in normal routers that is typically found in much more expensive routers such as IPS/IDS to look for harmful packets/stop attacks on network (though this is usually quite hard on weak CPUs), much higher control on WiFi like changing power to high levels not usually allowed in normal routers, fancier firewall rules, collecting network stats, VPN Support which might be disabled on stock software etc.

3. PFSense is FreeBSD based firewall/routing software, it is more suited towards x86/x64 CPUs unlike OpenWRT and has even more customizablity like newer Suricata IPS/IDS with much fancier graphs compared to OpenWRT's snort. While OpenWRT is suited towards really low end routers, PFSense is more suited towards full fledged computers. PFSense is most suited for things like OpenVPN,IDS/IPS because its usually installed on faster computers. Though there are ARM based routers/firewall which run on PFSense like SG-1100, these are typically suited for small networks and you can't run things like Suricata, OpenVPN without maxing the CPU.

Sources:

en.wikipedia.org

pfSense - Wikipedia

en.wikipedia.org en.wikipedia.org

WireGuard - Wikipedia

en.wikipedia.org en.wikipedia.org

OpenWrt - Wikipedia

en.wikipedia.org en.wikipedia.org

I would suggest going through these videos/channels as well:


Source


Source


Source


Source
 
Afaik Nope, OpenWRT can't be installed on ONTs (there may be some weird ONT which might support ONTs though but I OpenWRT doesn't support modifying Optical Module settings as far as I know). I would assume that blasting ONT at full power will have some effects such as overheating, noise,power limits etc. Like if WiFi power is increased way too much, devices close to AP will see degradation in speed usually.
 
Honestly I wonder to what extent is wireguard better than OpenVPN with AES-NI instructions. Is wireguard's cipher faster than hardware AES?
Tho wg is lightweight, that's for sure.
 

Top