Single device networks

Generally, when you sign a contract with an ISP, he will provide you with an all-in-one, cost-effective router that has everything built in. It is a Wi-Fi access point, router with integrated switch, and sometimes, it has an additional built-in battery backup or modem/converter if needed.

This setup is cheaper to produce in bulk orders but also easy to maintain—a single device is used as a point of failure, and it is easy to flash/update and replace when it fails and has standardized firmware to monitor and interact with by ISP support team.

Unfortunately, all-in-one box has some disadvantages:

1. It does come preconfigured with default DHCP and WiFi configuration and rarely has fields to configure specific network settings (like local DNS). Let’s be honest that this is not an issue for customers who want quick setup, don’t use advanced network isolation and VLANs, and don’t care about DNS. One box on the shelf, and you’re ready to go.
2. Firmware functionality is intentionally limited. Those all-in-one devices are often preloaded with ISP-customized firmware that has advanced functions hidden from the user. This is intentional to limit the possibility of misconfiguring network setup and then requesting ISP support to solve that issue. Again, it’s simpler for technicians to navigate through a unified, simplified environment to configure local networks.
3. Those devices feel cheap. With unified UI and proprietary hardware, ISPs can cut costs by using only specific components, ordering them in bulk, and producing the same device in thousands of units cheaper than a good, branded router. It’s also cost-effective for consumers, as often those devices are leased for free or with a small fee instead of paying for full-fledged and advanced hardware that only a fraction of customers will need.
4. Unified network. Most all-in-one devices use a flat network where all devices can communicate with each other by default. DHCP provides only dynamic IP assignments and IP pool assigned is quite large. There is also only single network segment available, most probably with IP address of 192.168.0.x or 192.168.1.x and subnet mask of /24. And it’s OK for most customers.

While this setup is effective for both ISP and consumer, my network needs are way far bigger than this.

Network isolation

My network configuration is slightly different. I do have 4 network zones:

1. Self-hosted server in DMZ server. This zone is isolated and has access from VPN (jump host / bastion host) to my website hosting server. Also this network does not have access to local IP pools.
2. Private wired network. At this network I have all my PCs, TVs, printers that are “wired” by standard Ethernet cable connection.
3. Isolated wired remote work network. A separate network where I connect my employer hardware so I can have both private and work networks separated.
4. The WiFi network is where everything else lives—mobile phones, tablets, IoT, Alexas, etc. If it has WiFi, it’s on this network.

The Ubiquiti EdgeRouter enforces configuration. Isolation is done by port, so each zone/network is represented by a specific Ethernet port on the router. It simplifies the configuration and monitoring of networks.

The first port of my router is dedicated to internet access, one that is connected to my ISP’s device with ISP-provided DHCP setup. Those dedicated ports allow me to apply different QoS and firewall rules per specific network.

My local server has DNS configured, NGINX proxy, monitoring, alerting software & communication (Prometheus, Grafana, Mattermost self-hosted instance) installed, so in case of Internet access failing, I still can have my printers working, custom-made IoT is up and running, and I can checkGrafana (visualizing my custom Go-based network monitoring tool) what’s going on with my Internet access. I can also use the AChat client as my AI model I’m using is running locally, too.

This works great so far as I had some serious issues with Xfinity and decided to change ISP to AT&T Fiber. While the new internet provider works excellent (it’s a communication company by origin, not an cable TV operator) I had also zero issues with switching provider. What I was required to do is to unpin the Ethernet cable from Xfinity gateway and plugged in the AT&T Ethernet cable to the same port. Ubiquiti router got new IP lease while my local network did not changed at all. Zero re-configuration from my side. The difference in Internet performance was noticeable immediately, and it was also visible on Grafana metrics checking latency.

(Current network latency / TTFB from web server)