Routing Between Sites: Fiber, VPN, and the UniFi Reality

The routing design for a second campus is four lines. The link it runs over is where the real decisions are: what the ISP hands you, what happens when it fails, and how much of the district's traffic you want crossing it.

This article assumes a UniFi stack at both ends, because that is the environment in question, and it calls out where the controller's abstractions help and where they hide something you need to know.

What the ISP is actually selling

Two delivery models, and they are not interchangeable:

Model	What you get	Who routes	Your job
Layer 2 (fiber, E-Line)	A transparent pipe between two ports	You, at both ends	Provide routers, addresses, policy
Layer 3 (IP handoff)	A routed link with a next hop	The ISP	Accept their prefix, route to their gateway

With a layer 2 handoff, the design in the previous article applies exactly: transit /30, static routes both ways, your firewall in the path.

With a layer 3 handoff, the ISP owns the transit and usually gives you a small block. Your routes point at their next hop, and any change on their side happens in a portal you do not control. Document the circuit ID, the handoff address, and the escalation path in the same place as the route table.

Option 1: fiber with a router at each end

Best case. You control both ends, latency is predictable, and the link is symmetric.

• Transit /30 between the two routers.
• Static summary at the core, default at the campus.
• Firewall policy applied at both ends, with the same zone model as the main campus.
• A second fiber on a different path if the budget allows, with a floating static behind it.

Option 2: IPsec over the internet

UniFi's site-to-site VPN is a competent IPsec implementation and is the right answer when fiber is not available or is too expensive for a small campus.

Design notes that matter:

• MTU. IPsec adds overhead. If the tunnel MTU is wrong, small packets (ping, DNS) work and large ones (TLS handshakes, SMB) stall. Set the tunnel MTU explicitly and clamp MSS on the routers.
• Asymmetric routing. If the fiber is primary and the VPN is backup, traffic must return the way it left. A floating static on one side without a matching route on the other produces one-way traffic that looks like a firewall problem.
• Policy both ends. The tunnel carries packets; the firewall still decides. A VPN that permits everything is a flat network with extra latency.
• Keepalives. Enable them, and alert on tunnel state rather than on a ping to a single host.

Option 3: point-to-point wireless

Viable for a short hop with clear line of sight, and a genuine option for a temporary campus or a building across a field. Treat it as a fiber link with worse weather: same routing design, plus a monitoring plan for signal and modulation, and a documented fallback to the VPN.

Where UniFi hides things

The UniFi controller is good at making the common case easy, and the common case is a single site. For multi-site routing, know these:

• Static routes live per site, in the site's own settings. A route added to the wrong site in the site switcher does nothing, silently.
• Site-to-site VPN is configured per site pair, and the "remote networks" list is what actually gets routed. If the new campus's 10.50.0.0/16 is missing from that list, the tunnel is up and nothing crosses it.
• The gateway's routing table is Linux. Policy-based routing exists in newer builds as traffic routes, but it is not a substitute for a correct table.
• NAT is on by default. Site-to-site traffic that traverses a WAN interface may be NATed unless you add an exception. NAT between sites breaks return routing and any protocol that embeds addresses.

Worked example

New campus 10.50.0.0/16, fiber primary, IPsec backup.

District core:

Prefix	Next hop	Distance	Note
10.50.0.0/16	10.0.0.10	1	Fiber, transit /30
10.50.0.0/16	10.0.0.22	250	VPN tunnel address

Campus router:

Prefix	Next hop	Distance	Note
0.0.0.0/0	10.0.0.9	1	Fiber
0.0.0.0/0	10.0.0.21	250	VPN backup
10.0.0.0/8	10.0.0.9	1	District block

Four lines at the core and campus together, plus a tunnel. The failure test is simple: unplug the fiber, confirm the VPN route installs, confirm a sourced ping from a campus host to 10.0.0.3 succeeds, then plug the fiber back in and confirm the primary returns.

What breaks in production

• The tunnel is up, the routes are not. VPN state is green; the remote network list is missing the prefix. Nothing crosses.
• MTU black hole. Ping succeeds, HTTPS hangs. Classic, and always the tunnel MTU.
• NAT applied to site traffic. The far side sees the tunnel address, not the real host, and return traffic goes to the wrong place.
• Both paths active with different latency, because two static routes share a distance. Sessions bounce between paths and applications time out.
• No monitoring on the backup. The first time you learn the VPN is broken is the first time the fiber fails.

Checklist

1. The handoff type (L2 or L3) is documented, with addresses and circuit IDs.
2. Transit addressing is outside both site blocks.
3. The backup path uses a different distance and has been tested end to end.
4. Tunnel MTU is set, and MSS clamping is configured if needed.
5. NAT is not applied between sites.
6. Tunnel and fiber state are both monitored, with alerts that reach a person.

Next

Design and links are in place. Continue with When Routes Break, the verification method that finds the one line that is wrong.