IPsec is the protocol behind most site-to-site tunnels, including UniFi's. It has a reputation for complexity that comes from its vocabulary, not its behavior: two phases, a set of proposals, and a rule about which traffic is interesting.
Learn five things and the rest is configuration.
The two phases
Phase 1 (IKE SA) builds a authenticated, encrypted channel between the two gateways. It answers: are you who you say, and can we agree on crypto?
Phase 2 (child SA) negotiates the actual data tunnel inside that channel. It answers: which traffic, with which algorithms, for how long?
| Phase | Protects | Fails when |
|---|---|---|
| 1 | The negotiation itself | PSK/cert mismatch, proposal mismatch, unreachable peer |
| 2 | Your data | Traffic selectors don't match, PFS mismatch, lifetime mismatch |
In IKEv2, phase 1 is IKE_SA_INIT + IKE_AUTH, and phase 2 is CREATE_CHILD_SA. The names change; the
shape does not.
The five settings that must match
| Setting | Example | Mismatch symptom |
|---|---|---|
| Encryption | AES-256 | Phase 1 never completes |
| Integrity | SHA-256 | Phase 1 never completes |
| DH group | 14 (2048-bit) | Phase 1 fails after proposal exchange |
| Authentication | PSK or certificate | Phase 1 fails with "no proposal chosen" or auth error |
| Traffic selectors | 10.50.0.0/16 ↔ 10.0.0.0/8 |
Phase 2 up, no traffic passes |
The first four are negotiation; the fifth is routing. A tunnel can be "up" and carry nothing because the selectors do not include the subnet you care about.
Traffic selectors are not a firewall
Selectors decide which packets enter the tunnel. The firewall decides whether they are allowed after
they arrive. A selector of 0.0.0.0/0 means everything is tunneled, not everything is permitted.
ESP, and why AH is gone
ESP encrypts and authenticates the payload. AH authenticates but does not encrypt, and it breaks under NAT, which is why almost nobody uses it.
| Mode | What it wraps | Use |
|---|---|---|
| Transport | Payload only, original IP header | Host-to-host, rare |
| Tunnel | Entire packet, new IP header | Site-to-site, standard |
NAT traversal
Two gateways behind NAT cannot exchange ESP directly. NAT-T encapsulates ESP in UDP 4500, and IKE moves to UDP 4500 after the first exchange. If UDP 4500 is blocked, the tunnel establishes and then dies as soon as data starts — a classic "phase 1 up, phase 2 flapping" symptom.
Worked example: district to new campus
| Setting | District | Campus |
|---|---|---|
| Peer | 198.51.100.5 |
203.0.113.20 |
| Auth | PSK (rotate quarterly) | same |
| Phase 1 | AES-256 / SHA-256 / DH14 / 8 h | same |
| Phase 2 | AES-256 / SHA-256 / PFS14 / 1 h | same |
| Selectors | 10.0.0.0/8 → 10.50.0.0/16 |
10.50.0.0/16 → 10.0.0.0/8 |
The selectors are mirrors. If one side lists 10.50.0.0/16 and the other lists only
10.50.40.0/21, the tunnel comes up and half the campus is unreachable — the half that is not in the
narrower selector.
What breaks in production
- Proposal mismatch, usually DH group or lifetime. The log says "no proposal chosen."
- PSK mismatch with a trailing space, which looks identical to a wrong password.
- Phase 2 up, no traffic, because the selectors are narrower than the subnets.
- UDP 4500 blocked, so the tunnel dies under load rather than at setup.
- MTU not set, so ping works and large transfers stall.
- Two tunnels to the same peer with overlapping selectors, which is undefined behavior in practice.
Checklist
- Proposals are documented on both ends, including lifetimes.
- Authentication material is rotated and stored outside the config.
- Traffic selectors cover every subnet that must cross, in both directions.
- UDP 500 and 4500 are permitted end to end.
- Tunnel MTU and MSS are set explicitly.
- Phase 1 and phase 2 state are both monitored, not just "tunnel up."
Next
IPsec is the enterprise standard. Continue with WireGuard in Practice for the lighter alternative.