IPsec troubleshooting: why your VPN is not connecting, phase by phase
Most IPsec failures are a mismatch between two peers. Work through the phases in order, read the right log, and the cause usually shows itself within minutes.
The short answer
IPsec troubleshooting goes fastest when you work through the phases in order and stop at the first one that fails: can the peers reach each other, does the IKE SA come up, does the child SA come up, and does traffic actually flow through it. Each phase has its own typical errors, and the log on the responder usually says more than the initiator's.
The examples use the shared test network: the IPsec server is 203.0.113.10 with 10.10.0.0/16 behind it, the branch is 198.51.100.20 with LAN 192.168.10.0/24.
IPsec troubleshooting commands for each platform
Run these on both peers. They show whether an IKE SA and child SAs exist, and the debug output says why a negotiation failed.
strongSwan (swanctl)
swanctl --list-sas # IKE and child SAs, bytes, rekey times
swanctl --list-conns # what is configured
swanctl --initiate --child branch-lan # bring it up and print the negotiation
journalctl -u strongswan -f # live log
ip -s xfrm state # kernel SAs with packet counters
tcpdump -ni eth0 "host 198.51.100.20 and \
(udp port 500 or udp port 4500 or ip proto 50)"
Cisco IOS XE
show crypto ikev2 sa detailed
show crypto ipsec sa peer 198.51.100.20
show crypto session remote 198.51.100.20 detail
debug crypto ikev2
debug crypto ipsec
undebug all
FortiGate (FortiOS 7.4)
get vpn ipsec tunnel summary
diagnose vpn ike gateway list name
diagnose vpn tunnel list name
diagnose vpn ike log filter rem-addr4 203.0.113.10
diagnose debug application ike -1
diagnose debug enable
diagnose debug disable # when done
Palo Alto (PAN-OS 11)
show vpn ike-sa gateway
show vpn ipsec-sa tunnel
show vpn flow
test vpn ike-sa gateway
test vpn ipsec-sa tunnel
less mp-log ikemgr.log
MikroTik (RouterOS 7)
/ip ipsec active-peers print
/ip ipsec installed-sa print
/ip ipsec policy print
/system logging add topics=ipsec,!packet
/log print where topics~"ipsec"
Phase 0: reachability
If the log on one side shows nothing at all when the other side initiates, the packets are not arriving. IKE needs UDP 500; once NAT is detected both peers switch to UDP 4500 and ESP travels inside it (NAT-T). Without NAT, data uses ESP, IP protocol 50, which many firewalls don't allow unless you name it explicitly.
- Capture on the receiving side (tcpdump,
diagnose sniffer packet, the PAN-OS packet capture). Packets arriving but no log entry: a local firewall drops them, or the remote address doesn't match any configured peer. - Check network ACLs and any firewall in front of either peer.
- IKE_AUTH with certificates can exceed 1500 bytes. If IKE_SA_INIT succeeds and IKE_AUTH times out, fragments are being dropped; enable IKEv2 fragmentation (RFC 7383) on both sides if supported.
- A peer behind NAT must use the NAT's public address as the remote address on the other side, and its ID usually stays its private address unless you set it explicitly.
Phase 1: IKE SA failures
NO_PROPOSAL_CHOSEN
The peers share no IKE proposal: encryption, integrity/PRF or DH group differ. Line up the exact algorithms on both sides, for example AES-256, SHA-256 and group 20 (ECP-384). Watch for a peer that defaults to group 2 or 14 when you expected 20, and for a GCM proposal on one side while the other only offers CBC with an HMAC. Also check the IKE version: an IKEv1 peer and an IKEv2 peer never agree.
AUTHENTICATION_FAILED
The proposals matched, but authentication didn't. With PSKs, strongSwan logs a line saying the shared key was tried but the MAC mismatched: the keys differ. Copy-paste errors and trailing spaces are common, so set the key again on both sides. With certificates, check the chain, the validity dates and that the CA is trusted on the verifying side.
Identity mismatch
Each side sends an ID (an IP address, FQDN or certificate DN) and the other side uses it to pick the configuration and the key. If 203.0.113.10 sits behind 1:1 NAT and sends its private address as its ID, the branch finds no match. On strongSwan this shows as no matching peer config found; on other platforms it often looks like a generic authentication failure. Set local and remote IDs explicitly on both peers.
Phase 2: child SA failures
TS_UNACCEPTABLE: traffic selector mismatch
The IKE SA is up but no child SA. The selectors must mirror each other exactly: 10.10.0.0/16 ↔ 192.168.10.0/24 on the server must be 192.168.10.0/24 ↔ 10.10.0.0/16 on the branch. A /24 on one side and a /16 on the other, a forgotten second subnet, or a host address instead of a network all cause it. Some vendors create one child SA per subnet pair while others combine them, which also leads to mismatches; route-based tunnels with 0.0.0.0/0 selectors avoid this class of problem.
PFS mismatch
In IKEv2 the first child SA is created inside IKE_AUTH without its own key exchange, so a PFS mismatch often does not show at first. The tunnel works for an hour, then fails at the first child SA rekey with NO_PROPOSAL_CHOSEN. If a tunnel drops at exactly the child SA lifetime, compare the PFS group on both sides.
A tunnel that fails at a regular interval (every hour, every eight hours) is almost always a rekey problem: compare PFS settings and the proposals used for rekeying, not just the initial negotiation.
Tunnel up, but no traffic
Both SAs are established, yet pings fail. Look at the SA counters: are packets being encrypted on one side and decrypted on the other?
- Routing. Hosts in 10.10.0.0/16 need a route to 192.168.10.0/24 via the IPsec gateway, and the branch hosts the reverse. In a route-based setup, the route must point at the tunnel interface.
- NAT exemption. If the gateway also NATs to the internet, traffic for the remote subnet must be excluded from that NAT, or its source address no longer matches the selector and it leaves unencrypted. The usual sign: encrypt counters stay at zero.
- Firewall policy. Many firewalls need an explicit policy between the LAN and the tunnel in each direction. FortiGate and PAN-OS will not pass tunnel traffic without one.
- Testing from the wrong address. A ping from the gateway's public IP doesn't match the selectors. Test from a host inside the protected subnets.
MTU and TCP MSS clamping
Pings work, small requests work, but file transfers, web pages and SSH sessions stall. That is an MTU problem: encryption adds overhead, full-size packets no longer fit, and the ICMP messages that would make path MTU discovery work are often filtered.
The overhead for a tunnel-mode ESP packet:
| Part | AES-256-GCM | AES-256-CBC + SHA-256 |
|---|---|---|
| New outer IP header | 20 | 20 |
| UDP header (NAT-T only) | 8 | 8 |
| ESP header (SPI, sequence) | 8 | 8 |
| IV | 8 | 16 |
| Padding + pad length + next header | 2 to 5 | 2 to 17 |
| Integrity check value | 16 | 16 |
| Worst case | 65 bytes | 85 bytes |
On a 1500-byte path the inner packet can be at most about 1415 bytes in the worst case, and less again if the path runs over PPPoE (1492). A tunnel MTU of 1400 leaves room for all of these. TCP MSS is the MTU minus 20 bytes of IP header and 20 bytes of TCP header, so 1400 − 40 = 1360. Clamping rewrites the MSS in TCP SYN and SYN-ACK packets crossing the gateway, so both ends send segments that fit without relying on fragmentation or ICMP.
On Cisco IOS XE, use ip tcp adjust-mss 1360 on the tunnel or inside interface. On Linux with nftables, in the forward chain:
tcp flags syn / syn,rst tcp option maxseg size set 1360
FortiGate, PAN-OS and MikroTik all have an MSS adjustment (in the firewall policy, the interface settings and a mangle rule respectively); check your version's documentation for the exact field. Clamp on at least one gateway in the path, applied to traffic in both directions.
Flapping tunnels
- DPD too aggressive. Dead peer detection with a short interval and few retries tears down a tunnel on a few seconds of packet loss. Use an interval of around 30 seconds, and if both peers are set to restart on DPD, consider letting only the initiator restart.
- Lifetimes. In IKEv2 lifetimes are not negotiated; each side rekeys on its own timer. Matching them (for example 28800 s for the IKE SA and 3600 s for child SAs) still makes behaviour predictable. In IKEv1, mismatched lifetimes can cause failures on some platforms.
- Rekey collisions. If both peers start a rekey at the same moment, IKEv2 resolves the collision, but some implementations end up with duplicate or orphaned SAs. Randomised rekey timing (strongSwan adds jitter by default) and making one side the clear initiator help.
- Duplicate tunnels. Two IKE SAs to the same peer, each with child SAs, can cause traffic to use one SA while the other side expects the other. Check
--list-sasor its equivalent for duplicates.
Decision table
| Symptom | Likely cause | Check first |
|---|---|---|
| Nothing in the responder's log | UDP 500/4500 blocked, wrong peer address | Packet capture, security groups |
| IKE_SA_INIT ok, IKE_AUTH times out | Fragmented IKE packets dropped | IKEv2 fragmentation, path MTU |
| NO_PROPOSAL_CHOSEN during setup | IKE proposals or IKE version differ | Encryption, integrity, DH group on both sides |
| AUTHENTICATION_FAILED | Wrong PSK, ID mismatch, untrusted certificate | Local and remote IDs, key, CA |
| TS_UNACCEPTABLE | Subnets don't mirror each other | Local and remote subnets on both sides |
| Drops at the child SA lifetime | PFS group mismatch | PFS setting on both sides |
| SAs up, encrypt counter at zero | Routing or missing NAT exemption | Routes, NAT rules |
| SAs up, decrypt counter rises, no replies | Firewall policy or return route on the far side | Policies, routes behind the peer |
| Ping works, TCP stalls | MTU | MSS clamping to 1360 |
| Tunnel up and down every few minutes | DPD, duplicate SAs, rekey collisions | DPD timers, list of SAs |
Getting ahead of it
Most of these failures are configuration mismatches between two devices configured by hand. The IPsec VPN server setup guide and the per-vendor examples (Cisco, FortiGate, Palo Alto, MikroTik, strongSwan) all use the same values, so you can compare line by line.
On TunnelYard, guided tunnel setup with presets per remote device and peer config export reduce the chance of mismatched settings, and live tunnel status, traffic graphs and per-tunnel logs show which phase failed. Every plan sends an email alert when a tunnel has been down for 2 minutes; Business adds webhooks, and Enterprise adds Slack and Teams. See features and use cases.
Bottom line
Work phase by phase, read the responder's log, and compare settings on both sides rather than trusting either one. When an IPsec VPN is not connecting, the cause is almost always one of the ten rows in the table above.