Definition
IPSec (Internet Protocol Security) is not a single protocol but a framework of related protocols that operate at the network layer to authenticate and encrypt IP packets. Its main components are AH (Authentication Header), which authenticates packets without encrypting them; ESP (Encapsulating Security Payload), which authenticates and optionally encrypts; and IKE/IKEv2, which negotiates the keys and parameters. ESP in tunnel mode is the configuration used for VPNs.
Modes of Operation
Transport mode encrypts only the payload of the packet, leaving the original IP header in place — used for host-to-host security. Tunnel mode encapsulates the entire original packet inside a new IP packet, hiding the inner addressing — used for VPN tunnels. The vast majority of VPN deployments use IPSec in tunnel mode with ESP, with IKEv2 handling key exchange.
Strengths and Weaknesses
IPSec is mature, well-audited, and supported natively by virtually every operating system and major networking vendor. It is the default for site-to-site VPNs connecting offices and clouds. Weaknesses include configuration complexity, fixed UDP ports that are easy to block, and a long history of subtle interoperability issues between different vendor implementations. WireGuard has emerged as a simpler alternative, though IPSec remains entrenched in enterprise deployments.