PPPOE and Path MTU Discovery

If you find that your PPPOE interface has problems with certain sites, it may be due to the general state of disarray of path MTU (Maximum Transmission Unit) discovery on the Internet.

PPPOE interfaces generally have an MTU of 1492 bytes, whereas the straight Ethernet interface that they sit on have an MTU of 1500 bytes.

When the stack initiates a TCP connection, it advertises the largest MTU of all its interfaces, as per RFC 1193, via the TCP mss option. This is done because, although the outgoing packets are going out on the PPPOE interface, there's a slight chance that packets coming back from the peer will actually come in on another interface, depending on the routing that occurs between you and the peer.

If a packet that's larger than 1492 comes back on the PPPOE interface, the router needs to handle it via fragmentation or path MTU discovery.

If the peer (in our example) sends a packet back to us with an MTU of 1500 (which we advertised as per the above) that happens to be routed back to our PPPOE interface of MTU 1492, the router before the PPPOE has two options:

• If the peer isn't using path MTU discovery, the router can fragment the packet into two smaller ones and send them on their way. This results in more overhead, but things should continue transparently.
• If the peer is using path MTU discovery in an effort to avoid the overhead of fragmentation, it has to set the DF (don't fragment bit) in the IP header. In doing this, it expects the router to drop the packet and send back an “ICMP fragmentation required” packet containing the new MTU (1492 in this example). This is where the problem often arises.

  Some routers appear not to generate the “ICMP fragmentation required” packet. More often, these packets are filtered out by intermediate firewalls. In either case, the peer has no indication that anything is wrong, and continues to generate packets of MTU 1500 with the DF bit set, which in turn are dropped by an intermediate router.

What can you do about this?

Setting the mss_ifmtu option on makes the stack use the MTU of the outgoing interface in the initial advertisement, rather than the maximum of all its interfaces. In accordance with RFC 1193, this option is off by default.

You can query and set the mss_ifmtu option by using the sysctl utility:

# sysctl net.inet.tcp.mss_ifmtu
net.inet.tcp.mss_ifmtu = 0
# sysctl -w net.inet.tcp.mss_ifmtu=1
net.inet.tcp.mss_ifmtu: 0 -> 1

For more information about sysctl, see the Utilities Reference.