Table of Contents

Network - Troubleshooting the routing table

To ensure that the routing table has been correctly configured.

Background

The routing table contains information used by the network stack to decide where each packet should be sent next. This decision may result in the packet being delivered locally, or forwarded to another machine, or rejected as unroutable. For packets that are forwarded the routing table is concerned only with selection of the next ‘hop’, at which point another routing decision will be made.

Within the routing table is a list of routes. Each route specifies an address range (expressed as a network address and a netmask), the interface to which packets matching that address range should be sent, and (optionally) the address of a gateway machine.

The route that is used for a given packet is the most specific one with an address range that matches the ultimate destination address. If the route has a gateway then the packet is forwarded to that gateway, otherwise it is forwarded directly to its final destination.

If the network address and netmask of a route are both zero then it will match packets with any destination address, but only if there is no other route that matches. Most routing tables have such an entry, which is known as the 'default route'.

There are a number of different ways in which routes may be created:

If your network is sufficiently complex to make use of a routing protocol such as BGP or RIP then these can automatically add any number of dynamic routes to the table.

Symptoms

A routing table error may result in:

Note that either of these issues could equally indicate a problem with iptables, and a failure to forward could indicate that forwarding has not been enabled.

In the unlikely event that you are using the NAT capabilities of iproute2 (as opposed to NAT in iptables) then a configuration error could additionally cause traffic to be given the wrong source or destination address. You should also be aware that iproute2 has the ability to support multiple routing tables, therefore the route taken by a packet may depend on more than just its destination address.

Scenario

Suppose that a machine has been configured to act as a boundary router between a local area network (connected to interface eth0 with the address 192.168.0.1/24) and the public Internet (connected to interface ppp0 with the address 203.0.113.144/32). The default gateway is 203.0.113.1. Because the local area network uses a private address range, iptables on the boundary router has been configured to SNAT them to its public IP address.

In order to test this configuration you have attempted to ping a machine on the public Internet (198.51.100.1) from a machine on the local area network (192.168.0.2), but this has failed.

Investigation

Strategy

There are two possibilities to consider:

It is usually worth quickly checking the first point before launching into any detailed investigation of the second.

Remember that routes created using the route and ip route commands are not persistent. Even if you are certain that the routing table was correct at some point in the past, that configuration will not necessarily survive a reboot.

Inspecting the routing table

The traditional method for viewing the content of the routing table is by means of the route command, however this predates iproute2 and the information it displays may be incomplete. The preferred method is to use the ip route command:

ip route show

The response is a list of routes, which for the scenario described above should be similar to:

203.0.113.1 dev ppp0  proto kernel  scope link  src 203.0.113.144
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.1
default via 203.0.113.1 dev ppp0

These routes specify that:

There is little benefit in attempting to predict the effect of the table on individual packets because this can be done more quickly and reliably using the automated method below. However you should aim to ensure that there are no spurious or missing routes, and that the destination addresses and device names are correct.

Testing the routing table

It is possible to test the effect of the routing table on a particular packet using the command ip route get. You need to know where the packet came from (IP address and interface name) and where it is supposed to go to (IP address).

For example, the outbound test packet in the scenario above originates from 192.168.0.2 and is received by the router on eth0. It is destined for 198.51.100.1. You can inquire how such packets are routed using the command:

ip route get to 198.51.100.1 from 192.168.0.2 iif eth0
<code>
 
which should give a response of the form:
 
<code>
198.51.100.1 from 192.168.0.2 via 203.0.113.1 dev ppp0  src 192.168.0.1
    cache <src-direct>  mtu 1500 advmss 1460 hoplimit 64 iif eth0

The main points to check are that the IP address of the next hop is correct (203.0.113.1 in this case) and that the packet is being sent onwards through the appropriate interface (ppp0).

You should also check the reverse path, because you will get no response to a ping unless it can be routed in both directions:

ip route get to 192.168.0.2 from 198.51.100.1 iif ppp0

This should give a response of the form:

192.168.0.2 from 198.51.100.1 dev eth0  src 203.0.113.144
    cache  mtu 1500 advmss 1460 hoplimit 64 iif ppp0

Points to note:

Variations

The route command

Though deprecated, the route command it adequate for inspecting most of the configurations that are likely to be encountered in practice:

route -n

The -n option causes IP addresses to be displayed numerically. This is usually preferable to converting them into names.

For the scenario described above the output should look similar to the following:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
203.0.113.1     0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         203.0.113.1     0.0.0.0         UG    0      0        0 ppp0