Update: This ended up being an iptables problem... The firewall rules I had were "stateful" so the icmp redirect was being killed by IPTables because it didn't identify the redirect as part of an already started connection. After changing it from stateful to non-stateful (I did this in the fwbuilder gui so I can't show you the exact IPTables code) the redirects started flowing and working as they should.

Suppose you have the following network setup.

The client computers default gateway is 10.2.3.254. When the client contacts it's default gateway while attempting to connect to a host on the 10.2.5.0 subnet the default gateway (10.2.3.254) sends an ICMP Redirect to tell the client that the correct router to use is 10.2.3.253.

By running tcpdump "icmp" on the default gateway router a redirect looks like this:


tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
09:56:37.410824 IP 10.2.3.254 > 10.2.3.15: ICMP redirect 10.2.5.5 to host 10.2.3.253, length 48


The client (in this case a Windows XP Pro workstation) will then update it's routing table to reflect the better route.

route print

Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0       10.2.3.254       10.2.3.15       30
        127.0.0.0        255.0.0.0        127.0.0.1       127.0.0.1       1
         10.2.3.0    255.255.255.0        10.2.3.15       10.2.3.15       30
        10.2.3.15  255.255.255.255        127.0.0.1       127.0.0.1       30
       10.2.3.255  255.255.255.255        10.2.3.15       10.2.3.15       30
         10.2.5.5  255.255.255.255       10.2.3.253       10.2.3.15       1
         10.2.5.7  255.255.255.255       10.2.3.253       10.2.3.15       1
        224.0.0.0        240.0.0.0        10.2.3.15       10.2.3.15       30
  255.255.255.255  255.255.255.255        10.2.3.15       10.2.3.15       1
Default Gateway:     10.2.3.254
===========================================================================

The router sending the redirect is governed by some kernel settings. The default settings for CentOS 5.x are shown here:

net.ipv4.conf.all.send_redirects = 1
net.ipv4.conf.eth0.send_redirects = 1
net.ipv4.conf.default.send_redirects = 1

net.ipv4.route.redirect_silence = 20480
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_load = 20

One problem I ran into was that the Windows XP clients, after a time forgot the route and then need to be redirected again. However the default gateway router seemed to have stopped sending ICMP Redirects. I don't know what the exact reason was, but a guess is that the redirect_number had been reached and the redirect silence time was so high that the router was not able to resend an ICMP Redirect soon enough.

So I have tried adding a lower redirect_silence value to /etc/sysctl.conf by placing net.ipv4.route.redirect_silence = 5120 in /etc/sysctl.conf and then running sysctl -p to load the value.

I am hoping this will stop the router from going silent due to hitting it's redirect_number and redirect_load values.

I got the 5120 value from a Ubuntu 8.04 box. Which has these defaults

net.ipv4.route.redirect_load = 5
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_silence = 5120

The original Fedora Core 1 Firewall which was in place before the CentOS had these defaults so 5120 seems alot closer than the CentOS 20480 value. I never had the ICMP Redirect going silent problem with FC1.

net.ipv4.route.redirect_silence = 2048
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_load = 2