Reply packets are sent over an unexpected interface

This document (7000318) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 10
SUSE Linux Enterprise Desktop 10

Situation

Symptoms

Query packets are received over one network interface, but response/reply packets are sent out over a different interface (typically, eth0), or
Packets destined for an IP address corresponding to a physically disconnected interface are still being received.

Goal

When a query packet is received over interface X, have the response packets to it be sent out over interface X (source based routing), or
when a network interface is physically disconnected, packets destined for the IP address associated with the interface are to be discarded.

Resolution

Set up split access routing

To route response/reply packets out over the same interface that the packets that they are a response to came in on, separate routing tables (based on the incoming interface) can be set up. For a detailed explanation, refer to the "Split Access" section of the "Rules - routing policy database" chapter in the Linux Advanced Routing & Traffic Control HOWTO (LARTC).

Additional Information

Background information: how Linux routes by default

In Linux, by default, packets are considered individually for routing purposes. Thus, all the routing algorithm considers is where to send a packet based on that packet itself, without taking into consideration that the packet may be a response packet of sorts. In a typical setup, this means that all outgoing traffic is going out over one interface, eth0.

If multiple NICs in Linux are configured with addresses in the same subnet, by default communication into that subnet will always be done by the first NIC in that subnet that is started.

Example setup

To obtain the expected behaviour (as stated under "goal" above), split routing needs to be set up.

Assume a single server with two NICs, one configured for the address 10.105.16.100; the other for 10.105.16.101.

First, create two routing tables, T1 and T2 to be used for packets sent to or from these NICs by adding the lines

252 T1
251 T2

to /etc/iproute2/rt_tables.

Next, set up the routing rules to route incoming and outgoing packets via these tables:

ip route add 10.105.16.0/24 dev eth0 src 10.105.16.100 table T1
ip route add default via 10.105.16.2 dev eth0 src 10.105.16.100 table T1
ip rule add from 10.105.16.100 table T1

ip route add 10.105.16.0/24 dev eth1 src 10.105.16.101 table T2
ip route add default via 10.105.16.2 dev eth1 src 10.105.16.101 table T2
ip rule add from 10.105.16.101 table T2

Note that these rules do not cover the routing of outgoing packets which are not response packets (e.g. outgoing ICMP ECHO ("ping") packets). These packets are still governed by the default route. In some setups, there may be a requirement to use both interfaces for such packets as well (e.g. for load balancing). Refer to the "load balancing" section of LARTC for details on how to implement such requirements.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.