High network latency between domUs on SLES 11 SP1 Xen host

This document (7009790) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 11 Service Pack 1

Situation

When pinging with larger-than-normal (97-byte) ICMP packets between domU systems on the same Xen host the latency increased several times when compared to pinging either the dom0 (Xen host domain), another system elsewhere on the network, or when sending smaller ICMP packets between domU systems. The change in latency went from sub-millisecond response times to response times of (on average) three to five milliseconds per packet. This occurred regardless of domU version on systems with SLES 11 SP1 as the host.

Resolution

First, update to the latest released kernel. Enhancements are made to performance as the kernel progresses, and new settings are made available for tuning.

Some changes implemented in the kernel for performance improvements were found to have caused delays in this specific situation. The options added are configurable by adding a line in the /etc/modprobe.conf.local file. The default setting is shown below:

options netbk     tasklets=0 bind=0

Adding this line to the file and changing EITHER tasklets OR bind to 1 (from zero) causes the latency to be returned to normal. The default setting is in place because the benefit, better overall scheduling behavior of the dom0, outweighs the penalty reported initially.

Modifying kernel and driver settings should be done after thoroughly understanding the settings and verifying the benefits compared to trade-offs. While this option is configurable it is recommended to use the defaults as implemented.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.