vNUMA vs.VCPU hotplug on VMware

This document (7015827) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 12
SUSE Linux Enterprise Server 11
SUSE Linux Enterprise Server 11 Service Pack 3 (SLES 11 SP3)
SUSE Linux Enterprise Server 11 Service Pack 2 (SLES 11 SP2)
SUSE Linux Enterprise Server 11 Service Pack 1 (SLES 11 SP1)
SUSE Linux Enterprise Server 10 Service Pack 4 (SLES 10 SP4)
SUSE Linux Enterprise Server 10 Service Pack 3 (SLES 10 SP3)
SUSE Linux Enterprise Server 10 Service Pack 1 (SLES 10 SP1)
SUSE Linux Enterprise Server 10 Service Pack 2 (SLES 10 SP2)

Situation

When VMware CPU Hot Add is enabled for a VM, vNUMA is disabled and UMA with interleaved memory access will be used. In this case the memory/cpu layout is that all cpu's are assigned to the first zone and the other zones are are empty,
This layout is not supported by the SLES kernel if NUMA is enabled in the guest OS.and can lead to strange behaviour, eg. memory corruptions.
Note: These mentioned memory corruptions will not occur with VMware versions > 6.x

Resolution

The resolution is to either disable VMware CPU Hot Add or disable NUMA in the guest OS.
This memory/cpu layout is fixed with in vSphere 6.x
The issue is also described by the VMware knowledge base article 2040375.

Cause

NUMA CPU assignments are done at boot time, there's currently no way to change these assignments at runtime.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.