Security Vulnerability: "Meltdown" and "Spectre" - Hypervisor Information.
This document (7022514) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 12
SUSE Linux Enterprise Server 11
SUSE Linux Enterprise Server 11
Situation
With the discovery and announcement of the "Meltdown" and "Spectre" vulnerabilities on speculative execution paths within CPUs (see TID 7022512 for details) it is important to understand the exposure and mitigation steps available in SUSE environments.
As disclosed it is possible to maliciously leverage the speculative execution capabilities of modern CPUs to read memory typically restricted to kernel space. On a single, bare-metal machine, this allows for user space process to access and read kernel space memory (Note: This vulnerability does not allow a user space process to write kernel space memory.) On a machine that is running as a hypervisor with one or more virtual machines (guests), the situation is more complex:
Background:
SUSE provides both Xen and KVM hypervisors as part of the SUSE Linux Enterprise product offerings. While the architecture of both hypervisors differ significantly, the risks of this speculative execution vulnerability are more dependent upon the virtualization mode of the guests rather than the hypervisor in use.
There are essentially two virtualization modes for virtual machines :
Mitigating the risks of accessing privileged memory within a virtual machine is similar to mitigating risks on bare-metal. Kernel updates should be applied to prevent user-space applications from exploiting the vulnerabilities. The greater risk with virtualized environments is the possibility of exploiting the vulnerability to attack the hypervisor itself and/or other guests.
The potential attack vectors and mitigations available to prevent attacks from a guest against the hypervisor and/or other guests looks like this:
1.For KVM hosts, existing updates for kernel and QEMU must be applied to prevent vulnerability. KVM guest kernel updates should also be applied, but failure to update a guest kernel will not allow the guest to access memory on a patched host. Unpatched guests will remain vulnerable within the memory confines of the guest itself.
2. In KVM HVM environments, the guest kernel should be patched to prevent in-guest vulnerabilities. When running on a patched KVM host, an unpatched guest does not have the ability to access memory of the KVM host, or other guests running on the host.
3. In Xen HVM environments the address space accessible to virtual machines is isolated from the host and restricted to memory allocated to that specific guest. In this environment, vulnerability variant 3 ("Meltdown"), cannot be used to access memory on the host or memory allocated to other machines running on the same host. However, vulnerability variant 2 ("Spectre v2") may be exploitable. Vulnerability variant 1("Spectre v1") has not been found to be applicable to Xen.
4. In Xen PV environments guest and hypervisor share the address space. From the hypervisor's perspective, the guest kernel runs as a user space process in a low privilege level, and the hypervisor runs at the highest privilege level. In this environment, the speculative execution vulnerability can be used from within a guest to gain access to all memory visible to the hypervisor - this includes memory allocated to other PV guests.
Note: 32bit PV guests are unable to exploit the variant 3 (“Meltdown”) vulnerability as they cannot address the higher memory addresses used by the hypervisor. Vulnerability of a PV guest to Spectre attacks match that of a Xen HVM guest.
As disclosed it is possible to maliciously leverage the speculative execution capabilities of modern CPUs to read memory typically restricted to kernel space. On a single, bare-metal machine, this allows for user space process to access and read kernel space memory (Note: This vulnerability does not allow a user space process to write kernel space memory.) On a machine that is running as a hypervisor with one or more virtual machines (guests), the situation is more complex:
Background:
SUSE provides both Xen and KVM hypervisors as part of the SUSE Linux Enterprise product offerings. While the architecture of both hypervisors differ significantly, the risks of this speculative execution vulnerability are more dependent upon the virtualization mode of the guests rather than the hypervisor in use.
There are essentially two virtualization modes for virtual machines :
- Fully Virtual mode (also known as a Hardware Virtual Machine), including the performance optimized Xen PVHVM mode. We subsume both as "HVM" in this document.
- Para-Virtual (PV) mode
Mitigating the risks of accessing privileged memory within a virtual machine is similar to mitigating risks on bare-metal. Kernel updates should be applied to prevent user-space applications from exploiting the vulnerabilities. The greater risk with virtualized environments is the possibility of exploiting the vulnerability to attack the hypervisor itself and/or other guests.
The potential attack vectors and mitigations available to prevent attacks from a guest against the hypervisor and/or other guests looks like this:
| KVM (only HVM) | XEN HVM only | XEN PV (at least one PV guest) | |
| Variant 1 / "Spectre v1" | Host must be updated | Not vulnerable, development review ongoing | Not vulnerable, development review ongoing |
| Variant 2 / "Spectre v2" | Host must be updated | Host must be updated | Host must be updated |
| Variant 3 / "Meltdown" | Not vulnerable | Not vulnerable | Host must be updated |
1.For KVM hosts, existing updates for kernel and QEMU must be applied to prevent vulnerability. KVM guest kernel updates should also be applied, but failure to update a guest kernel will not allow the guest to access memory on a patched host. Unpatched guests will remain vulnerable within the memory confines of the guest itself.
2. In KVM HVM environments, the guest kernel should be patched to prevent in-guest vulnerabilities. When running on a patched KVM host, an unpatched guest does not have the ability to access memory of the KVM host, or other guests running on the host.
3. In Xen HVM environments the address space accessible to virtual machines is isolated from the host and restricted to memory allocated to that specific guest. In this environment, vulnerability variant 3 ("Meltdown"), cannot be used to access memory on the host or memory allocated to other machines running on the same host. However, vulnerability variant 2 ("Spectre v2") may be exploitable. Vulnerability variant 1("Spectre v1") has not been found to be applicable to Xen.
4. In Xen PV environments guest and hypervisor share the address space. From the hypervisor's perspective, the guest kernel runs as a user space process in a low privilege level, and the hypervisor runs at the highest privilege level. In this environment, the speculative execution vulnerability can be used from within a guest to gain access to all memory visible to the hypervisor - this includes memory allocated to other PV guests.
Note: 32bit PV guests are unable to exploit the variant 3 (“Meltdown”) vulnerability as they cannot address the higher memory addresses used by the hypervisor. Vulnerability of a PV guest to Spectre attacks match that of a Xen HVM guest.
Resolution
Mitigation:
To mitigate the risk of these vulnerabilities within a physical or virtual machine, SUSE has released updated kernels (containing support for barriers, retpolines, KPTI and other safeguards) which should be deployed to all environments. On some hardware platforms microcode updates which add CPU flags associated used to control branch prediction. (See TID 7022512 for additional information on microcode update requirements.) SUSE kernels use these flag, along with CPU make and model, to determine the mitigation approach for the current platform. As default mitigations vary depending on the platform, the following command can be used to determine which mitigations are in use :
In addition to an updated kernel (and potentially microcode), KVM environments require updated QEMU and libvirt packages to pass branch prediction related CPU features to virtual machines for Spectre v2 mitigation. However these new CPU features are only provided to KVM guests if the guest’s CPU model supports the features (e.g. IvyBridge-IBRS and Skylake-IBRS), or if the features are explicitly added to the CPU definition (e.g. spec-ctrl, ibrs, and ibpb). This may require configuration changes for existing guests.
Meltdown mitigation in Xen environments is available through updated Xen hypervisor packages which restrict mappings of hypervisor memory to specific regions while 64-bit PV guests run. This approach, called XPI, mitigates Meltdown at the hypervisor level with no changes required within 64-bit PV guests. (In-machine Meltdown mitigation for 32-bit PV guests does require an updated kernel within the guest.)
The updated Xen hypervisor packages also provide Spectre v2 mitigation through utilizing branch prediction control and barriers at the hypervisor level,, and passing support for branch prediction control related CPU features to guests. Depending on the hardware platform in use, microcode updates may be required for full mitigation. Patching the hypervisor, and applying any required microcode updates, is sufficient to restrict guests from exploiting Spectre v2 to attack the host (and other guests). However, guests require an updated kernel to prevent the vulnerability within the virtual machine itself.
The Xen hypervisor has not been found to be susceptible to the Spectre v1 vulnerability. However, development reviews are still in progress and patches will be released as risks are identified and mitigated. Xen HVM guests should update to the latest available kernel to ensure the in-machine vulnerability is mitigated.
Mitigation Related Parameters :
Recent SUSE Linux kernels and hypervisors default to secure settings with Spectre and Meltdown mitigation enabled (if the hardware is detected as vulnerable). If your environment does not require this mitigation, it is possible to increase performance by disabling, or tuning this mitigation to match your needs.
Bare-Metal, KVM Host and HVM Guest Environments :
In depth information on these switches is available at https://www.suse.com/support/kb/doc/?id=7022512
Spectre related kernel parameters :
Meltdown related kernel parameters :
Xen Environments :
Spectre related hypervisor parameters:
The “cpuid=<value>” hypervisor parameter is used to enable or disable CPU features - including features related to branch prediction.
The “xpti=<value>” hypervisor parameter is used to enable, disable or tune XPTI for mitigation of Meltdown in the Xen hypervisor. This setting confines the address spaces of PV guests., with no changes required in the guest itself.
As Xen related parameters are somewhat complicated, the following examples show how to use all available parameters to disable mitigations :
Package version list:
SUSE has released various virtualization related package updates which provide the mitigation support detailed in this document.
The following list is the minimum recommended patch level:
SLES 12 SP3
SLES 12 SP2
SLES 12 SP1 – LTSS
SLES 12 – LTSS
SLES 11 SP4
SLES 11 SP3 – LTSS
Note: See TID 7022512 for details on updated kernel-default and microcode versions.
To mitigate the risk of these vulnerabilities within a physical or virtual machine, SUSE has released updated kernels (containing support for barriers, retpolines, KPTI and other safeguards) which should be deployed to all environments. On some hardware platforms microcode updates which add CPU flags associated used to control branch prediction. (See TID 7022512 for additional information on microcode update requirements.) SUSE kernels use these flag, along with CPU make and model, to determine the mitigation approach for the current platform. As default mitigations vary depending on the platform, the following command can be used to determine which mitigations are in use :
# grep -H . /sys/devices/system/cpu/vulnerabilities/*In the event no mitigations are available, the word “Vulnerable” will be seen rather then the word “Mitigation”, as shown in the following output :
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Barriers
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline, IBPB
# grep -H . /sys/devices/system/cpu/vulnerabilities/*Note: As hardware and software mitigations evolve, the actual mitigations in use may change. As long as the word “Mitigation:” is seen, the vulnerability should be addressed.
/sys/devices/system/cpu/vulnerabilities/meltdown:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user pointer sanitization
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline
In addition to an updated kernel (and potentially microcode), KVM environments require updated QEMU and libvirt packages to pass branch prediction related CPU features to virtual machines for Spectre v2 mitigation. However these new CPU features are only provided to KVM guests if the guest’s CPU model supports the features (e.g. IvyBridge-IBRS and Skylake-IBRS), or if the features are explicitly added to the CPU definition (e.g. spec-ctrl, ibrs, and ibpb). This may require configuration changes for existing guests.
Meltdown mitigation in Xen environments is available through updated Xen hypervisor packages which restrict mappings of hypervisor memory to specific regions while 64-bit PV guests run. This approach, called XPI, mitigates Meltdown at the hypervisor level with no changes required within 64-bit PV guests. (In-machine Meltdown mitigation for 32-bit PV guests does require an updated kernel within the guest.)
The updated Xen hypervisor packages also provide Spectre v2 mitigation through utilizing branch prediction control and barriers at the hypervisor level,, and passing support for branch prediction control related CPU features to guests. Depending on the hardware platform in use, microcode updates may be required for full mitigation. Patching the hypervisor, and applying any required microcode updates, is sufficient to restrict guests from exploiting Spectre v2 to attack the host (and other guests). However, guests require an updated kernel to prevent the vulnerability within the virtual machine itself.
The Xen hypervisor has not been found to be susceptible to the Spectre v1 vulnerability. However, development reviews are still in progress and patches will be released as risks are identified and mitigated. Xen HVM guests should update to the latest available kernel to ensure the in-machine vulnerability is mitigated.
Mitigation Related Parameters :
Recent SUSE Linux kernels and hypervisors default to secure settings with Spectre and Meltdown mitigation enabled (if the hardware is detected as vulnerable). If your environment does not require this mitigation, it is possible to increase performance by disabling, or tuning this mitigation to match your needs.
Bare-Metal, KVM Host and HVM Guest Environments :
In depth information on these switches is available at https://www.suse.com/support/kb/doc/?id=7022512
Spectre related kernel parameters :
nospectre_v2 Disables all Spectre v2 mitigation
nospec Disables microcode-based Spectre v2 mitigation (x86_64 only)
spectre_v2=<value>
on - unconditionally enable the mitigation
off - unconditionally disable the mitigation (same as "nospectre_v2")
auto - default setting where the kernel detects whether the CPU model is vulnerable
retpoline - replace indirect branches
retpoline,generic - google's original retpoline
retpoline,amd - AMD-specific minimal thunk
Meltdown related kernel parameters :
nopti Disables Meltdown mitigation (KPTI)
pti=<value>on - force-enables PTI even on AMD
off - force-disable PTI even on Intel
auto - PTI on for Intel, off for AMD (default)
Xen Environments :
Spectre related hypervisor parameters:
The “cpuid=<value>” hypervisor parameter is used to enable or disable CPU features - including features related to branch prediction.
cpuid=<List of comma separated CPU feature booleans (i.e. yes/true, no/false)>ibrsb=(yes/no) - disable IBRS
ibpb=(yes/no) - disable IBPB
stibp=(yes/no) - disable STIBP
NOTE :
The “bti=<value>” hypervisor parameter is used to enable, disable or tune Branch Target Injection (BTI), software-based mitigations (including retpoline, lfence and any usage of speculative execution control features provided by the hardware).
- Default settings enable ibrs, ibpb and stibp.
- Underlying hardware must support capabilities enabled through cpuid
- Features can be ignored (e.g. ‘no-ibrsb’) which disables the feature at the hypervisor level, and prevents the feature from being passed to guests.
- Hardware features are automatically passed through to guests, but may be disabled to improve migration compatibility.
bti=<List of comma separated BTI mitigation tunings>thunk=retpoline|lfence|jmp - Determines mitigation method:
retpoline – default, preferred for Intel
lfence – preferred for AMD
jmp – minimal overhead thunk
ibrs=(yes/no) - controls hypervisor support for IBRS
ibpb=(yes/no) - controls hypervisor support for IBPB on vCPU context switches
rsb_{vmexit,native}=(yes/no) - controls when Return Stack Buffers are overwritten by the hypervisor
NOTE :
Meltdown related hypervisor parameters:
- By default, the most appropriate mitigation is selected based on compiled in support, loaded microcode, and hardware details.
- Manually adjusting the bti parameter is only recommended for use when disabling mitigation.
- Disabling hypervisor support for IBRS (bti=ibrs=no) does not prevent Xen from providing IBRS to guests.
- Additional information can be found in /usr/share/doc/packages/xen/misc/xen-command-line.markdown
The “xpti=<value>” hypervisor parameter is used to enable, disable or tune XPTI for mitigation of Meltdown in the Xen hypervisor. This setting confines the address spaces of PV guests., with no changes required in the guest itself.
xpti=<value>Disabling ALL mitigations in Xen environments :on - enable XPTI mitigation (default on all platforms except AMD)
off - disable XPTI mitigation
As Xen related parameters are somewhat complicated, the following examples show how to use all available parameters to disable mitigations :
- To disable everything on the Xen hypervisor, while allowing guests access to CPU features related to speculative execution control:
bti=thunk=jmp,ibrs=no,ibpb=no,rsb_native=no,rsb_vmexit=no xpti=no
- To disable everything on the Xen hypervisor, and prevent any mitigation related CPU features from being passed to guests :
cpuid=no-ibrsb,no-ibpb,no-stibp bti=thunk=jmp,rsb_native=no,rsb_vmexit=no xpti=no
Package version list:
SUSE has released various virtualization related package updates which provide the mitigation support detailed in this document.
The following list is the minimum recommended patch level:
SLES 12 SP3
- qemu-2.9.1-6.12.1 released Wednesday, 21st of March 2018
- libvirt-3.3.0-5.19.2 released Wednesday, 11th of April 2018
- xen-4.9.1_08-3.26.1 released Wednesday, 14th of February 2018
SLES 12 SP2
- qemu-2.6.2-41.37.1 released Tuesday, 27th of March 2018
- libvirt-2.0.0-27.37.1 released Friday, 13th of April 2018
- xen-4.7.5_02-43.27.1 released Tuesday, 10th of April 2018
SLES 12 SP1 – LTSS
- qemu-2.3.1-33.6.1 released Tuesday, 9th of January 2018
- xen-4.5.5_24-22.43.1 released Monday, 5th of March 2018
- kernel-xen-3.12.74-60.64.85.1 released Thursday, 29th of March 2018
SLES 12 – LTSS
- qemu-2.0.2-48.37.1 released Thursday, 4th of January 2018
- xen-4.4.4_28-22.62.1 released Monday, 5th of March 2018
- kernel-xen-3.12.61-52.125.1 released Wednesday, 28th of March 2018
SLES 11 SP4
- kvm-1.4.2-60.6.1 released Thursday, 4th of January 2018
- libvirt-1.2.5-23.9.1 released Friday, 13th of April 2018
- xen-4.4.4_28-61.23.2 released Thursday, 8th of March 2018
- kernel-xen-3.0.101-108.35.1 released Tuesday, 27th of February 2018
SLES 11 SP3 – LTSS
- kvm-1.4.2-53.14.1 released Monday, 8th of January 2018
- xen-4.2.5_21-45.19.1 released Wednesday, 14th of March 2018
- kernel-xen-3.0.101-0.47.106.19.1 released Monday, 12th of March 2018
Note: See TID 7022512 for details on updated kernel-default and microcode versions.
Cause
CVE-2017-5753 (Spectre - variant 1)
CVE-2017-5715 (Spectre - variant 2)
CVE-2017-5754 (Meltdown - variant 3)
CVE-2017-5715 (Spectre - variant 2)
CVE-2017-5754 (Meltdown - variant 3)
Additional Information
Additional steps and reading
Another method of mitigating Meltdown is to convert existing 64-bit Xen PV guests to HVM guests. The steps required for this process are described in the SUSE Virtualization Best Practices Guide
Windows guests :
Windows guests run in HVM mode and therefore cannot exploit the variant 3 ("Meltdown") vulnerability to gain access to hypervisor address space. However, KVM hosts must be patched to prevent Windows guests from using variants 1 and 2 ("Spectre") against the hypervisor.
Windows guests themselves should also be patched to prevent in-guest exploits using any of the possible variants. (Contact Microsoft to obtain patches specific to the version of Windows in use.) SUSE's Virtual Machine Driver Pack (VMDP) paravirtual drivers do not play a role in either mitigating or contributing to these vulnerabilities.
Third party hypervisors :
SUSE advises to upgrade all SLE based virtual machines, when they are deployed on virtualization platforms other than Xen/KVM detailed here.
Another method of mitigating Meltdown is to convert existing 64-bit Xen PV guests to HVM guests. The steps required for this process are described in the SUSE Virtualization Best Practices Guide
Windows guests :
Windows guests run in HVM mode and therefore cannot exploit the variant 3 ("Meltdown") vulnerability to gain access to hypervisor address space. However, KVM hosts must be patched to prevent Windows guests from using variants 1 and 2 ("Spectre") against the hypervisor.
Windows guests themselves should also be patched to prevent in-guest exploits using any of the possible variants. (Contact Microsoft to obtain patches specific to the version of Windows in use.) SUSE's Virtual Machine Driver Pack (VMDP) paravirtual drivers do not play a role in either mitigating or contributing to these vulnerabilities.
Third party hypervisors :
SUSE advises to upgrade all SLE based virtual machines, when they are deployed on virtualization platforms other than Xen/KVM detailed here.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:7022514
- Creation Date: 04-Jan-2018
- Modified Date:03-Mar-2020
-
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com
