Nova DB out of sync - VM not running on expected hypervisor
This document (7021199) is provided subject to the disclaimer at the end of this document.
Environment
Situation
nova list
...
ID | Name | Tenant ID | Status | Task State | Power State | Networks |
+--------------------------------------+-----------+----------------------------------+---------+------------+-------------+--------------------------+
...
| aa8e645c-c737-49c5-ae41-e7ff19f75ea6 | testserver | 3ea7d60e677d4012b757284f56a159fc | ACTIVE | - | Running | test_network=10.10.10.9 |
nova show <instance-id>
| OS-EXT-SRV-ATTR:hypervisor_hostname | d00-00-00-c7-01-01.cloud.example.com
| name | testserver
nova hypervisor-servers <compute-node-fqdn>
+--------------------------------------+-------------------+---------------+-------------------------------------------+
| ID | Name | Hypervisor ID | Hypervisor Hostname |
+--------------------------------------+-------------------+---------------+-------------------------------------------+
..
| aa8e645c-c737-49c5-ae41-e7ff19f75ea6 | instance-00000035 | 1 | d00-00-00-c7-01-01.cloud.example.com |
...
on compute node:
root@:~ # virsh list --all
Id Name State
----------------------------------------------------
1 instance-00000036 running
but there is no such instance here
On other compute node where instance is running:
nova hypervisor-servers d00-00-00-c7-01-02.cloud.example.com
+----+------+---------------+---------------------+
| ID | Name | Hypervisor ID | Hypervisor Hostname |
+----+------+---------------+---------------------+
...
an empty list for nova.
but virsh sees the instance:
d00-00-00-c7-01-01:~ # virsh list
Id Name State
----------------------------------------------------
187 instance-00000035 running
Resolution
Resolution would be to update nova database and reboot instance
On controller retrieve database details:
grep postgr /etc/nova/nova.conf.d/*
access to database:
psql -U nova -d nova -W -h <ip-from-config>
SELECT * FROM instances WHERE display_name = 'testserver';
Get the uuid from this query and update:
UPDATE instances SET host = 'd00-00-00-c7-01-01' WHERE uuid = 'UUID_OF_INSTANCE';
Exit the database and then run nova reboot testserver.
After reboot you should check if nova list points to the correct host for that instance.
Cause
Additional Information
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:7021199
- Creation Date: 14-Aug-2017
- Modified Date:03-Mar-2020
-
- SUSE Open Stack Cloud
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com
