XEN: Error about Xend not running when attempting to manage DomU's

SUSE Linux Enterprise Server 10 Service Pack 1
SUSE Linux Enterprise Server 10 Service Pack 2

When attempting to manage, create, delete or start a Xen DomU (virtual machine) the process errors out. Xend is running.

If the Xenstore managment database becomes corrupted, then the management of Xen virtual machines via "xm" commands, YaST, Virt-Manager or any management interface that uses the Xen API will not be able to manage the Xen hosts. Unmanaged DomU's will not be able to be started. A common cause of the corruption is when the volume where "/var" resides becomes full.

matching the error
The following steps will help you to match the error.

1. Check the status of Xend
   service xend status
   should have a status of "running"
2. Check the status of libvirtd
   service libvirtd status
   

   should have a status of "running"
3. Check the /var/log/xend-debug.log
   tail /var/log/xen/xend-debug.log
   

   May have something similiar to

       self.__target(*self.__args, **self.__kwargs)
     File "/usr/lib64/python2.4/site-packages/xen/web/httpserver.py", line 312, in run
       (sock, addr) = self.accept()
     File "/usr/lib64/python2.4/site-packages/xen/web/httpserver.py", line 331, in accept
       return self.socket.accept()
     File "/usr/lib64/python2.4/socket.py", line 169, in accept
       sock, addr = self._sock.accept()
   error: (22, 'Invalid argument')
   
   Exception starting xend: not well-formed (invalid token): line 11, column 2

If the above matches, then this TID should be considered as a potential solution.

remove the xend database
Managed Xen DomU's store their definitions in /var/lib/xend/domains. The format uses a Python syntax. If the managed database becomes corrupted then you can lose the virtual machine definitions. If you have made any changes using "xm" commands, Virt-Manager or YaST to any virtual machine definition, then it is very likely that the configuration at install time has been altered. Because of this concern, it is strongly recommended that you backup the database before you do anything.

Therefore, it is recommended that you RENAME /var/lib/xend to /var/lib/xend.bk (or something similiar).

1. Backup the /var/lib/xend. Renaming the directory will be sufficient.
2. Shutdown the xend service
   service xend stop
   
3. Shutdown the libvirtd service
   service libvirtd stop
   
4. Make sure that /var/lib/xend does NOT exist. When xend is started again, this directory will be recreated.
   
5. Start up the xend service
   service xend start
   
6. Start up the libvirtd service
   service libvirtd start
   
7. Test to make sure a simple "xm" command works. No DomU's will appear in the management database.
   xm list
   
   
   

unmanaged domains (legacy configuration)
If you do not use a managed database, use the DomU's in a clustered environment  then there is no further configuration needed. You should now be able to start, stop, create and otherwise manage your virtual machines normally.

managed domains
Managed domains are DomU's that are manipulated using the "xm" commands such as "block-attach", "network-attach", or are managed through YaST or Virt-Manager. If you use a graphical utility to manage the DomU then the DomU's are most likely managaged. These steps will help you recover the configuration for those DomU's.

1. Go to the backup made of the /var/lib/xend.
2. Go to the "domain" directory.
3. There should directories for each DomU; the directories will be named with a GUID.
   
4. For each of the GUID directories, there is a "config.sxp" that defines the virtual machines.
5. Import the .sxp DomU configuration
   xm new -F config.sxp
6. See if the re-imported DomU appears in a DomU listing
   xm list
7. Test whether or not they start normally
   xm start DOMU
8. Repeat step 5 through 7 for each directory in step 4

Running DomU's: If there are running DomU's, then you can safely follow the steps described in this TID.