jabberd initialization fails with "db: couldn't open environment: Resource temporarily unavailable"

This document (7015878) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Manager 3.0
SUSE Manager 2.1
SUSE Manager 1.7

Situation

When jabberd is attempting to start the following error can be seen in /var/log/messages and jabberd fails to load:

Nov 11 10:05:12 suma1 jabberd/sm[7216]: db: couldn't open environment: Resource temporarily unavailable
Nov 11 10:05:12 suma1 jabberd/sm[7216]: initialisation of storage driver 'db' failed
Nov 11 10:05:12 suma1 jabberd/sm[7216]: failed to initialise one or more storage drivers, aborting
Nov 11 10:05:14 suma1 jabberd/c2s[7233]: db: couldn't open environment: Resource temporarily unavailable
Nov 11 10:05:14 suma1 jabberd/c2s[7233]: failed to initialize auth module 'db'

Resolution

This issue is caused by a corrupted jabberd database, which can be fixed by performing the following:

rcosa-dispatcher stop
rcjabberd stop
cd /var/lib/jabberd/db
rm *
rcjabberd start
rcosa-dispatcher start

After removing the database, jabberd should start without issue.

Cause

jabberd database corruption can occur for many reasons, the most common cause is taskomatic running out of memory (found in TID 7012598 - Taskomatic reports out of JAVA heap space)

Additional Information

Note: This document is valid for SUSE Manager 1.7, 2.1 and 3.0 _only_!
SUSE has seen reports where customers ran a cronjob with the above commands frequently.

BEFORE upgrading to SUSE Manager 3.1, it is imperative/mandatory to remove this cronjob.

SUSE Manager 3.1 uses a different database which is not recreated automatically; deleting the jabberdb on 3.1 is not recommended (nor a good idea).

If the db already is deleted (file is created with 0 bytes) and OSAD/Jabberd misbehave, run:

/usr/share/spacewalk/setup/jabberd/create_sqlite3_database

to recreate the database with the correct content, then restart osad-dispatcher and jabberd.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.