Large XML files exceed nodeset limits

This document (7023892) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15
SUSE Linux Enterprise Server 12

Situation

While processing very large XML files the following error was observed:

XPath error : Memory allocation failed : growing nodeset hit limit

The following test results were noted:

SLES15 SP0 libxml2-2-2.9.7-3.6.1   - FAILED
SLES12 SP4 libxml2-2-2.9.4-46.15.1 - FAILED
SLES12 SP3 libxml2-2-2.9.4-46.15.1 - FAILED
SLES12 SP2 libxml2-2-2.9.4-46.12.1 - FAILED
SLES11 SP4 libxml2-2.7.6-0.77.10.1 - WORKED

Resolution

A newer version of libxml2-2 is required that introduces a new environment variable LIBXML_MAX_NODESET_LENGTH that can be set to increase the nodeset limit.

Variable: LIBXML_MAX_NODESET_LENGTH
Default Value: 10000000
Minimum Value: 1
Maximum Value: INT_MAX (Generally 2147483647, but this depends on the compiler)

If LIBXML_MAX_NODESET_LENGTH is less than 1 or greater than INT_MAX, the default value is used.

One possible use case is to define:

export LIBXML_MAX_NODESET_LENGTH=20000000

in a start up script before launching your application that requires a higher limit.

A PTF is available as needed from support.

Cause

A patch was introduced to hard code the XPATH_MAX_NODESET_LENGTH to avoid denial of service attacks. The patch was never back ported to SLES11 limiting the nodeset value.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.