Recommended update for jemalloc

Announcement ID: SUSE-RU-2024:3335-1
Rating: moderate
References:
Affected Products:
  • Basesystem Module 15-SP6
  • Development Tools Module 15-SP6
  • openSUSE Leap 15.6
  • SUSE Linux Enterprise Desktop 15 SP6
  • SUSE Linux Enterprise Real Time 15 SP6
  • SUSE Linux Enterprise Server 15 SP6
  • SUSE Linux Enterprise Server for SAP Applications 15 SP6

An update that contains one feature and has two fixes can now be installed.

Description:

This update for jemalloc fixes the following issues:

Update to release 5.3.0 (jsc#PED-7454):

  • Added the thread.idle mallctl which hints that the calling thread will be idle for a nontrivial period of time.
  • Allow small size classes to be the maximum size class to cache in the thread-specific cache, through the opt.[lg_]tcache_max option.
  • Make the behavior of realloc(ptr, 0) configurable with opt.zero_realloc.
  • Support C++17 over-aligned allocation.
  • Add the thread.peak mallctl for approximate per-thread peak memory tracking.

  • Add high resolution timestamp support for profiling.

  • Set lg-page to 2^16 on aarch64 to be compatible with 64k page size kernel

  • Enable LTO as it works (bsc#1133136).
  • Re-enable profiling on armv6/armv7 since bsc#1105633 is fixed

Update to version 5.2.1:

  • Bug fixes:
  • Fix a severe virtual memory leak on Windows. This regression was first released in 5.0.0.
  • Fix size 0 handling in posix_memalign(). This regression was first released in 5.2.0.
  • Fix the prof_log unit test which may observe unexpected backtraces from compiler optimizations. The test was first added in 5.2.0.
  • Fix the declaration of the extent_avail tree. This regression was first released in 5.1.0.
  • Fix an incorrect reference in jeprof. This functionality was first released in 3.0.0.
  • Fix an assertion on the deallocation fast-path. This regression was first released in 5.2.0.
  • Fix the TLS_MODEL attribute in headers. This regression was first released in 5.0.0.
  • Optimizations and refactors:
  • Implement opt.retain on Windows and enable by default on 64-bit.
  • Optimize away a branch on the operator delete path.
  • Add format annotation to the format generator function.
  • Refactor and improve the size class header generation.
  • Remove best fit.

  • Avoid blocking on background thread locks for stats.

  • Disable LTO (bsc#1133136).

  • Drop static library package, nothing seems to use it.

  • Remove openSUSE 11.1 build support.

Update to version 5.2.0:

New features:

  • Implement oversize_threshold, which uses a dedicated arena for allocations crossing the specified threshold to reduce fragmentation.
  • Add extents usage information to stats.
  • Log time information for sampled allocations.
  • Support 0 size in sdallocx.
  • Output rate for certain counters in malloc_stats.
  • Add mallctl interfaces:
  • opt.oversize_threshold
  • stats.arenas.<i>.extent_avail
  • stats.arenas.<i>.extents.<j>.n{dirty,muzzy,retained}
  • stats.arenas.<i>.extents.<j>.{dirty,muzzy,retained}_bytes
  • Optimizations and refactors:
  • Refactor the TSD module.
  • Implement opt.oversize_threshold which uses a dedicated arena for requests crossing the threshold, also eagerly purges the oversize extents. Default the threshold to 8 MiB.
  • Improve error handling for THP state initialization.
  • Refactor and optimize the tcache fill / flush paths.
  • Optimize sync / lwsync on PowerPC.
  • Bypass extent_dalloc() when retain is enabled.
  • Lower the default number of background threads to 4 (when the feature is enabled).
  • Use arena index for arena-matching checks.
  • Avoid forced decay on thread termination when using background threads.
  • Disable muzzy decay by default.
  • Only initialize libgcc unwinder when profiling is enabled.
  • Bug fixes:
  • Fix background thread index issues with max_background_threads.
  • Fix stats output for opt.lg_extent_max_active_fit.
  • Properly trigger decay on tcache destroy.
  • Detect whether explicit extent zero out is necessary with huge pages or custom extent hooks, which may change the purge semantics.
  • Fix a side effect caused by extent_max_active_fit combined with decay-based purging, where freed extents can accumulate and not be reused for an extended period of time.
  • Fix a missing unlock on extent register error handling.
  • Incompatible changes:
  • Remove --with-lg-page-sizes.

Update to version 5.1.0:

New features:

* Implement transparent huge page support for internal metadata. (@interwq)
* Add opt.thp to allow enabling / disabling transparent huge pages for all mappings. (@interwq)
* Add maximum background thread count option. (@djwatson)
* Allow prof_active to control opt.lg_prof_interval and prof.gdump. (@interwq)
* Allow arena index lookup based on allocation addresses via mallctl. (@lionkov)
* Allow disabling initial-exec TLS model. (@davidtgoldblatt, @KenMacD)
* Add opt.lg_extent_max_active_fit to set the max ratio between the size of the active extent selected (to split off from) and the size of the requested allocation. (@interwq, @davidtgoldblatt)
* Add retain_grow_limit to set the max size when growing virtual address space. (@interwq)
* Add mallctl interfaces:
* arena.&lt;i&gt;.retain_grow_limit (@interwq)
* arenas.lookup (@lionkov)
* max_background_threads (@djwatson)
* opt.lg_extent_max_active_fit (@interwq)
* opt.max_background_threads (@djwatson)
* opt.metadata_thp (@interwq)
* opt.thp (@interwq)
* stats.metadata_thp (@interwq)

Portability improvements:

* Support GNU/kFreeBSD configuration. (@paravoid)
* Support m68k, nios2 and SH3 architectures. (@paravoid)
* Fall back to FD_CLOEXEC when O_CLOEXEC is unavailable. (@zonyitoo)
* Fix symbol listing for cross-compiling. (@tamird)
* Fix high bits computation on ARM. (@davidtgoldblatt, @paravoid)
* Disable the CPU_SPINWAIT macro for Power. (@davidtgoldblatt, @marxin)
* Fix MSVC 2015 &amp; 2017 builds. (@rustyx)
* Improve RISC-V support. (@EdSchouten)
* Set name mangling script in strict mode. (@nicolov)
* Avoid MADV_HUGEPAGE on ARM. (@marxin)
* Modify configure to determine return value of strerror_r. (@davidtgoldblatt, @cferris1000)
* Make sure CXXFLAGS is tested with CPP compiler. (@nehaljwani)
* Fix 32-bit build on MSVC. (@rustyx)
* Fix external symbol on MSVC. (@maksqwe)
* Avoid a printf format specifier warning. (@jasone)
* Add configure option --disable-initial-exec-tls which can allow jemalloc to be dynamically loaded after program startup. (@davidtgoldblatt, @KenMacD)
* AArch64: Add ILP32 support. (@cmuellner)
* Add --with-lg-vaddr configure option to support cross compiling. (@cmuellner, @davidtgoldblatt)

Optimizations and refactors:

* Improve active extent fit with extent_max_active_fit. This considerably reduces fragmentation over time and improves virtual memory and metadata usage. (@davidtgoldblatt, @interwq)
* Eagerly coalesce large extents to reduce fragmentation. (@interwq)
* sdallocx: only read size info when page aligned (i.e. possibly sampled), which speeds up the sized deallocation path significantly. (@interwq)
* Avoid attempting new mappings for in place expansion with retain, since it rarely succeeds in practice and causes high overhead. (@interwq)
* Refactor OOM handling in newImpl. (@wqfish)
* Add internal fine-grained logging functionality for debugging use. (@davidtgoldblatt)
* Refactor arena / tcache interactions. (@davidtgoldblatt)
* Refactor extent management with dumpable flag. (@davidtgoldblatt)
* Add runtime detection of lazy purging. (@interwq)
* Use pairing heap instead of red-black tree for extents_avail. (@djwatson)
* Use sysctl on startup in FreeBSD. (@trasz)
* Use thread local prng state instead of atomic. (@djwatson)
* Make decay to always purge one more extent than before, because in practice large extents are usually the ones that cross the decay threshold. Purging the additional extent helps save memory as well as reduce VM fragmentation. (@interwq)
* Fast division by dynamic values. (@davidtgoldblatt)
* Improve the fit for aligned allocation. (@interwq, @edwinsmith)
* Refactor extent_t bitpacking. (@rkmisra)
* Optimize the generated assembly for ticker operations. (@davidtgoldblatt)
* Convert stats printing to use a structured text emitter. (@davidtgoldblatt)
* Remove preserve_lru feature for extents management. (@djwatson)
* Consolidate two memory loads into one on the fast deallocation path. (@davidtgoldblatt, @interwq)

Bug fixes (most of the issues are only relevant to jemalloc 5.0):

* Fix deadlock with multithreaded fork in OS X. (@davidtgoldblatt)
* Validate returned file descriptor before use. (@zonyitoo)
* Fix a few background thread initialization and shutdown issues. (@interwq)
* Fix an extent coalesce + decay race by taking both coalescing extents off the LRU list. (@interwq)
* Fix potentially unbound increase during decay, caused by one thread keep stashing memory to purge while other threads generating new pages. The number of pages to purge is checked to prevent this. (@interwq)
* Fix a FreeBSD bootstrap assertion. (@strejda, @interwq)
* Handle 32 bit mutex counters. (@rkmisra)
* Fix a indexing bug when creating background threads. (@davidtgoldblatt, @binliu19)
* Fix arguments passed to extent_init. (@yuleniwo, @interwq)
* Fix addresses used for ordering mutexes. (@rkmisra)
* Fix abort_conf processing during bootstrap. (@interwq)
* Fix include path order for out-of-tree builds. (@cmuellner)

Incompatible changes:

* Remove --disable-thp. (@interwq)
* Remove mallctl interfaces:
* config.thp (@interwq)

Documentation:

* Add TUNING.md. (@interwq, @davidtgoldblatt, @djwatson)
  • Disable THP for ARM (32 bits). This fix build by passing all tests successfully

Patch Instructions:

To install this SUSE update use the SUSE recommended installation methods like YaST online_update or "zypper patch".
Alternatively you can run the command listed for your product:

  • openSUSE Leap 15.6
    zypper in -t patch SUSE-2024-3335=1 openSUSE-SLE-15.6-2024-3335=1
  • Basesystem Module 15-SP6
    zypper in -t patch SUSE-SLE-Module-Basesystem-15-SP6-2024-3335=1
  • Development Tools Module 15-SP6
    zypper in -t patch SUSE-SLE-Module-Development-Tools-15-SP6-2024-3335=1

Package List:

  • openSUSE Leap 15.6 (aarch64 ppc64le s390x x86_64 i586)
    • libjemalloc2-5.3.0-150600.15.4.1
    • jemalloc-debugsource-5.3.0-150600.15.4.1
    • jemalloc-5.3.0-150600.15.4.1
    • libjemalloc2-debuginfo-5.3.0-150600.15.4.1
    • jemalloc-devel-static-5.3.0-150600.15.4.1
    • jemalloc-devel-5.3.0-150600.15.4.1
  • Basesystem Module 15-SP6 (aarch64 ppc64le s390x x86_64)
    • jemalloc-debugsource-5.3.0-150600.15.4.1
    • libjemalloc2-debuginfo-5.3.0-150600.15.4.1
    • jemalloc-devel-5.3.0-150600.15.4.1
    • jemalloc-devel-static-5.3.0-150600.15.4.1
    • libjemalloc2-5.3.0-150600.15.4.1
  • Development Tools Module 15-SP6 (aarch64 ppc64le s390x x86_64)
    • jemalloc-debugsource-5.3.0-150600.15.4.1
    • jemalloc-5.3.0-150600.15.4.1

References: