How to obtain systemd service core dumps

echo '|/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e' > /proc/sys/kernel/core_pattern

1. define kernel.core_pattern configuration

echo 'kernel.core_pattern=|/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e' > /etc/sysctl.d/50-coredump.conf

2. reboot

sles12sp1:~ # systemd-coredumpctl list 
TIME                           PID   UID   GID SIG EXE
Fri 2016-01-08 15:01:10 MST   1088     0     0   6 /usr/sbin/sshd
Fri 2016-01-08 15:34:04 MST   1655     0     0   6 /usr/sbin/sshd
Fri 2016-01-08 15:34:05 MST  21491     0     0   6 /usr/sbin/sshd
Fri 2016-01-08 15:35:27 MST   1361     0     0   6 /usr/sbin/cron
Fri 2016-01-08 15:35:36 MST  21501     0     0   6 /usr/sbin/sshd
Fri 2016-01-08 15:35:39 MST  21530     0     0   6 /usr/sbin/cron

sles12sp1:~ # systemd-coredumpctl -o core.sshd dump /usr/sbin/sshd
TIME                                         PID   UID   GID SIG EXE
              Fri 2016-01-08 15:35:36 MST  21501     0     0   6 /usr/sbin/sshd
More than one entry matches, ignoring rest.

sles12sp1:~ # ls -l core.sshd
-rw-r--r-- 1 root root 954368 Jan  8 15:49 core.sshd

To get other coredumps, by PID# run:
 

systemd-coredumpctl -o FileName dump <#pid>
systemd-coredumpctl -o core.sshd dump 21491

NOTE: You must run systemd-coredumpctl dump to extract any core dumps you want out of the systemd journal before rebooting the server. The core dumps stored in the systemd journal will not persist after a server reboot. See systemd-coredumpctl(1) for more options.

Application cores can be stored in /var/lib/systemd/coredump depending on the storage setting in /etc/systemd/coredump.conf, see the coredump.conf(5) man page for options.


For SLES12SP3 to SLES15SP3

NOTE: The systemd command to manage core files changed from "systemd-coredumpctl" to simply "coredumpctl" and the default core pattern is already properly set to:
 

|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %e

Also note that systemd is already by default configured to write core files to disk (/var/lib/systemd/coredump/). It should thus be possible to simply use getappcore against any existing application core files without the need to first use coredumpctl.


getappcore

Usually uploading  core file and list of installed packages (list of installed packages in included in supportconfig ) is enough for SUSE Technical Support. But getappcore(8) could be used to creates an archive containing the application core, and all files required to analyze it - including the binary which created the core, and all required shared libraries, plus RPM version information for further investigation by SUSE.

Run genappcore with core file path if using non-compressed core file:
 

getappcore <corefile>

of if using compressed core file then add path to binary of the process which crashed:
 

# coredumpctl info 1141
           PID: 1141 (sshd)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 6 (ABRT)
     Timestamp: Tue 2021-11-09 10:33:54 CET (18min ago)
  Command Line: sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
    Executable: /usr/sbin/sshd
 Control Group: /system.slice/sshd.service
          Unit: sshd.service
         Slice: system.slice
       Boot ID: 752fe79f770444d4892c84daea7409e2
    Machine ID: cf02db235794ef8c4a358db66183e547
      Hostname: 192
       Storage: /var/lib/systemd/coredump/core.sshd.0.752fe79f770444d4892c84daea7409e2.1141.1636450434000000.zst
       Message: Process 1141 (sshd) of user 0 dumped core.
                
                Stack trace of thread 1141:
                #0  0x00007f59bef5f2c6 __select (libc.so.6 + 0xf72c6)
                #1  0x0000556956cc4815 n/a (sshd + 0x10815)
                #2  0x00007f59bee8d34d __libc_start_main (libc.so.6 + 0x2534d)
                #3  0x0000556956cc5eea n/a (sshd + 0x11eea)


# getappcore -b /usr/sbin/sshd /var/lib/systemd/coredump/core.sshd.0.752fe79f770444d4892c84daea7409e2.1141.1636450434000000.zst 
####################################################################
Get Application Core Tool, v1.52.8_dev2
Date:     11/09/21, 10:53:37
Server:   192
OS:       SUSE Linux Enterprise Server 15 SP3
Kernel:   5.3.18-57-default (x86_64)
Corefile: /var/lib/systemd/coredump/core.sshd.0.752fe79f770444d4892c84daea7409e2.1141.1636450434000000.zst
####################################################################

Local configuration file... None
Validating core file... Done
Validating binary file... Done
Checking Source Binary with chkbin... Done
Building list of required libraries... Done
Building list of required RPMs... rpm: no arguments given for query
Done
Building list of debuginfo RPMs...  Done
Setting gdb environment variables... Done
Creating gdb startup files... Done
Creating core archive... Done

Affected Binary: /usr/sbin/sshd
Coredump File:   /var/lib/systemd/coredump/core.sshd.0.752fe79f770444d4892c84daea7409e2.1141.1636450434000000.zst
Archive Name:    /var/log/scc_192_sshd_211109_105337_appcore.txz
Removing required files and directories ... Done

Finished!

[750.396864] systemd-journald[574]: Failed to write entry (25 items, 104362632 bytes) despite vacuuming, ignoring: Argument list too long

• memory is insufficient, at least 2Gb of memory
• also adjust the /etc/systemd/journald.conf changing the below parameters and reboot:

RuntimeMaxUse=1G
RuntimeMaxFileSize=1G