tuned.service times out when saptune solution is applied on boot

This document (000019953) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server for SAP Applications 15 SP2
SUSE Linux Enterprise Server for SAP Applications 15 SP1
SUSE Linux Enterprise Server for SAP Applications 12 SP5
SUSE Linux Enterprise Server for SAP Applications 12 SP4

Situation

tuned.service timed out during boot while trying to apply a previously configured saptune solution.

# systemctl status tuned.service

● tuned.service - Dynamic System Tuning Daemon
   Loaded: loaded (/usr/lib/systemd/system/tuned.service; enabled; vendor preset: disabled)
   Active: failed (Result: timeout) since <date>
     Docs: man:tuned(8)
           man:tuned.conf(5)
           man:tuned-adm(8)
 Main PID: 137870 (code=exited, status=0/SUCCESS)

systemd[1]: Starting Dynamic System Tuning Daemon...
systemd[1]: tuned.service: Start operation timed out. Terminating.
systemd[1]: Failed to start Dynamic System Tuning Daemon.
systemd[1]: tuned.service: Unit entered failed state.
systemd[1]: tuned.service: Failed with result 'timeout'.

/var/log/tuned/tuned.log

2020-09-09 17:16:19,341 INFO     tuned.daemon.daemon: starting tuning
2020-09-09 17:16:19,363 INFO     tuned.plugins.plugin_script: calling script '/usr/lib/tuned/saptune/script.sh' with arguments '['start']'
2020-09-09 17:16:19.603 WARNING  saptune.ini.go:189: [block] section detected: Traversing all block devices can take a considerable amount of time.
2020-09-09 17:16:19.603 INFO     saptune.ini.go:250: Trying scheduler in this order: noop, none.
2020-09-09 17:16:19.603 INFO     saptune.ini.go:252: 'noop' will be used as new scheduler for device 'sda sdaa sdab sdac sdad sdae sdaf sdag sdah sdai sdaj sdak sdal sdam sdan sdao sdap sdaq sdar sdas sdat sdau sdav sdaw sdax sday sdaz sdb sdba sdbb sdbc sdbd sdbe sdbf sdbg sdbh sdbi sdbj sdbk sdbl sdbm sdbn sdbo sdbp sdbq sdbr sdbs sdbt sdbu sdbv sdbw sdbx sdby sdbz sdc sdca sdcb sdcc sdcd sdce sdcf sdcg sdch sdci sdcj sdck sdcl sdcm sdcn sdco sdcp sdcq sdcr sdcs sdct sdcu sdcv sdcw sdcx sdcy sdcz sdd sdda sddb sddc sddd sdde sddf sddg sddh sddi sddj sddk sddl sddm sddn sddo sddp sddq sddr sdds sddt sddu sddv sddw sddx sddy sddz sde sdea sdeb sdec sded sdee sdef sdeg sdeh sdei sdej sdek sdel sdem sden sdeo sdep sdeq sder sdes sdet sdeu sdev sdew sdex sdey sdez sdf sdfa sdfb sdfc sdfd sdfe sdff sdfg sdfh sdfi sdfj sdfk sdfl sdfm sdfn sdfo sdfp sdfq sdfr sdfs sdft sdfu sdfv sdfw sdfx sdfy sdfz sdg sdga sdgb sdgc sdgd sdge sdgf sdgg sdgh sdgi sdgj sdgk sdgl sdgm sdgn sdgo sdgp sdgq sdgr sdgs sdgt sdgu sdgv sdgw sdgx sdgy sdgz sdh sdha sdhb sdhc sdhd sdhe sdhf sdhg sdhh sdhi sdhj sdhk sdhl sdhm sdhn sdho sdhp sdhq sdhr sdhs sdht sdhu sdhv sdhw sdhx sdhy sdhz sdi sdia sdib sdic sdid sdie sdif sdig sdih sdii sdij sdik sdil sdim sdin sdio sdip sdiq sdir sdis sdit sdiu sdiv sdiw sdix sdiy sdiz sdj sdja sdjb sdjc sdjd sdje sdjf sdjg sdjh sdji sdjj sdjk sdjl sdjm sdjn sdjo sdjp sdjq sdjr sdjs sdjt sdju sdjv sdjw sdjx sdjy sdjz sdk sdka sdkb sdkc sdkd sdke sdkf sdkg sdkh sdki sdkj sdkk sdkl sdkm sdkn sdko sdkp sdkq sdkr sdks sdkt sdku sdkv sdkw sdkx sdky sdkz sdl sdla sdlb sdlc sdld sdle sdlf sdlg sdlh sdli sdlj sdlk sdll sdlm sdln sdlo sdlp sdlq sdlr sdls sdlt sdlu sdlv sdlw sdlx sdly sdlz sdm sdma sdmb sdmc sdmd sdme sdmf sdmg sdmh sdmi sdmj sdmk sdml sdmm sdmn sdmo sdmp sdmq sdmr sdms sdmt sdmu sdmv sdmw sdmx sdmy sdmz sdn sdna sdnb sdnc sdnd sdne sdnf sdng sdnh sdni sdnj sdnk sdnl sdnm sdnn sdno sdnp sdnq sdnr sdns sdnt sdnu sdnv sdnw sdnx sdny sdnz sdo sdoa sdob sdoc sdod sdoe sdof sdog sdoh sdoi sdoj sdok sdol sdom sdon sdoo sdop sdoq sdor sdos sdot sdou sdov sdow sdox sdoy sdoz sdp sdpa sdpb sdpc sdpd sdpe sdpf sdpg sdph sdpi sdpj sdpk sdpl sdpm sdpn sdpo sdpp sdq sdr sds sdt sdu sdv sdw sdx sdy sdz'.
2020-09-09 17:17:48,736 INFO     tuned.daemon.controller: terminating controller
2020-09-09 17:17:48,736 INFO     tuned.daemon.daemon: stopping tuning
2020-09-09 17:17:48,763 ERROR    tuned.plugins.plugin_script: script '/usr/lib/tuned/saptune/script.sh' error output: '    WARNING: [block] section detected: Traversing all block devices can take a considerable amount of time.'
2020-09-09 17:17:48,763 ERROR    tuned.plugins.plugin_script: script '/usr/lib/tuned/saptune/script.sh' returned error code: -15

Resolution

Some device handling optimizations have been ported back to saptune version 2 (Need to request a PTF) or one of the following workaround solutions can be used until saptune version 3 is released.

1. Increase the systemd timeout for the tuned.service by adding "TimeoutSec=900" in [Service] section.

# systemctl edit --full tuned.service

[Service]
.....>
TimeoutSec=900

or also

2. Add a start delay in the tuned.service file by adding "ExecStartPre=/bin/sleep 30" to [Service] section.

systemctl edit --full tuned.service

[Service]
.....>
ExecStartPre=/bin/sleep 30

Note: Editing a systemctl configuration file will copy current configuration into /etc/systemd/system/ directory and if it exists, systemd will use it; otherwise systemd normally loads configurations from /usr/lib/systemd/system/ directory.

Cause

Traversing through many block devices to set the I/O scheduler was time consuming.

Additional Information

man 8 saptune
man 1 systemctl
If --full is specified, this will copy the original units instead of creating drop-in files.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.