SLE-Micro in the Public Cloud

The time has arrived to get out of, what is probably one of the longest, “soft launch” periods, and make the availability of SLE-Micro images in the Public Cloud more widely known. I am taking the liberty of defining “soft launch” in this case as

• publish images
• don’t talk about the images
• let users discover the images and struggle with the things that they may not expect on their own

especially the last part is of course not very nice and I aim to put an end to it. Let’s take these topics one at a time.

Publish Images

SLE-Micro images have been available in AWS EC2, Azure, and GCE since version SLE-Micro 5.1. The most recent release is SLE-Micro 5.3 and we are working on version 5.4 to be published in the not too distant future. Image information is in pint and can therefore also be seen on https://pint.suse.com.

The life-cycle of the images follows the published image-life-cycle and is on a 3 months rolling release schedule. The SLE-Micro product has a different life-cycle than other SLE products, for example SUSE Linux Enterprise Server and SUSE Linux Enterprise Server For SAP Applications. As such stage transitions are different and we have more than 1 active image.

At present SLE-Micro is available as a BYOS option with a PAYG option on the horizon. This means you will need a SUSE Customer Center registration code to receive updates for SLE-Micro.

Don’t Talk About The Images

Well with this blog the not talking about the images period is finally over. Of course this could have happened some time ago, but the trouble with time is that it is a limited resource and …..

Struggling With Things That Are Different

Here I am hoping to provide sufficient guidance to make the use of SLE-Micro in the Public Cloud not a struggle. First and foremost SLE-Micro is very different than other SLE products. It is intended as a container host, i.e. a more focused implementation of an operating system built for a particular use case. As such underlying assumptions are different for the implementation of the distribution than for the general purpose SUSE Linux Enterprise Server distribution. As with our other images for other products our goal is to modify the underlying distribution as little as possible for the use in the various cloud frameworks. This goal serves the idea that users using SLE-Micro in their Data Center or on other devices, such as on the Edge, will encounter, as much as possible, the same behavior of the distribution in the Cloud.

For details about administering SLE-Micro please see the administration guide.

With this as background let’s start with highlighting differences for those that may be used to using SLES, SLES For SAP, or SLE HPC images in the Public Cloud. SLE-Micro has a read-only root file system that uses btrfs. The use of btrfs enables transactional-updates which means updates are atomic, i.e. the running system is not modified when new packages are installed or installed packages are updated. Rather a new snapshot is created and the updates to the system are installed into the new snapshot. The updates are activated by a reboot.

Transactional-updates have a number of nice features as highlighted in the documentation. The requirement to reboot the system for every update may appear as a hindrance at first sight. This is where the purpose built nature of SLE-Micro shines through. In a world where SLE-Micro is used as a container host the reboot is actually not an issue. The containers on the host can reasonably easily be moved to a different host, the host gets rebooted and then the containers are moved back. Of course in the Cloud the updates -> evacuate -> reboot -> re-populate cycle is an approach that is somewhat counter intuitive. A start-new-instance-from-new-image -> integrate into cluster -> kill-old-instance cycle does away with the update/upgrade of a running system and may feel more “cloud native”. Anyway, both approaches have their merit and people will probably argue over the pros and cons of each approach for years to come.

Another major difference for those used to using other SLE based images is the instance initialization. For SLE-Micro this happens with Ignition and Afterburn. This implies that adding existing cloud-init user data on instance start up will not have any effect. Also the initial/default user for the system is different, compared to other SLE product images. On all CSPs the user is “suse“. The choice of a consistent user name across all cloud frameworks was partially driven by the idea to create uniformity to aid with automatic integration of instances into running clusters without having to make CSP (Cloud Service Provider) specific adjustments.

In the Ignition/Afterburn combination of instance initialization user creation is the responsibility of Ignition. While ignition accesses the Cloud provider instance metadata service it does so for the purpose of consuming so called user data. Ignition does not parse additional data from the metadata service, as such a user specified in Azure via the web-console or the cli does not get created by ignition. This is also the reason that an ssh connection from the GCE web-console cannot be established to a SLE-Micro instance, by default.

In order to create additional users an ignition configuration file needs to be passed as user data through the metadata service, for example

{
    "ignition": { "version": "3.1.0" },
    "storage": {
        "filesystems": [
            {
                "path": "/home",
                "device": "/dev/disk/by-label/ROOT",
                "format": "btrfs",
                "wipeFilesystem": false,
                "mountOptions": [
                    "subvol=/@/home"
                ]
            }
        ]
    },
    "passwd": {
        "users": [
            {
                "name": "foo",
                "uid": 1100
            }
        ]
    }
}

will create the additional user named “foo” in the instance. This will only create the user, you will have to specify an ssh key to login. The ssh key used to start the instance and passed via the instance metadata service (IMDS) will be setup for the “suse” user by Afterburn. The reason for this is that the image automatically starts the systemd afterburn-sshkeys@suse service and it is this service which is part of Afterburn that transfers the ssh key from IMDS to the created user’s home directory.

The pre-configured “suse” user is setup via the ignition configuration file /usr/lib/ignition/base.d/base.ign. The use of this file is triggered by a dracut module, /etc/dracut.conf.d/85-ign-usr-config.conf. The “suse” user is also setup to have sudo access via /etc/sudoers.d/builtin. Meaning once you ssh to your SLE-Micro instance as the “suse” user you can switch to the “root” user via the

sudo -i

command.

User configuration via ignition only happens on first boot. In order to differentiate first boot from every other boot a flag file is written, /boot/writable/firstboot_happened.

It is very likely that our default user configuration does not match with what you would like to see. You have a couple of options to make the system setup conform to your liking.

Using the useradd command

There is a reasonable chance that you are familiar with the useradd command. That said, since SLE-Micro is btrfs based, and as I stated we try to provide the same behavior as much as possible than in other environments. Therefore, in the Public Cloud images the filesystem layout is the same as documented in the SLE-Micro Deployment Guide. This means the /home directory is in a subvolume and as such the useradd command must be used accordingly

useradd --btrfs-subvolume-home --create-home bar

for example to add the user bar to the system. The /etc directory is mounted as an overlayFS and can be edited, as such if you want the newly added user to have root access via sudo you need to make the necessary configuration changes as you would on any other SLE system. These changes take effected immediately, i.e. no reboot is required.

You can also get rid of the “suse” user. Be mindful to set up another user to have “sudo” access or you will loose your access to administer the system.

You may also modify the default ignition configuration file. Since the root file system is read-only you will need to make changes using the

transactional-update shell

command. This will create a new snapshot that is mounted read-write and you can make changes as you see fit. Once you exit the shell, the transactional-update code will set the created snapshot to read-only and will make it the default snapshot for the next boot. In order to have those changes applied on the next reboot the configuration file also needs to be built into the initrd since ignition and afterburn both run from the initrd. To re-build the initrd run

transactional-update initrd

. If you want the newly created user to have access via the framework provided ssh key you also need to make sure that

systemctl start afterburn-sshkeys@$YOUR_NEW_USERNAME

runs in order to ingest the shh key from the IMDS into the instance for the newly created user. Once you are done with that you can reboot. Note that anytime the transactional-update command is used to modify the system, i.e. via transactional-update up or transactional-update shell a new snapshot is created. To keep modifying 1 snapshot use the –continue option.

What else?

As with other SLE images the root volume is set to the size specified by our partners, i.e. 10 GB in AWS and GCE and 30 GB in Azure. The recommended minimum size for the root volume of SLE-Micro is 12 GB. The size you choose for the root volume is of course dependent on how many containers you think you will have on the system and how big those containers are. Also every time you update the system, i.e. transactional-update up, a snapshot will be created. While btrfs snapshots are space efficient they still take up space. The root volume will automatically grow as you change the size of the underlying disk just as it does with other SLE images. This means when you notice that your are running low on disk space you can always stop your instance and make the system disk bigger, up to the limitation of individual disk sizes in each Cloud framework.

At this time the container runtime in the Public Cloud images is docker. This helps with the integration into kubernetes clusters. That said, as has been well publicized, kubernetes has dropped docker support upstream and as such this change is making it’s way into the managed kubernetes environments, AKS, EKS, GKE. Therefore, at some point in the future this default setup will change and this blog will be updated accordingly. dockerd is running by default and docker pull and docker run will work out of the box.

SLE-Micro also supports podman and systemd-container if you want/need to switch the used container runtime on the host to make it fit in with your environment. Both runtimes are available in the repositories. Which brings me to the topic of registration. You can use the SUSEConnect command to register your instance with your SUSE registration code to the SUSE Customer Center (SCC) or you can use registercloudguest, as outlined here to use the SUSE operated Public Cloud update infrastructure as a proxy to SCC.

You may also ask yourself whether or not the internal snapshots btrfs keeps have any relation or interfere with the external snapshots of the disk that can be created using the Cloud framework tools for backup purposes or to create new images. The answer is no. There is no relationship to the external snapshots that can be created through the framework tools and image creation works as expected.

As you read the SLE-Micro documentation you will also run across an initialization option called Combustion. Combustion is not included in the Public Cloud images. While the “execute anything in a script” feature of combustion provides basically unlimited flexibility combustion does not have the ability to read from IMDS. As such in order to configure your instance with combustion it would be required to create a separate disk in the framework, set it up to meet the necessary directory structure and then create an instance with this configuration disk attached. If the use of combustion is needed for integration into your environment you can build your own images using keg and keg-recipes. The keg-recipes project contains the image description SUSE uses to build the images we publish. Adding the combustion package to create a custom image description should be reasonably straight forward.

Another topic that you will find in the documentation is system registration. For the cloud instances you do not need to use the transactional-update command to register the system. As mentioned previously SUSEConnect or registercloudguest are the tools you want to use. You can use the transactional-update command but it will always connect you to the SUSE Customer Center (SCC). Not all images have the registercloudguest command available as it is a feature that was added only recently.

Update 2023-02-09

And of course I forgot something in my initial post. In Azure it is not possible to install extensions. The image does not include the Azure agent as extensions are pulled from the Azure backend and installed as executables on the root filesystem. However, as mentioned previously the root filesystem is read only and as such that mechanism does not work.

End update 2023-02-09

And with this the “secret” is out of the bag and I hope that highlighting the differences to other SLE images is helpful when you want to deploy container based applications on SLE-Micro in the Public Cloud.