How SUSE builds its Enterprise Linux distribution – PART 2

This is the second blog of a series in which we will provide some insight into SUSE Linux Enterprise product development. You will get a first-hand overview of SUSE, the SLE products, what the engineering team do to tackle the challenges coming from the increasing pace of open source projects, and the new requirements from our customers, partners and business-related constraints.

How SUSE builds its Enterprise Linux distribution:

• PART 1 – Introduction
• PART 2 – Linux Distribution
• PART 3 – Release Management
• PART 4 – SUSE Linux Enterprise Schedules
• PART 5 – openSUSE and SLE

Whether you are a long-term SUSE customer, a new SUSE customer, a SUSE partner, or just an openSUSE fan or open source enthusiast, we hope you will find these blogs interesting and informative.

GNU/Linux

The common understanding is that an Operating System is composed by a “kernel” and some basic tools around it. In short the OS is the interface between the computer hardware and the software but also provide a security layer to protect software from each others. This apply to all Operating Systems out there, not just Linux/Unix based ones.

Speaking about “Linux”, you might not be aware of the “GNU/Linux naming controversy“, where the name “Linux” refers precisely to the “Linux Kernel” and “GNU” refers to the basic components like GNU Compiler Collection (GCC), the GNU C library (glibc), and GNU Core Utilities (coreutils), GNU Bash shell and more. At this point, come to realise that the Linux Kernel and the GNU tools are in a “symbiosis“, and one cannot be used independently of the other. So it is technically more correct to refer to “GNU/Linux Operating System” than a “Linux Operating System”.

However when the words “Linux system or Linux server” is used, it often includes far more that just “GNU/Linux”. For instance, your preferred web server, database, programming language librairies or Graphical Environment (like GNOME) is not part of the “GNU/Linux” group, and this is where the name “Linux Distribution” makes sense.

Linux Distribution

Linux Distribution regroups GNU/Linux + all software, technically referred as packages, provided by your Linux vendors or communities and it could be described as a big block construction where each block is a package/software.

Of course all blocks are not equal, the Linux Kernel is the most essential of the system, but linked to smaller blocks (GNU and friends) that are critical for running a system. Which brings us to the inter-dependency of each block by function but also by code, which means that to a certain extend you can’t just change or modify some blocks of the system individually, without adaptation to all the dependency chain.

One particular software will strictly differentiate Linux Distribution from each others, it’s a basic and yet essential one: the package manager. This software will allow you to add, remove or updates packages for your entire Linux Distribution. This package manager usually supports a specific package format, the two most well known package format are RPM used by SUSE Linux Enterprise Server, openSUSE, Red Hat Enterprise Linux, Fedora, CentOS and Deb used by Debian and its derivatives Ubuntu, Linux Mint, etc.

That being said, your preferred Linux Distribution could be seen as a big open source project on it’s own, but in fact consist of a set of multiple open source projects. GNU, Linux, GNOME, Firefox but also hundreds of open source projects and packages are “glued” together to form a Linux Distribution.

Another big differentiator between Linux Distribution is the cherry-picking of packages versions in their releases. The vast majority of well known packages are used across Linux Distributions, but you might not find the same versions of it from one to the others.

Of course, “the devil is in the detail” and there is more than just package managers and the package versions to differentiate Linux Distribution. Just to name a few more examples: Installation process, configuration tools, supported architectures, licences used, configuration files, etc are important assets of a Linux Distribution.

SUSE Linux Enterprise Server

To give you some concrete numbers, SUSE Linux Enterprise Server has around 18 000 packages, and when developing a new SLES release we might have to update hundreds of packages, sometimes adding new ones or deprecating obsolete ones. And of course with each new SLES release, the system needs to be functional, includes bug fixes and adds/removes features which require a certain micro (packages) and macro (system) development effort. One note, and not a small one, SLES is supported on multiple hardware architectures, ranging from computers that you can lift with one finger to computers that no human can lift alone, weighing hundreds of kilograms!

Looking at the available CPU architectures, it changed over the years but we now support the 4 main architectures: AMD64/Intel 64 (x86-64), POWER (ppc64le), IBM Z (s390x) and ARM 64-Bit (AArch64).
Which means that SUSE Linux Enterprise Server should be built, released and supported on 4 different architectures.

It might depend on your environment and needs but usually the majority of people uses one or two architectures but no more. However the SUSE Linux Enterprise Team needs to build, test and support 4 “versions” of our Linux Distribution for each architectures. Remember the 18 000 packages of SLES? The majority of them need to be built, tested and supported on multiple platform before being released and available to our users.

Also important to note, we require to release any given SLES version on all supported architectures on the same date! Which can be seen as a challenge when we have to fix a particular issue specific to one of the architecture without delaying the entire SLES releases. During the development phase it is not uncommon to find an issue in a specific package only for a specific architecture!

Last but not least, we are not only supporting one SLES versions at a given time but two SLES versions, as of today we have SLES 12 and SLES 15.

We will talk about Release Management and Schedule in upcoming posts.

Further readings:

• https://www.gnu.org
• https://www.gnu.org/gnu/linux-and-gnu.en.html
• https://en.wikipedia.org/wiki/Comparison_of_instruction_set_architectures