Using the OpenSUSE Build Service to Create and Distribute Kernel Module Packages

Using the OpenSUSE Build Service to Create and Distribute Kernel Module Packages

1. Introduction
2. Scope
3. What is OBS?
4. Using OBS to Create KMPs
4.1 Code Preparation
4.1.1 Example
4.2 Creating an OBS Project
4.2.1 Example
4.3 Defining Target Distributions
4.3.1 Example
4.4 Adding Packages to a Project
4.4.1 Example
4.5 Setting Target Distributions for a Package
4.5.1 Example
4.6 Building KMPs
4.6.1 Example
4.7 Publishing KMPs
4.7.1 Example
5. Using OBS Repositories
5.1 Example
6. Summary
Appendix A – Sample Files
Appendix B – Building for Red Hat Enterprise Linux (RHEL)
References

1. Introduction

As Linux gains market-share in the server, desktop, and laptop operating system space, demand for Linux drivers and kernel modules is increasing. Independent hardware and software vendors (IHVs and ISVs) can expand their customer bases significantly by providing proper kernel-space support for their products.

Novell promotes having all Linux drivers in the official kernel.org source tree. However, Novell also recognizes that sometimes IHV and ISV partners do need to provide drivers separate from the Linux kernel. Novell encourages partners who need to provide such external driver(s) to use specially-built rpms called Kernel Module Packages (KMPs). By using KMPs, partners can ensure that their modules will be installed appropriately and handled correctly during OS updates..

Novell’s Partner Linux Driver Process web site¹ provides full instructions for building KMPs on one’s local development systems. But Novell also provides another way to build packages: the openSUSE Build Service (OBS). OBS allows developers to build rpms (including KMPs) remotely on hardware maintained by Novell. By using OBS, partners can 1) build KMPs for multiple architectures and distributions, and 2) easily provide KMPs in repositories that customers can access and that interact well with the SUSE Linux installation and update utilities.

2. Scope

This document explains how to use OBS to build KMPs for the Code 11 SUSE Linux distributions. However, Appendix B includes information on how to use OBS to build KMPs (from the same source code and packaging files) for Red Hat Enterprise Linux (RHEL) as well.

With respect to the OBS web and command-line interfaces, this document only describes the specific steps and commands required to build a KMP. For complete information on the OBS web and command-line interfaces, see the full OBS user documentation².

This document does not provide information about developing or porting driver source code.

3. What is OBS?

The openSUSE Build Service is an open and complete distribution development platform which provides software developers with tools to compile, release and publish their software for different Linux distributions and hardware architectures2. Novell/SUSE’s development teams use OBS to build packages for openSUSE, SLES, SLED, etc. For the purposes of this document, the term “OBS” refers to both the web interface as well as the systems used to perform actual builds.

There are a number of benefits to building packages using OBS rather than local development or build systems. Developers who build locally must install physical and/or virtual machines for each architecture and distribution they wish to support. Obtaining and maintaining such a large number of build systems can be expensive and cumbersome. By using OBS, developers can concentrate on developing code and offload much of their build work to the automated OBS processes.

Another benefit to using OBS is the ability to easily create repositories and publish packages. Traditionally, developers have had to build their packages then work with their distribution and/or IS&T departments to provide those packages to customers. But OBS takes care of the distribution work: after building, OBS places packages in automatically-created YUM repositories that SUSE Linux (and other Linux distributions) can use as installation and update sources.

Even given the above benefits, it is important to note that OBS is not suited to all development projects. Partners are encouraged to consider the following points before using OBS to build KMPs:

• OBS is essentially public: Any source code uploaded to OBS is visible and available to all other persons with OBS logins.
• Source code uploaded to build KMPs must be licensed under the GPL in order to comply with legal and policy requirements.^3,4 OBS must not be used to build non-GPL kernel modules.
• Partners should not use OBS to mark their modules as “supported: external” without prior explicit approval from Novell. (The privilege of marking modules as “supported external” is reserved for partners with whom Novell has L3-level support agreements.)
• Currently, OBS does not provide a means to build for specific SLE service pack versions or security/maintenance updates. OBS only supports building for the first customer ship version of each major SLE release.
• Currently, OBS only supports building packages for ix86 and x86_64 architectures.

4. Using OBS to Create KMPs

The following sections detail the basic steps to follow when using OBS to build and publish a KMP. Each sub-section describes a particular step in the process, then provides a hands-on example. The example in each sub-section builds upon the example from the previous sub-section.

4.1 Code Preparation

OBS creates packages from source code and packaging files. Therefore, before using OBS to build a KMP, a partner needs to have source code that is ready to be packaged: development should be complete, the code should build as an external module in compliance with the information in /usr/src/linux/Documentation/kbuild/modules.txt, and the resulting module(s) should function as desired. As a simple test, if a partner’s source code builds by simply issuing a “make -C <target-kernel-source-directory> M=`pwd`” command in the source code’s top-level directory, then it should be very straightforward to package the code into a KMP.

4.1.1 Example

The example used in this and following sections is structured around the common use case of a system vendor needing to provide an external driver to support SLE 11 on a particular system. To work through the example, assume the following:

• The system to be supported is “samplesystem”.
• The driver to be provided is “sampledriver”.

To complete the “Code Preparation” portion of the example:

• Create a “sampledriver” directory in a work location on your development system. E.g.:

  mkdir -p /work/sampledriver
  
  

• Create a “sampledriver-1.0” directory in the sampledriver directory:

  mkdir /work/sampledriver/sampledriver-1.0
  
  

• In the sampledriver directory, create a sampledriver.spec file by copying and pasting the text from the sampledriver.spec file in Appendix A.
• In the sampledriver-1.0 directory, create a sampledriver.c file by copying and pasting the text from the sampledriver.c file in Appendix A.
• In the sampledriver-1.0 directory, create a Kbuild file by copying and pasting the text from the Kbuild file in Appendix A.
• Your directory structure should now look like:

  work
      sampledriver
         sampledriver.spec
         sampledriver-1.0
               sampledriver.c
               Kbuild
  
  

• From within the sampledriver-1.0 directory, test building this source code by issuing the command

  make -C /usr/src/linux M=`pwd`
  
  

  and verifying that it creates a sampledriver.ko file. Note: In order for this make command to succeed, /usr/src/linux should be the top-level directory of a SLE 11 kernel source tree which has been prepared for module build in accordance with the instructions in section 2.4 of /usr/src/linux/Documentation/kbuild/modules.txt.

• From within the sampledriver directory, create a tarball of the sampledriver-1.0 directory:

  tar cvjf sampledriver-1.0.tar.bz2 sampledriver-1.0
  
  

4.2 Creating an OBS Project

The next step in using OBS to build a KMP is to create an OBS project.

An OBS project is basically an abstract container for packages that are related in some way. OBS projects may contain sub-projects to further categorize packages. OBS projects and sub-projects correspond to how repositories are presented to users.

OBS projects can be created at the root level or in existing OBS project containers such as “home” or “isv”. The root level is generally reserved for official community projects, the “home” container is intended for individual developers to use as a testing area, and the “isv” container is where Novell encourages partners to create their projects.

Note that any OBS project repository URL contains the name of its associated project. For this reason, Novell encourages partners to establish project names that reflect their organizations. E.g., a Novell partner with the name “AAA Computers” might wish to establish the project “aaa-computers” in the “isv” container. The URL for a SLE 11 repository associated with the “aaa-computer” project would be http://download.opensuse.org/repositories/isv://aaa-computers/SLE_11.

While no authorization is needed to create projects in the “home” container, projects in the “isv” container are more controlled. Partners who wish to create a project in the “isv” container should post a request to the opensuse-buildservice mail list2 or contact their Novell representative.

IHV/ISV partners can certainly use their main projects for all their KMPs. However, often partners have sets of packages that apply to different hardware and software configurations. For instance, a partner “AAA Computers” might wish to provide one set of drivers for “Peripheral-A” and another set of drivers for “Peripheral-B”. The partner could use a main “isv:aaa-computers” project for both the Peripheral-A and Peripheral-B drivers… with the result that all the KMPs would be provided in a single repository (e.g., http://download.opensuse.org/repositories/isv://aaa-computers/SLE_11 ). A more useful approach might be for AAA Computers to create two sub-projects: “isv:aaa-computers:peripheral-a” and “isv:aaa-computers:peripheral-b”, each containing its own drivers and each providing its own repository.

4.2.1 Example

To create the samplesystem sub-project using the OBS web interface:

• Login to http://build.opensuse.org using your Novell login. (The http://build.opensuse.org website also provides a link to obtain a Novell login if necessary.)
• On the main OBS page, click the “Home Project” link to display your OBS home:<loginname>:samplesystem project page.
• Click to create a subproject. Name the subproject “samplesystem”. Enter and save the appropriate information, then OBS should display your samplesystem subproject page.

4.3 Defining Target Distributions

Once an OBS user has created a project, he/she can define the target distributions that apply to the project. A project should include a distribution as a target if any of the packages in the project will — or even may — need to be built for that distribution. Whether a particular package is to be built for a particular target distribution can be specified on a package level; the project should include all distributions that might be needed.

Note that OBS refers to a project’s target distributions as “Build Repositories”: collections of packages needed to build project components against a particular target. With respect to OBS projects, there are the Build Repositories that users select as target distributions (i.e., repositories used during build)… and there are the Build Repositories that OBS creates to provide the project’s built packages. Any Build Repositories that OBS creates to provide a project’s built packages can be selected as target distribution repositories for other projects.

4.3.1 Example

To set target distributions for your OBS samplesystem project:

• On the OBS samplesystem subproject page, use the “Build Repositories” section to add SLE_11 and RHEL_5. (While the bulk of this document discusses building for SLE 11, Appendix B describes how to use the same build structure and source code to build for RHEL 5 as well.)

4.4 Adding Packages to a Project

OBS projects can contain one or more packages, but the name of each package must be explicitly specified on the OBS project page before uploading any source code. (Using the OBS project page to add a package name is analogous to adding a directory which will hold the package’s source code and packaging files.) Once an OBS user adds a package name to his project, he can upload his source code (generally in the form of a tarball). As noted in section 4.1 above, partners who wish to build KMPs should ensure that their source code is structured in accordance with /usr/src/linux/Documentation/kbuild/modules.txt.

Uploading module source code (including Kbuild files and/or Makefiles) provides OBS with the information needed to create the pertinent kernel modules. But to create rpms, OBS also needs package meta-data provided via a *.spec file.

The basic spec file template for building SLE 11 KMPs is available in the Kernel Module Packages Manual – Code 115. By using this template as a basis to create his own spec file, an IHV/ISV can ensure that his modules will be installed and configured to interact correctly with SUSE Linux. The spec file in Appendix A is taken from the KMP spec file template and modified to apply to the “sampledriver” package.

4.4.1 Example

To add a sampledriver package to your samplesystem project:

• On the OBS home:<loginname>:samplesystem project page, click the appropriate link to add a package. Name the package “sampledriver”. Complete the required fields and save the data, then OBS will display the sampledriver package page.

Now that the sampledriver package exists, source code and build files can be uploaded to it. Uploading files to OBS can be done using either the OBS web-interface or command-line client. To upload the sampledriver files using the OBS web-interface:

• In the “Files” section, click to add a file to the sampledriver package. Upload the sampledriver-1.0.bz2 tarball from the /work/sampledriver directory.
• In the “Files” section, click to add a file to the sampledriver package. Upload the sampledriver.spec file from the /work/sampledriver directory.

To upload the sampledriver files using the OBS command-line client:

• Install the “osc” package from the SLE 11 SDK (available via https://download.suse.com). The osc package provides the “osc” utility, which is used to interact with OBS. (Note: Readers familiar with the Subversion Version Control System may find it useful to analogize the “osc” command to the “svn” command. Use the “osc help” command to view options.)
• From a staging area on your development system (e.g., /work), check out the sampledriver package from the home:<loginname>:samplesystem project:

  co home:<loginname>:samplesystem sampledriver

• The above command will create the following directory structure:

  work
  		home:<loginname>:samplesystem
  			.osc
  			sampledriver
  

• Copy the sampledriver.spec and sampledriver-1.0.tgz files from /work/sampledriver into the empty /work/home:<loginname>:samplesystem/sampledriver directory:

  	cd /work/home:<loginname>:samplesystem/sampledriver
  	cp /work/sampledriver/* .
  
  

• Upload (check in) the sampledriver source code and build files to OBS:

  	osc add *
  	osc ci
  
  

4.5 Setting Target Distributions for a Package

By default, a package is “enabled” to build for all the target distributions associated with the package’s parent project. However, OBS users can override this default and disable building a specific package for a particular target distribution.

4.5.1 Example

To set the target distributions for your sampledriver package:

• On your OBS sampledriver package page, click the appropriate links in the “Package Flags” section to specify that the package should be built for SLE 11 only (i.e., disable building for RHEL 5).

4.6 Building KMPs

When an OBS package is enabled to build on a specific target distribution, the package will be built/rebuilt automatically whenever updates are made to the package or to the target distribution. However, OBS users can also manually trigger builds for each distribution by clicking on appropriate items in the “Build Status” section of the OBS package page or by using the OBS command-line client.

4.6.1 Example

To build your sampledriver package using the OBS web interface:

• In the “Build Status” section of the OBS sampledriver page, click on the SLE_11 icon to trigger rebuilds for this target distribution.
• Refresh the page (by using the “Build Status” refresh icon) until the status changes for the SLE_11 sampledriver builds.

To build your sampledriver package using the OBS command-line interface:

• Change directory to the directory containing the home:<loginname>:samplesystem project (e.g., /work or other staging location created in section 4.1.1).
• Tell OBS to rebuild the sampledriver package (for all enabled target distributions):

  osc rebuildpac home:<loginname>:samplesystem sampledriver

• After issuing the above command, view the status from your build request:

  osc results home:<loginname>:samplesystem sampledriver

4.7 Publishing KMPs

One of the great benefits that partners gain by using the openSUSE Build System is the ability to provide not just packages (or in this case, KMPs), but also repositories that end-users can access to install the KMPs. OBS makes it very simple to provide a repository that is fully compatible and integrated with the SUSE Linux installation and update utilities.

By default, OBS creates and publishes a separate repository for each of a project’s target distributions. The OBS project page “Goto Repository” link for each target distribution provides an easy way to view the actual repositories. Alternatively, users can access each repository directly by using the URL http://download.opensuse.org/repositories/home://loginname>/<sub-project-name>/<target-distribution>/ . Note that while publishing is enabled by default, OBS users can enable or disable publishing for specific repositories as desired.

4.7.1 Example

To view the repositories for the samplesystem subproject:

• Access the OBS home:<loginname>:samplesystem project page. In the “Build Repositories” section, click the “Goto Repository” link for the SLE_11 target distribution. Navigate into the separate architecture directories to view the built sampledriver KMPs.

5. Using OBS Repositories

Once a partner makes KMPs available in an OBS repository, anyone (partners, customers, etc.) can add the repository URL as an installation/update source from which to install the KMPs. There are several common use cases in which OBS repositories might be used:

• A system integrator might wish to sell pre-loaded systems that include the OBS repository as a pre-configured installation/update source. In this case, registering the repository (and installing appropriate initial KMP(s) from the repository) would be done by system integrator as part of the system installation process. After customers purchase the systems, they might access the repository to get updated KMPs.
• An IHV might wish to provide a KMP to support a particular hardware product. In this case, the partner might provide customers with documentation that details the repository URL and explains how to register the repository and install KMPs from it.
• A partner who creates a SLE-based appliance might wish to include an OBS repository as an installation/update source. In this case, the process of registering the repository (and installing appropriate initial KMP(s) from the repository) would be part of the appliance build process.

5.1 Example

To add the OBS samplesystem repository and install the sampledriver KMPs from it:

• Install a system with SLES/SLED 11. Using YaST2->”Software Repositories” or YaST2->”Add-on Product”, add the OBS SLE 11 samplesystem repository by specifying the URL http://download.opensuse.org/repositories/home://<loginname>/samplesystem/SLE_11/ .
• Using YaST2->”Software Management”, click to show packages in the samplesystem repository, then install the desired KMP(s).

6. Summary

IHVs and ISVs who provide drivers and other kernel modules face some unique development challenges in ensuring that their products will interact correctly with the SUSE Linux operating environments. By using Novell’s openSUSE Build Service, these partners can greatly simplify the process to build, package and distribute their kernel-space products.

Appendix A – Sample Files

The following sampledriver.c and Kbuild files may be used to build the sampledriver.ko module referenced in Section 4 above. The following sampledriver.spec file is based on the spec file template in the Kernel Module Packages Manual – Code 11⁵.

sampledriver.c

 * sampledriver.c - A demo driver
 *
 * Copyright (C) 2009
 * Ann Davis <andavis@novell.com>, Novell Inc.
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License as
 * published by the Free Software Foundation.
 *
 * A copy of the GNU General Public License can be obtained from
 * http://www.gnu.org/.
 *
 */

#include <linux/module.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
#include <linux/vmalloc.h>

MODULE_AUTHOR("Ann Davis <andavis@novell.com>");
MODULE_DESCRIPTION("Sample driver");
MODULE_LICENSE("GPL");

static struct proc_dir_entry *proc_entry;
static char *sampledriver_message = "Hello from sampledriver.";

int sampledriver_read(char *page, char **start, off_t off, int count, int *eof, void *data)
{
	int retVal = 0;

        retVal = sprintf(page, "%s\n", sampledriver_message);

        return retVal;
}

int __init init_sampledriver(void)
{
	int retVal = 0;

	printk(KERN_INFO "In sampledriver init function...\n");
	proc_entry = create_proc_entry("sampledriver_info", 0644, NULL);
	if (proc_entry == NULL) {
		printk(KERN_INFO "sampledriver:  Unable to create proc entry.\n");
		retVal = -ENOMEM;
	}
	else {
		proc_entry->read_proc = sampledriver_read;
	}

	return retVal;
}

void __exit exit_sampledriver(void)
{
	printk(KERN_INFO "In sampledriver exit function...\n");
	remove_proc_entry("sampledriver_info", NULL);
}

module_init(init_sampledriver);
module_exit(exit_sampledriver);

Kbuild

obj-m += sampledriver.o

sampledriver.spec

#norootforbuild

Name:			sampledriver
BuildRequires:	%kernel_module_package_buildreqs
License:		GPL
Group:			System/Kernel
Summary:		Sample Driver
Version:		1.0
Release:		0
Source0:		%name-%version.tar.bz2
BuildRoot:		%{_tmppath}/%{name}-%{version}-build

%kernel_module_package

%description
This package contains the sampledriver.ko module.

%prep
%setup
set -- *
mkdir source
mv "$@" source/
mkdir obj

%build
for flavor in %flavors_to_build; do
        rm -rf obj/$flavor
        cp -r source obj/$flavor
        make -C %{kernel_source $flavor} modules M=$PWD/obj/$flavor
done

%install
export INSTALL_MOD_PATH=$RPM_BUILD_ROOT
export INSTALL_MOD_DIR=updates
for flavor in %flavors_to_build; do
        make -C %{kernel_source $flavor} modules_install M=$PWD/obj/$flavor
done

%changelog
Mon Oct 19 2009 – andavis@novell.com
- Initial version.

Appendix B – Building for Red Hat Enterprise Linux (RHEL)

Red Hat and Novell have participated with other Linux community members in the Linux Foundation’s efforts to establish cross-vendor standards for building, installing, and distributing drivers.⁶ While differences still exist between distributions, partners can now create SLES kernel-module-package build structures that need very few (or even no) modifications to build correctly for RHEL..

Because Red Hat’s Driver Update Program⁷ spec file template is very similar to Novell’s, the sampledriver code provided in Appendix A requires only a minor modification to build appropriately for RHEL 5: While the SLE install location for external modules is /lib/modules/<kernel-version>/updates, the RHEL 5 install location is /lib/modules/<kernel-version>/extra/%{name}. Fortunately, OBS has the ability to perform distro-specific functions (see http://en.opensuse.org/Build_Service/cross_distribution_package_how_to ), so there is no need to create a separate OBS sampledriver package (or even a separate sampledriver.spec file) for RHEL. All that is required to build for both RHEL and SLE is to modify the sampledriver.spec file as noted below, then enable both the RHEL and SLE target distributions for the OBS home:<loginname>:samplesystem project.

sampledriver.spec (modified to build for both SLE and RHEL)

#norootforbuild

Name:			sampledriver
BuildRequires:	%kernel_module_package_buildreqs
License:		GPL
Group:			System/Kernel
Summary:		Sample Driver
Version:		1.0
Release:		0
Source0:		%name-%version.tar.bz2
BuildRoot:		%{_tmppath}/%{name}-%{version}-build

%kernel_module_package

%description
This package contains the sampledriver.ko module.

%prep
%setup
set -- *
mkdir source
mv "$@" source/
mkdir obj

%build
for flavor in %flavors_to_build; do
        rm -rf obj/$flavor
        cp -r source obj/$flavor
        make -C %{kernel_source $flavor} modules M=$PWD/obj/$flavor
done

%install
export INSTALL_MOD_PATH=$RPM_BUILD_ROOT
%if 0%{?rhel_version} == 501
export INSTALL_MOD_DIR=extra/%{name}
%else
export INSTALL_MOD_DIR=updates
%endif
for flavor in %flavors_to_build; do
        make -C %{kernel_source $flavor} modules_install M=$PWD/obj/$flavor
done

%changelog
* Tue Oct 20 2009 – andavis@novell.com
- Add RHEL support.
* Mon Oct 19 2009 – andavis@novell.com
- Initial version.

References

• [1] SUSE SolidDriver/Partner Linux Driver Process (PLDP). Project website at https://drivers.suse.com/doc .
• [2] openSUSE Build Service (OBS). Documentation website at https://build.opensuse.org/documentation/obs . Forums website at http://forums.opensuse.org/opensuse-build-service-obs/ .
• [3] Free Software Foundation. Licenses website at http://www.gnu.org/licenses .
• [4] Linux Foundation. Kernel Driver Statement. Website at https://www.linuxfoundation.org/en/Kernel_Driver_Statement .
• [5] SolidDriver/Partner Linux Driver Process, Andreas Gruenbacher et al. Kernel Module Packages Manual for Code 11. https://drivers.suse.com/doc/kmpm .
• [6] Linux Foundation. Driver Backport Workgroup. Website at http://www.linuxfoundation.org/collaborate/workgroups/driver-backport .
• [7] Red Hat, Inc. Driver Update Program. Website at http://dup.et.redhat.com/ .