Rancher Controller for the Kubernetes Ingress Feature

Alena is a principal software engineer at Rancher Labs.
Rancher has supported Kubernetes as one of our orchestration framework
options since March 2016. We’ve incorporated Kubernetes as an essential
element within Rancher. It is integrated with all of the core Rancher
capabilities to achieve the maximum advantage for both platforms.
Writing the Rancher ingress controller to backup the Kubernetes Ingress
feature is a good example of that. In this article, I will give a
high-level design overview of the feature and describe what steps need
to be taken to implement it from a developer’s point of view.

What is the Ingress Resource in Kubernetes?

When it comes to orchestration for an app that has to perform under
heavy load, the Load Balancer (LB) is a key feature to be considered.
Three key LB functionalities will be of significant interest for a
majority of the apps that would be involved:

• Distributing traffic between backend servers – key feature of the
  LB.
• Providing a single entry point for users accessing the app.
• Ability to pick a particular LB route based on the target domain
  name and/or URL path as it applies to L7 balancers

The L7 LB functionality enables support for routing multiple domains (or
subdomains) to different hosts or clusters.
The Kubernetes Ingress resource allows you to define LB routing rules in
a clear and user friendly way. Ingress for host based routing is shown
below:
The Ingress for path based routing:
Most importantly, Kubernetes Ingress can be backed up by any other Load
Balancer in your configuration. However, the question that remains to be
answered is, how do you bridge the Ingress resource and your LB? You
would have to write something called an Ingress Controller which is an
app that will have to perform the following functions:

• Listen to Kubernetes server events.
• Deploy Load Balancer and program it with the routing rules defined
  in Ingress.
• IMPORTANT: configure Ingress “Address” field with the Public
  Endpoint of your Load Balancer.

Assuming the Ingress controller is up and running, and you are a
Kubernetes user that wants to balance traffic between apps, this is what
you need to do:

• Define an Ingress
• Wait for the Ingress controller to do its magic and fill in the
  Ingress Address field.
• Use the Ingress Address field as an entry point for the LB. Voila!

Rancher Ingress Controller

Load Balancer service has always been a key feature in Rancher.
We’ve continued to invest into its feature growth. It has support for
Host name routing, SSL offload and can be horizontally scaled. So,
choosing Rancher LB as a provider for Kubernetes Ingress doesn’t
require any changes to be done on the Rancher side. All that need be
done is to write the Ingress controller. Rancher Ingress controller is
a containerized microservice written in golang. It is deployed as a
part of the Kubernetes system stack in Rancher. That means it will have
access to both the Rancher and Kubernetes API servers:
The diagram below provides high level architecture of the Rancher
Ingress controller:
The controller from the diagram above would:

1. Listen for Kubernetes events like:

• Ingress Create/Remove/Update
• Backend service Create/Remove/Update
• Backend services’ endpoints Create/Remove/Update

1. Generate a Load Balancer config based on the information retrieved
   from Kubernetes
2. Pass it to the LB provider

The provider would

1. Create Rancher Load Balancer for every newly added Kubernetes Ingres
2. Remove Rancher Load Balancer when corresponding Ingress gets removed
3. Manage Rancher Load Balancer based on changes in the Ingress
   resource:

• Add/Remove/Update Rancher LB host name routing rules when Kubernetes
  Ingress changes
• Add/Remove backends when newKubernetes Services get defined/removed
  in Kubernetes Ingress
• Add/Remove backend servers when Kubernetes Services’ endpoints get
  changed

1. Retrieve Public Endpoint set on Rancher Load Balancer, and pass it
   back to controller.

• The controller would make an update call to Kubernetes Ingress
  to update it with the end point returned by the provider:

The provider would
constantly monitor Rancher Load Balancer, and in case the LB instance
dies and gets recreated on another host by Rancher HA, the Kubernetes
Ingress endpoint will get updated with the new IP address.

LB Public Endpoint and Backend Services’ Endpoints

I’ve used these terms quite a lot in the previous paragraphs, but need
to elaborate on them a bit more. What is a LB public endpoint? It’s an
IP address by which your LB can being accessed. In the context of this
blog, it is the IP address of the Host where the Rancher LB container is
deployed. However, it can also be multiple IP addresses if your LB is
scaled cross multiple hosts. Public endpoints get translated to the
Kubernetes Ingress Address field as shown below.
But, what acts as the real backend Kubernetes service endpoint which
is the backend IP address where LB is supposed to forward traffic to? Is
it the address of the host where the service is deployed? Or is it a
private IP address of the service’s container? The answer is that
it depends on the Load Balancer implementation. If your Ingress
controller uses an external LB provider (like for example GCE
ingress-controller does),
it would be the IP address of the host, and your backend Kubernetes
service has to have ”type=NodePort“. But, if your LB provider is
deployed in the Kubernetes cluster (like Nginx
ingress-controller or Rancher
ingress-controller), it
can be either the service clusterIP, or the set services’ Pod IPs.
Rancher Load Balancer balances between Kubernetes Services podIPs.

Pluggable Load Balancer Provider Model

The controller has no knowledge about provider implementation. All
communications are done using Load Balancer Config, which is pretty
generic. I’ve chosen to use this model, as it makes it easy to re-use
the code base through:

• Re-use of the controller code by choosing a provider of your choice
• Re-use of the provider code by choosing a controller of your choice.

Provider and controller get defined as arguments passed to the
application entry point. Kubernetes is the default LB controller. The
Rancher Load Balancer is the default LB provider.

Future Release Implementation

We’ve just released the first version of the Rancher
ingress-controller. We are planning to continue actively contributing
into it. Features to cover in the next release include:

• Horizontal scaling of the Rancher LB by defining scale in the
  Kubernetes Ingress metadata
• Non-standard http public port support. Today, we support default
  port 80, but also want the user to be able to decide which public
  port to use.
• TLS support

Useful Links

Kubernetes ingress
doc – here you can read
more on the Kubernetes Ingress Resource. rancherKubernetes ingress
controllers contrib github
repo
– very helpful for developers who want to contribute to Kubernetes, and
want to know best practices and common solutions. Rancher ingress
controller github repo –
Rancher ingress controller github repo. Rancher Ingress Controller
Documentation

Alena
Prokharchyk If you have any questions or feedback, please contact me on
twitter: @lemonjet https://github.com/alena1108