The Science and Technology Facilities Council (STFC) provides scalable computational resources to researchers and academic institutions throughout the UK. While STFC’s resources support a broad range of research, a growing number of UKRI-funded researchers focus on pushing boundaries in climate science. They are helped by JASMIN – a high-performance computer and storage facility operated jointly by STFC through Scientific Computing (SC) and the Centre for Environmental Data Analysis (CEDA) on behalf of the UK Natural Environment Research Council (NERC).
JASMIN hosts the CEDA Archive, a repository of about 30PB of environmental data that is of immense value to research communities in areas including atmospheric science, Earth observation and climate change. By giving researchers browser-based access to powerful analytical and visualization tools that draw data directly from the CEDA Archive, JASMIN is helping scientists address some of the biggest challenges facing our planet.
At-a-Glance
With demand growing for data-driven research tools, STFC adopted Kubernetes containers managed using Rancher Prime to make it faster, easier and more efficient to deploy and manage container-based services. With Rancher Prime, STFC has a naturally scalable and flexible platform for delivering services to a growing community of environmental scientists.
The journey to containers
As demand for data analytics grew in the UK environmental science community, STFC wanted to ensure that JASMIN could continue to deliver reliable, highperformance services at a greater scale. Whenever the CEDA development team wanted to build a new service running in the JASMIN environment for end users, it would ask the SC infrastructure team to set up a new virtual machine (VM), then configure it as required. If demand for the new service increased, the solutions team would go back to the infrastructure team to request more resources.
In addition to requiring a time-consuming back-and-forth exchange between the two teams, this arrangement resulted in a growing number of overprovisioned VMs, representing inefficient use of the physical infrastructure.
To enable a more scalable, flexible and automated approach to developing new collaborative research tools, STFC decided to move to a containerized environment.
Why Rancher Prime?
Having decided to use Kubernetes as its container orchestration platform, STFC wanted to avoid complexity in setting up and managing the technology. Rancher Prime from SUSE gave STFC the openness and modularity of Kubernetes without the need to dedicate time and effort to choosing components. SUSE pre-selects components that work together, reducing the work involved in getting up and running.
Having decided to use Kubernetes as its container orchestration platform, STFC wanted to avoid complexity in setting up and managing the technology. Rancher Prime from SUSE gave STFC the openness and modularity of Kubernetes without the need to dedicate time and effort to choosing components. SUSE pre-selects components that work together, reducing the work involved in getting up and running.
While Rancher Prime significantly simplifies the setup of new Kubernetes environments, it does not reduce their flexibility. The CEDA development team at STFC has used the Rancher Prime marketplace to add tools in the form of pre-built Helm charts, such as Gatekeeper for Kubernetes pod security policies.
The impact of Rancher Prime
Scalable operations delivered with ease
Rancher Prime provides a stable, scalable platform for building and running scientific research tools at STFC. Alongside Jupyter Notebooks, the organization also hosts several of its user-facing services on Rancher Prime, including its user management portal and cloud portal, and a number of Kubernetes-based data servers. The infrastructure team values the stability and ease of management offered by an off-the-shelf product, which enables it to focus on improving services.
The reliability of Rancher Prime enables STFC to make changes to the hardware underneath the Kubernetes nodes without needing to inform the software owners. Similarly, software owners no longer need to inform the IT team when they are doing deployments.
Greater efficiency in infrastructure and operations
STFC is continuing to move services that previously ran on VMs to containers on Rancher Prime. This enables physical compute resources to be shared in a more granular way, freeing up capacity that was previously locked away in overprovisioned VMs. Server capacity at STFC is now pooled and easier to share efficiently between containerized services.
The maintenance burden is also significantly lower with containerization versus a VM-based approach. Given the inherent scalability of Rancher Prime, the solution team no longer needs to ask the infrastructure team to provision more memory or more CPU.
Continuous integration with Rancher Prime
The solutions team at STFC has integrated its CI (continuous integration) pipeline in GitLab with Rancher Prime for greater efficiency and repeatability in deploying new software. The team created a script that automates a number of actions: logging into Rancher Prime, polling information, deploying Helm charts and so on.
The integration between GitLab and Rancher Prime enables STFC to save time and effort by setting up automations in its CI pipeline. The organization can also deal with security issues more quickly, because it can rebuild and push out images much faster with Rancher Prime.
What’s next for STFC?
With climate change at the top of the global agenda, JASMIN and the CEDA Archive are increasingly important resources for environmental science projects. As the STFC team builds and manages new data-driven analytics and visualization services for researchers, Rancher Prime will continue to help by simplifying deployment and management, improving infrastructure efficiency and reducing workload for IT specialists.