Introducing Toyota Material Handling (TMH)
TMH, part of Toyota Industries Corp., crafts innovative material handling equipment, ranging from hand pallet jacks to high-capacity forklifts. Established in 1956 and recently restructured in 2020, TMH operates from a cutting-edge facility in Columbus, Indiana. Offering both standard and tailored solutions, TMH serves diverse industries with applications that span from compact indoor spaces to expansive warehouses and ports. Championing Toyota Lean Management, TMH is spearheading lean manufacturing into the future with Industry 4.0 – an intelligent blend of digital technology and automation that amplifies productivity through connected, data-driven systems. Key to this transformation are Rancher Prime and Longhorn.
At-a-Glance
Embarking on a journey to Industry 4.0, Toyota Material Handling (TMH) has revolutionized its IT infrastructure. After initially struggling with the management of its homegrown Kubernetes environment, TMH sought out more efficient solutions, ultimately selecting Rancher Prime. This choice marked a significant turning point, elevating TMH’s operational efficiency and pushing it further into the Industry 4.0 landscape. Alongside this, TMH implemented Longhorn for reliable Kubernetes-native persistent storage. With the guidance of SUSE Support, TMH has leveraged Rancher Prime to reap the benefits of a production-grade, multi-cluster, multi-cloud Kubernetes management experience.
Accelerating toward Industry 4.0
Steeped in a culture of continuous improvement, or Kaizen, TMH employees relentlessly seek to cut waste, a philosophy known as Muda. From this premise, the internal application development team turned to the infrastructure team in 2018, requesting application modernization conducive to streamlined DevOps best practices and microservices.
Up to that point, all systems serving the company’s internal departments, from HR to application developers to the production floor, shared resources on two IIS servers. Much to the team’s frustration, updating one system in this interconnected environment inadvertently affected others. Equally impacted by this inefficient setup, the IT team hoped that modernization would enhance resource availability and enable developers to boost application performance without causing cascading effects on connected systems.
In answer to the call, a tenacious and gifted engineer, Michael Nichols – then a senior infrastructure engineer – manually built TMH’s first Kubernetes cluster. Relying on community versions of Ubuntu Linux and community forums for guidance, Nichols created micro-environments for development. Partnering with the internal AppDev team lead, Donald Perry, who streamlined a continuous integration and delivery (CI/CD) pipeline, the collaborators optimized resources for DevOps workflows, using Azure DevOps. After successfully completing this initial build, Nichols repeated the arduous process and built three more clusters for testing, staging and production over a period of two years.
With the groundwork laid for a dynamic IT environment, the push toward Industry 4.0 gathered momentum in early 2020 with the arrival of Richard Keefer, manager of infrastructure services. Keefer noted a familiar state at TMH common among mature manufacturing facilities: antiquated IT systems, manual processes and aging hardware. Committed to Muda and Kaizen, he led the modernization of the network architecture: segregating IT and operational technology networks, migrating VMware environments from Dell’s VCE VBlock solution to Dell’s VxRail appliance, integrating Citrix and implementing Fortinet security solutions.
“We’re modernizing and preparing our organization for Industry 4.0,” asserts Keefer.
“Toyota is about lean management, and having the ability to spend cycles on other priorities instead of on a manually built system is indeed a big win. It used to take us hours or even days to work on an outage, but we haven’t had to do that since implementing Rancher Prime."
Journey to SUSE solutions
Rancher Prime: optimizing Kubernetes management
The groundwork that Nichols laid in establishing TMH’s initial Kubernetes cluster eased Keefer’s subsequent modernization endeavors. However, challenges still lingered in the Kubernetes ecosystem. Despite enhancing automation, the IT team faced an uphill battle managing the clusters, dealing with lost nodes, system downtimes, rebuilds and expired certificates.
Exacerbated by the pitfalls of manually managing an increasingly complex Kubernetes environment, the team turned to the community edition of VMware Tanzu in early 2022. Once in place, Tanzu’s management cluster simplified the process of deploying additional clusters for the team. However, over time, Tanzu began to systematically fail. Without adequate support and with the complexities of Tanzu, the team decided to enlist enterprise support and research the best solution for its needs. The team considered the enterprise version of VMware Tanzu, but it ultimately selected Rancher Prime. According to research reports, Rancher Prime was easy and quick to implement. Its top-tier spot in Gartner’s quadrant affirmed its dominance in Kubernetes management. Moreover, Rancher Prime’s features and enterprise support perfectly aligned with Keefer’s vision for embracing Industry 4.0 at TMH.
“After reviewing all the research, Rancher Prime was hands down a better return on our investment,” says Keefer. “I asked, where can we be in five years from now compared to today? And after putting all of the pieces together, I knew Rancher Prime would be the right choice.”
Longhorn: elevating storage management
In the early days of implementing its Kubernetes environment, the IT team didn’t regard storage as a priority. However, that view changed as the Advanced Research and Development (ARD) department began using a diverse set of workloads. As it was engaged in innovation and prototyping, ARD required a trustworthy system for storing and retrieving data across multiple clusters and processes – a capability that native Kubernetes and host storage couldn’t provide.
Having recognized the necessity for a sturdy persistent storage solution and foreseeing the growing importance of such storage in Industry 4.0, the IT team turned to Longhorn. With its centralized dashboard and proficient storage management, Longhorn emerged as a reliable solution for TMH’s data management strategy.
The impact
Rancher Prime
TMH’s transition to Rancher Prime was seamless, with the platform being set up in less than a day on their Dell VCE VBlock solution – quite the contrast to the weekslong endeavor of implementing TMH’s first Kubernetes cluster by hand.
“Rancher Prime acted as a turnkey solution and got us back up and running to where we needed to be,” says Nichols.
Rancher Prime catalyzed a seismic shift in TMH’s operational efficiency. Not only does it dovetail with TMH’s modernization goals, but it also delivers practical benefits that propel TMH into an authentic Industry 4.0 ecosystem and further hone Toyota’s lean management practices.
A year post-implementation, Keefer affirms, “Thanks to Rancher Prime, we are in a better place today than ever before.”
Empowers teams
Rancher’s Role-based Access Control (RBAC) empowers TMH’s application developers and other teams by providing self-service capabilities for managing and maintaining their respective clusters. By providing a user-friendly portal, Rancher Prime enables teams to work autonomously. The establishment of RBAC ensures streamlined work procedures, reduces complexities and dependencies, and frees the IT team to innovate.
Accelerates development
For TMH’s ARD team, Rancher Prime provides a centralized hub for tools and resources – a pivotal change that significantly reduces the time and effort required for prototyping. “Previously, our ARD team had to rely on a couple of distributed systems, which wasn’t the best setup,” says Nichols. “Rancher Prime unifies everything, thereby helping them move through their projects more efficiently.”
Simplifies hybrid-infrastructure management
Rancher Prime affords the IT team effortless management of resources, whether on-premises or in the cloud. This cohesive approach drastically simplifies the monitoring and management tasks across varied environments.
“We’ve spun up a couple of AKS clusters in Azure, and we would have to remember to go there to manage them because they were individual islands,” says Nichols. “From an infrastructure perspective, being able to use Rancher Prime as a single pane of glass to manage our mixed environment is critical. It helps us understand where our deployments are and enables us to address issues faster.”
Streamlines troubleshooting
Rancher Prime streamlines the process of identifying and resolving issues, thereby conserving the IT team’s precious time and energy.
Keefer says, “Using the Rancher Prime interface, we can easily identify any potential issues. We can also effortlessly tear down and bring up instances and nodes.”
While Keefer was conducting his initial analysis of Rancher Prime, the cost of engineers’ time played an important factor in his decision. “What I consider as my return on investment is not just the product itself,” he says. “Michael spent four hours working on an issue with VMware Tanzu. In Rancher Prime, resolving the same issue took 30 minutes. An 87% reduction in troubleshooting time is a good return on investment.”
Increased reliability
TMH, a paragon of lean management, can’t afford downtime. Even a slight hiccup could cost millions. Before Rancher Prime, the older Kubernetes system went down during a critical network upgrade. However, because the network upgrade was a priority, it took the team two weeks to fix the prolonged outage.
With Rancher Prime, which is hardened for enterprise use cases, the team no longer has to navigate the vulnerabilities of a manually built system, thereby reducing downtime and increasing reliability.
“Toyota is about lean management, and having the ability to spend cycles on other priorities instead of on a manually built system is indeed a big win,” says Nichols. “It used to take us hours or even days to work on an outage, but we haven’t had to do that since implementing Rancher Prime.”
Longhorn
Initially, the IT team found the concept of persistent block storage within Kubernetes elusive. The team struggled to grasp the intricacies of persistent volume claims and the lifecycle of data management.
Longhorn proved to be a game changer. With its intuitive dashboard, Longhorn puts the IT team in the driver’s seat for managing persistent volume claims and offers insights into resource utilization. But that’s not all. Longhorn syncs data across nodes, ramps up redundancy and shields data from loss — these are crucial features for the ARD department. What was once an enigma, Longhorn has morphed into a robust, effortlessly manageable part of the IT team’s Kubernetes setup.
“Not only did Longhorn help us overcome technical challenges in managing our data, but it also facilitated our Industry 4.0 initiatives for sustainable growth and innovation by optimizing our data management strategies,” says Nichols.
SUSE Support
Guiding TMH through the initial setup of Rancher Prime and Longhorn, SUSE Support has become a trusted adviser on TMH’s modernization journey, acting as a safety net and illuminating areas that conventional research doesn’t cover. This proves particularly helpful when team members are uncertain about what to search for.
“Sometimes it’s just easier to reach out to people, especially if someone else has experienced the same issue. They can quickly guide you to the solution,” says Nichols. “It saves considerable time compared to searching through droves of community posts.”
In one instance, Nichols encountered a roadblock when setting up Rancher Prime’s network profile configurations with VMware. Nichols then received a timely solution from SUSE Support, enabling him to move forward.
One aspect of SUSE Support that the team particularly appreciates is having a dedicated Slack channel. “Our Slack channel is a godsend for answering questions. It allows us to connect with experts without having to make a phone call,” says Keefer.
Having a team of experts on standby empowers TMH to proceed with its modernization efforts without the fear of facing any issues alone.
“Through this partnership, we hold biweekly meetings, which is outstanding,” says Keefer. “I don’t see many organizations willing to do that. SUSE is just as invested in us as we are in them.”
What’s next for Toyota Material Handling?
Having integrated Rancher Prime and Longhorn onto Dell VxRail, TMH has effectively applied the principles of Kaizen and reduction of Muda – paving the way for further process innovation and refinement.
As TMH embraces the Industry 4.0 era, safeguarding containerized applications and data is crucial. Consequently, TMH plans to incorporate the zero trust security solution, NeuVector Prime, to fortify its environment and protect the integrity and confidentiality of its data and applications.
TMH is also aiming to broaden its Rancher Prime presence in Azure. This expansion will enhance the interconnectedness of systems, optimize cloud computing and facilitate advanced data analytics, all in accordance with Industry 4.0 standards. The result will be a more agile and responsive operational structure, well-equipped to handle shifting market demands.
Empowered by open source technologies, TMH is swiftly transitioning toward a fully integrated, ultra-efficient manufacturing ecosystem. This reflects Toyota’s enduring commitment to innovation, quality and Kaizen – the fundamental ethos of the Toyota brand. As TMH plans to implement public Wi-Fi and private cellular networks for IoT (Internet of Things) connectivity, its strides toward Industry 4.0 manufacturing standards will undoubtedly set it apart from the competition.