Linux Conversations | Episode 3: Fixing the Unfixable with William Preston

Welcome back to “Linux Conversations,” our blog series where we chat with SUSE experts about the challenges and innovations shaping the world of Linux. This is Episode 3, and today we’re diving into the critical world of L3 support with William Preston, an L3 team lead at SUSE. L3 support is the last line of defense, tackling the most complex technical challenges our customers face. Join us as we explore the unique role this team plays and the intricate problems they solve.

SUSE’s enterprise Linux solutions power critical systems worldwide. But what happens when those systems encounter problems? That’s where the L3 support team comes in, acting as the last line of defense against the most complex technical challenges. Today, we talk with William Preston, to understand the critical role this team plays and the unique challenges they face.

Interview

Rick Spencer: William, can you tell us a bit about your role at SUSE and how long you’ve been here?

William Preston: I’m one of the L3 team leads at SUSE. I’ve been with the company for about 11 years now. My interest in Linux goes way back, to the early days of floppy disk installations and the challenges that came with them. It’s come a long way since then!

Rick Spencer: SUSE has certainly evolved. It’s now a critical part of so many enterprises. So, what exactly is L3 support?

William Preston: In a nutshell, we provide code fixes for our customers’ most challenging issues. It’s more than just fixing bugs, though. We’re a whole organization dedicated to helping customers. Our support team resolves the majority of issues, but the really tricky ones (those requiring changes) escalate to us. Our job is to get them a fix, and that often means coordinating with specialists across SUSE.

Rick Spencer: So you’re pulling in experts from the core kernel team, networking, memory management, etc.?

William Preston: Exactly. We might need a specialist in kernel memory management, networking, or any other component that is needed. Our role is to identify where the problem lies. With these massive servers we see today, pinpointing the root cause can be incredibly complex. Is it a networking bottleneck? A memory management issue? Our job is to find the needle in the haystack.

Rick Spencer: Can you give an example of a complex issue your team has tackled?

William Preston: We had a customer running mission-critical Kubernetes jobs on SUSE Linux Enterprise Server (SLES). They experienced intermittent slowdowns that, while rare, were unacceptable for their SLAs. The L3 team led the effort to debug the issue, which was eventually traced to a kernel problem, and provided a patch.

Rick Spencer: That leads to an important point. How does SUSE handle fixes, especially in the context of open source?

William Preston: We have an upstream-first policy. Ideally, fixes go back to the kernel community. This benefits everyone, including our enterprise customers, because it ensures the problem stays fixed in future versions of Linux. They don’t have the burden of maintaining a patch themselves.

Rick Spencer: That’s a huge advantage.

William Preston: It is. Sometimes it means we have to create multiple versions of a patch: one for upstream and another for older software versions that some customers are running. But it’s lower maintenance for us and keeps the customer from being tied to us.

Rick Spencer: But what about immediate needs? If a customer’s service is degraded, they can’t wait for an upstream fix, right?

William Preston: Absolutely. We have hot fixes, what we call PTFs, that can be deployed immediately. It’s a compromise. We want everything upstream, but we understand customers’ immediate needs. They get the best of both worlds: immediate relief and a long-term solution.

Rick Spencer: It sounds like L3 support requires a unique skillset.

William Preston: Experience is key. Our developers aren’t just writing thousands of lines of code. Sometimes a one-line fix, after two months of investigation, is perfect. They need to see the big picture, understand how systems work, and have a strong problem-solving mindset. Attention to detail and patience are crucial.

Rick Spencer: Any final thoughts you’d like to share?

William Preston: Just that it’s a rewarding challenge. We’re often dealing with the most complex issues, and it’s satisfying to find those critical fixes.

Conclusion

William’s insights into SUSE’s L3 support highlight the dedication and expertise required to keep critical systems running smoothly. It’s a testament to the power of open source and the commitment to providing long-term, sustainable solutions for enterprise customers. This concludes Episode 3 of “Linux Conversations.”

Explore other episodes of “Linux Conversations” for more insights from SUSE experts. Learn more about SUSE Linux Enterprise Server and support offerings. Subscribe to our blog to stay updated on the latest Linux news and trends.

More from this Series:

If you found this discussion insightful, check out previous episodes in our Linux Conversations series, where we explore key challenges and solutions in enterprise Linux:

• Linux Conversations | Episode 1: Running Multi-Linux Environments in Production with Donald Vosburg
• Linux Conversations | Episode 2: The Future of SUSE Multi-Linux with Johannes Hahn
• Linux Conversations | Episode 3: Fixing the Unfixable with William Preston

Stay tuned for more interviews in our Linux Conversations series—follow our blog or subscribe for the latest insights from SUSE Linux experts.

*Disclaimer: This transcript has been lightly edited for clarity and readability.*