If you’re running Linux servers, one of your top priorities should be Linux patch management. Linux is powerful, stable, built on open source, and almost infinitely customizable. As if those weren’t enough benefits, you can also fine-tune your Linux experience by choosing one of the huge variety of Linux flavors out there (officially known as “distributions” or “distros” for short).
Essentially, where Windows promises a one-size-fits-all, out-of-the-box experience, Linux gives you both broad and granular control over your own environment.
While every Linux distribution has certain commonalities, you only have to work with a couple of them to realize how big the differences are. Each flavor has its own strengths and weaknesses, and this is nowhere more true than when it comes to patching and updates. Where Microsoft maintains fairly rigid control over patching, with Linux, the path is nowhere near as straight and narrow.
That’s an important distinction—because while patching is good when it comes to bug fixes and driver or software issues, it’s absolutely mission-critical when it comes to remediating security vulnerabilities.
Gone are the days when security was less of a problem for Linux users—back when hackers focused on what they saw as more commercial OSes. Today, there are more than one and a half times more web servers running on Linux (42.7% for Linux, compared to 24.9% for Windows, according to stats gurus W3Techs ). With so many businesses running mission-critical data and operations on this operating system, unfortunately, hackers have shifted their focus to Linux, too.
That makes it more important than ever to keep up with patching, which could be a challenge.
According to a recent ZDNet article, most Linux distributions are very secure, with the main security problem, according to the article, being “simple system administrator incompetence.” But is that really true?
The truth is that sysadmins aren’t “incompetent,” and they certainly understand the importance of patching to their networks’ security. It’s just that patching in Linux is complicated. Fortunately, organizations today are not alone, and there’s lots of information out there along with tools to make the process simpler.
In this post, we’ll take a birds-eye view of what makes patching such a challenge in a Linux environment, then look at some of the most popular Linux distributions on the market today and explore how each of them handles patching.
The Problems with Linux Patching
Why is Linux patching so much more complicated than, say, patching Windows servers?
In Linux, as in other OSes, patching involves at least three key phases, from an operations standpoint:
- Scanning endpoints for missing patches
- Downloading patches from vendor sites
- Deploying patches to endpoints
However, unlike with Windows, where patches are generally released in an orderly way through the Microsoft Security Response Center in a monthly process known as Patch Tuesday, with Linux, there are numerous vendor sites to consult, especially if you’re running more than a single distribution, and the timing is nowhere near as predictable.
Downloading and deploying patches will involve a variety of different repositories as well as different commands on each distribution.
True, for some distributions, advisories are available. These are similar to Microsoft updates, which bundle updates and provide a report describing the issues addressed by the advisory. However, for other distributions, only package-level updates are available, which are less predictable in terms of their impact on the endpoint.
And let’s face it. When choosing a Linux distribution, few organizations place patching simplicity as the number one priority. The task itself usually takes a backseat to other considerations, such as cost, stability, desktop environment, and infrastructure compatibility. Patching tends to be one of those “we’ll cross that bridge when we come to it” issues.
Typically, when it comes to patching, the Linux community can be very “DIY” and hands-on, with administrators happily diving in and creating scripts to automate and simplify the process. However, this is changing quickly, especially as enterprises come to realize the complexities of patching at scale in complex network environments that include BYOD, on-premises, cloud, IoT, and a range of other endpoints.
Some automated configuration management systems promise to automate patching to save you work, including on Linux systems. However, in practice, this still often involves creating custom scripts, and even basic automation may be lacking from these solutions out of the box.
This recent Security Boulevard article, like the ZDNet article mentioned above, blames system administrators for poor patching practices. The author writes that while patching is crucial for security, “unfortunately, many Linux users neglect to put these patches into action“. As if IT security departments don’t have a million other demanding tasks on their plate.
Instead of blaming hardworking system administrators, let’s acknowledge a hard truth: Sometimes, patching—especially patching across a range of Linux distributions—is just too hard to keep up.
Let’s run down the various popular flavors of Linux today and take a look at how patching is handled for each distribution.
Linux Distributions
In this section, we’ll explore five of today’s most popular Linux distributions, their pros and cons, and focus, in particular, on how well they handle patching.
CentOS/Red Hat Enterprise Linux (RHEL)
These two distributions have the same core functionality; the primary distinction between them is that CentOS is a free, community-based distribution, while RHEL comes with enterprise-level perks including support, with a matching price tag. Both are based on Fedora, a free, open-source classic Linux distribution.
THE PROMISE (RHEL): “The world’s leading enterprise Linux platform”
THE PROMISE (CentOS): “Community-driven free software effort focused on delivering a robust open-source ecosystem around a Linux platform.”
PROS: This is probably the most common Linux distribution with a massive user base. An older kernel with a long release cycle, it’s a popular choice for die-hard Linux devotees—highly customizable, secure, and stable.
CONS: The biggest con of CentOS is that it will reach the end of its lifespan at the end of 2021 and is therefore considered a dead end. Organizations looking for a community-supported distribution will have to look elsewhere, such as to Oracle Linux, Amazon Linux, or CentOS Stream, a confusing new branch that has yet to win a massive following among disgruntled former CentOS users. The good news is that with the demise of CentOS, RHEL has increased its free offerings to up to 16 systems, apparently with no strings attached.
PATCHING (RHEL): Updates are available on a subscription-only basis with pricing determined by the number of servers the organization is running. Advisories provide some additional information to help prioritize patching, such as the ranked severity of the vulnerability. Patches are done using yum (short for “Yellow dog Updater, Modified”) or a similar command-line tool.
PATCHING (CentOS): There are no advisory-level patches that can be deployed directly to the machine. However, CentOS does translate advisory announcements from RHEL to CentOS and distributes this content via email lists, giving system administrators one more source to track and yet another manual process, since most patching tools are fairly crude and can’t make use of this information. While other tools are available, updates are generally handled through yum, a command-line utility with no graphical interface that retrieves updates from CentOS and third-party repositories. At the end of the OS version’s lifetime, the repository shifts to an archive that must be configured manually.
Ubuntu
Ubuntu is working hard to change its lightweight rep, repositioning itself as a fully cloud-ready enterprise server product in order to attract migrating CentOS users. It’s earned its reputation as the friendliest Linux flavor with good reason: It emphasizes a fast, intuitive GUI for many functions, with the simplest and most intuitive software installation in the Linux world. For these reasons, it has traditionally had a popular following among home users, especially on older machines that can’t cope with Windows. It is based on Debian, an entirely free, open-source classic Linux distribution.
THE PROMISE: “Better security. More packages. Newer tools. All your open source, from cloud to edge.”
PROS: Ubuntu is generally very stable and user-friendly, especially for Linux novices coming from more GUI-based OSes who are not comfortable working with the command line. You have lots of “plug and play” compatibility, several major productivity and other applications are available, and the distribution is highly customizable.
CONS: Application choice is very limited with this distribution, and as a relative newcomer to the serious web server market, it remains to be seen how it compares relative to more established players.
PATCHING: Probably the biggest drawback when it comes to patching in Ubuntu is that advisories only address security issues. That means that you’re on your own when it comes to other types of updates, such as bug fixes. This distribution has earned a bad name for itself for causing things to break when it comes to OS updates; for this reason, some organizations prefer to stick with long-term support (LTS) updates, which are stable releases every two years.
OpenSUSE and SLES (SUSE Linux Enterprise Server)
OpenSUSE, a desktop OS, and SLES, its hardened enterprise product, are both distantly related to RHEL and represent one of the oldest and most stable Linux distributions. This distribution is known for its extreme flexibility and the freedom of the end-user to determine their own configuration, sometimes resulting in compromised user-friendliness.
THE PROMISE (OpenSUSE): “The makers’ choice for sysadmins, developers, and desktop users.”
THE PROMISE (SLES): “A modular operating system that paves the way for IT transformation in the software-defined era.”
PROS: You get a very simple install and setup thanks to YaST, its configuration tool. SUSE used to have a strong reputation for user-friendliness and customizability, although Ubuntu has overtaken it in the last few years. It’s considered more polished, professional, and fully featured than Ubuntu.
CONS: Hardcore users claim that this distribution has been damaged by its association and continued ties with Novell and Microsoft. In practice, there are also issues with the installer and software updating; some users report that they are simply unable to get SUSE to work for them at all.
PATCHING: SLES uses multiple extensions that are required for multiple environments and applications. Each extension requires its own repository, and when remediating an advisory, there is a need to make sure it is done for every extension deployed. Hence, SLES patching process is fairly complex and requires time and expertise. Also, patch rollback is extremely difficult and not always possible.
Oracle
It’s little surprise that Oracle, too, is swooping in to try to fill the gap left by CentOS leaving the market. This free distribution has primarily been popular among small-to-mid-sized organizations, especially those currently using Oracle database products. It is based on Red Hat, and any adaptations have primarily been to ensure compatibility with other Oracle software and hardware products.
THE PROMISE: “Virtualization, management, and cloud-native computing tools, along with the operating system, in a single support offering.”
PROS: The biggest plus of Oracle Linux is its 100% compatibility with and similarity to RHEL, with additional compatibility advantages for customers using other Oracle products.
CONS: Oracle’s poor UI is probably its biggest drawback, plus this distribution is known for compatibility problems with non-Oracle hardware, firmware, and, in particular, virtualization software. It also offers less by way of community support than other distributions.
PATCHING: Oracle Linux actually has a reasonable reputation for being relatively simple to patch. Patches are available at the advisory level, with no subscription fee, and are billed as being easy to roll out with its Ksplice tool. However, due to some of its larger drawbacks, you will almost certainly need to rely on at least one other Linux distribution in your organization, making the big picture far more complicated.
Amazon Linux 2
Amazon Linux 2, like a number of other distributions, is based on RHEL. The replacement to Amazon Linux AMI back in December 2020, is essentially a highly minimized version of RHEL optimized for use as a Linux image in the cloud. It is also available as a downloadable virtual machine so it can be run locally.
THE PROMISE: “Secure, stable, and high-performance execution environment to develop and run cloud and enterprise applications.”
PROS: This is a popular free option for current Amazon AWS cloud customers, as it is highly compatible with other AWS services such as System Manager.
CONS: This distribution is still a fairly obscure choice, though gaining in popularity due to its strong ties to other AWS products. There may be issues with single-vendor lock-in, but the strong engineering team at AWS may counteract this somewhat. However, migration to other platforms may prove problematic—as it is when trying to break free from any single-vendor solution.
PATCHING: As with Ubuntu, advisories are only released for security patches, so you’re on your own for other updates. Because of this, while Amazon brags that live-patching functionality has been rolled out to make patching simple, “fixes that change assembly code or modify function signatures may not receive kernel live patches.”
JetPatch: Working for You Behind the Scenes
Many Linux distributions have their own tools to help with patch management. However, what you probably won’t find out of the box is a single tool that works well across distributions. And even fewer of these tools let you automate and streamline patch management to truly eliminate manual patching.
And as we all know, any time you’re introducing multiple tools, it can quickly start making your tasks more complex instead of simpler.
If you’re looking for a way to bring all your Linux patching together in one place, you’ll want to check out JetPatch. It’s a modern patching tool that simplifies patching, no matter what environment you’re operating in.
JetPatch has been designed to make security teams’ jobs easier, rolling out seamlessly across a massive range of platforms. JetPatch works with Windows, Unix (Solaris, AIX), and all these flavors of Linux:
- RHEL
- CentOS
- Oracle
- Amazon
- SUSE
- Ubuntu
Plus, with JetPatch Remote Workforce patching solution, you can support an even wider variety of endpoints.
When it comes to Linux, JetPatch manages updates at the repository level, meaning it will identify all applicable updates and automate deployment across all your Linux endpoints, no matter which distributions you’re using across your organization.
From end to end, from servers to portable (BYOD) devices, JetPatch unifies and automates your entire patch management strategy, giving you a single up-to-date dashboard view with insights into your entire network. With JetPatch taking care of your Linux patching best practices, it will automatically keep track of…
- The version number of your Linux distributions
- Location of all relevant repositories
- Which machines still need patching
This means there’s no more need for custom scripting or manual deployment. JetPatch also handles dozens of other details for you behind the scenes so you can finally quit chasing Linux updates.
Take the hassle and guesswork out of Linux patching—get JetPatch on your team. Get in touch to find out the easiest way to get started today.
