kGraft is a feature of the Linux kernel that implements live patching of a running kernel, which allows kernel patches to be applied while the kernel is still running. By avoiding the need for rebooting the system with a new kernel that contains the desired patches, kGraft aims to maximize the system uptime and availability. At the same time, kGraft allows kernel-related security updates to be applied without deferring them to scheduled downtimes.[1][3] Internally, kGraft allows entire functions in a running kernel to be replaced with their patched versions, doing that safely by selectively using original versions of functions to ensure per-process consistency while the live patching is performed.[4]

kGraft
Developer(s)SUSE
Initial releaseMarch 27, 2014 (2014-03-27)[1]
Stable release
5.12[2] / 25 April 2021; 3 years ago (25 April 2021)
Repository
Operating systemLinux
TypeKernel extension
LicenseGNU GPL versions 2 and 3
Websitewww.suse.com/products/live-patching

kGraft is developed by SUSE, with its source code licensed under the terms of versions two and three of the GNU General Public License (GPL).[3][5] In April 2014, kGraft was submitted for inclusion into the Linux kernel mainline,[4] and the minimalistic foundations for live patching were merged into the Linux kernel mainline in kernel version 4.0, which was released on April 12, 2015.[6]

Internals

edit

Internally, kGraft consists of two parts – the core kernel module executes the live patching mechanism by altering kernel's inner workings, while userspace utilities prepare individual hot patch kernel modules from source diffs. Live kernel patching is performed at the function level, meaning that kGraft can replace entire functions in the running kernel with their patched versions, while relying on the mechanisms and infrastructure established by ftrace to "route around" old versions of functions. No changes to the kernel's internal data structures are possible; however, security patches, which are one of the natural candidates to be used with kGraft, rarely contain changes to the kernel's data structures.[3][4][7]

While applying hot patches, kGraft does not require a running kernel to be stopped for patched versions of functions to be introduced into it. Instead of replacing functions atomically, kGraft provides consistent "world views" (or "universes") to userspace processes, kernel threads and interrupt handlers, which are monitored during their execution so the original versions of patched kernel functions can continue to be used. To accomplish that, kGraft maintains original versions of patched functions in a read-copy-update (RCU) fashion, and dynamically selects between the original and patched versions depending on which process, kernel thread or interrupt handler executes them. More specifically, original versions of functions continue to be used‍—‌at the time when a hot patch is applied‍—‌for processes currently executing within the kernel space, for kernel threads until they reach their completion points, and for currently executing interrupt handlers.[3][4][8][9]

Due to its design, kGraft does not introduce additional latency while applying hot patches. As the downside, original versions of patched kernel functions may be required to be maintained for extended periods of time in case there are processes that remain for too long within the kernel space; for example, a process may wait for I/O on a network socket. Also, as both original and patched versions of functions are allowed to be executed in parallel, troubles may arise if they use kernel's internal data structures in different ways.[4][9]

"World views" or "universes", maintained by kGraft in a read-copy-update fashion[3]: 14, 15, 17 
Without additional logic in place, not replacing patched functions atomically could lead to inconsistencies.
Each process is monitored so it executes a patched function consistently within a single system call.
After everything migrates to a new "universe", trampoline-style checks are no longer needed.

History

edit

SUSE announced kGraft in January 2014[10] and released it publicly in March 2014 under the terms of the GNU General Public License version 2 (GPLv2) for the kernel part, and under the terms of version 3 (GPLv3) for the userspace part.[1][5] It was released shortly after Red Hat released its own live kernel patching implementation called kpatch.[11] kGraft aims to become merged into the Linux kernel mainline, and it was submitted for the inclusion in April 2014.[4][9]

kGraft was made available for SUSE Linux Enterprise Server 12 on November 18, 2014, as an additional feature called SUSE Linux Enterprise Live Patching.[12][13]

Minimalistic foundations for live kernel patching were merged into the Linux kernel mainline in kernel version 4.0, which was released on April 12, 2015. Those foundations, based primarily on the kernel's ftrace functionality, form a common core capable of supporting hot patching by both kGraft and kpatch, by providing an application programming interface (API) for kernel modules that contain hot patches and an application binary interface (ABI) for the userspace management utilities. However, the common core included into Linux kernel 4.0 supports only the x86 architecture and does not provide any mechanisms for ensuring function-level consistency while the hot patches are applied.[6][14][15]

Since April 2015, there is ongoing work on porting kGraft to the common live patching core provided by the Linux kernel mainline.[15] However, implementation of the required function-level consistency mechanisms has been delayed because the call stacks provided by the Linux kernel may be unreliable in situations that involve assembly code without proper stack frames; as a result, the porting work remains in progress as of September 2015. In an attempt to improve the reliability of kernel's call stacks, a specialized sanity-check stacktool userspace utility has also been developed.[16][17]

See also

edit
  • Dynamic software updating, a field of research focusing on upgrading programs while they are running
  • kexec, a method for loading a whole new Linux kernel from a running system
  • Ksplice and KernelCare, other Linux kernel live patching technologies developed by Ksplice, Inc. (later acquired by Oracle) and CloudLinux, respectively

References

edit
  1. ^ a b c "SUSE Releases kGraft for Live Patching of Linux Kernel". SUSE. March 27, 2014. Retrieved November 7, 2014.
  2. ^ "Release 5.12". April 25, 2021. Retrieved May 14, 2021.
  3. ^ a b c d e Vojtěch Pavlík (March 28, 2014). "kGraft: Live patching of the Linux kernel" (PDF). linuxfoundation.org. Retrieved November 7, 2014.
  4. ^ a b c d e f Jonathan Corbet (April 30, 2014). "The initial kGraft submission". LWN.net. Retrieved November 7, 2014.
  5. ^ a b Chris Duckett (February 4, 2014). "Suse's kGraft looks to put live kernel patching back into Linux". ZDNet. Retrieved February 11, 2020.
  6. ^ a b "Linux kernel 4.0, Section 1.2. Live patching". kernelnewbies.org. April 26, 2015. Retrieved April 27, 2015.
  7. ^ Vojtěch Pavlík (January 31, 2014). "kGraft: Live Kernel Patching". suse.com. Retrieved November 7, 2014.
  8. ^ Libby Clark (March 4, 2014). "SUSE Labs Director Talks Live Kernel Patching with kGraft". linux.com. Archived from the original on March 10, 2016. Retrieved November 7, 2014.
  9. ^ a b c Michael Larabel (May 1, 2014). "SUSE Posts kGraft, Red Hat Posts Kpatch Patches". Phoronix. Retrieved November 7, 2014.
  10. ^ "SUSE Develops kGraft for Live Patching of Linux Kernel". SUSE. January 31, 2014. Retrieved December 28, 2014.
  11. ^ Josh Poimboeuf; Seth Jennings (February 26, 2014). "Introducing kpatch: Dynamic Kernel Patching". redhat.com. Retrieved February 11, 2020.
  12. ^ "SUSE Linux Enterprise Live Patching Now Available". SUSE. November 18, 2014. Retrieved November 23, 2014.
  13. ^ Sean Michael Kerner (November 18, 2014). "SUSE Brings Live Patching and Ceph Storage to Its Enterprise Linux". serverwatch.com. Retrieved November 23, 2014.
  14. ^ Jonathan Corbet (February 25, 2015). "A rough patch for live patching". LWN.net. Retrieved April 27, 2015.
  15. ^ a b "Linux kernel source tree: kernel/git/torvalds/linux.git: Pull live patching infrastructure from Jiri Kosina". kernel.org. February 11, 2015. Retrieved April 27, 2015.
  16. ^ Jonathan Corbet (September 30, 2015). "Compile-time stack validation". LWN.net. Retrieved October 2, 2015.
  17. ^ Josh Poimboeuf (September 24, 2015). "Linux kernel documentation: Documentation/stack-validation.txt (from the v13 patch)". LWN.net. Retrieved October 2, 2015.
edit