This week, I’ve been working on an application that sets up IPsec connections. PF_KEYv2 is the standard interface for configuring IPsec. PF_KEYv2 is widely-available; even systems with more advanced IPsec stacks (like XFRM on Linux) provide a PF_KEYv2-compatible wrapper. I want my application to work on OS X and Linux, so I’m targeting PF_KEYv2 instead of OS-specific APIs.

Unfortunately, PF_KEYv2 is difficult to use. It’s a socket-based protocol between the application and the kernel (like the BSD routing interface), so the application developer is responsible for details like socket management, serialization, and padding. Other IPsec management interfaces provide detailed errors, but when PF_KEYv2 encounters a problem it reports only a single, generic errno value, like EINVAL (“invalid argument”). As a result, it’s difficult to get the details right.

The kernel was consistently rejecting my requests as invalid, but I couldn’t figure out why. I read the kernel source but couldn’t figure out which requirement I’d failed. I wanted to step through the kernel-side execution in a debugger. I thought this would be straightforward, but it was much harder than I’d anticipated: it took me two full days of stumbling through others’ blog posts to figure it out. I’m writing up this experience in the hope that it helps others avoid this pain.


To debug the kernel, you’ll need two machines:

  • The debugger machine hosts the debugging tool. We’ll use GDB as our debugging tool because the kernel has GDB-specific scripts that will make us more productive. In my case, the debugger machine runs Windows 10(!) I use the Windows Subsystem for Linux to run Ubuntu 16.04 in the Windows environment. WSL is phenomenal: it emulates enough of the Linux ABI to run almost all of my Linux applications—including kernel debugging—without a problem.
  • The debuggee machine runs the kernel that is being debugged. I used a virtual machine running inside VMware Workstation 14 Player on the debugger machine. Player is free, and it has a nifty, GDB-compatible debugger interface that lets you avoid futzing with serial ports. I run Ubuntu 17.10 x64 with kernel 4.13.0-19-generic on this VM. Your kernel version will differ; substitute in your own version (from uname -r) in these instructions.

This tutorial assumes that both machines are running Ubuntu and that the debuggee is a VMware VM.

To start, we’ll need to prepare a debugging environment. We’ll build the environment on the debuggee machine and then transfer it to the debugger machine. This way, we don’t have to match the two machines’ configurations perfectly. (For example, I’m using an older version of Ubuntu on the debugger machine.)

Get the symbols

First, get the debug symbols for the debuggee’s current kernel. Debug symbols are annotations for compiled artifacts that contain information like variable names, function names, and source code locations (paths and line numbers). They’re usually omitted from release binaries to save space, but they’re critical to productive debugging: they enable you to match the machine’s behavior to the relevant source code.

The Ubuntu debug symbols are published on a separate archive with a different signing key. Here’s how to install the packages (as described by the Ubuntu wiki and VisualKernel):

# Trust the debug symbol signing key
debuggee$ sudo apt-key adv --keyserver --recv-keys C8CAB6595FDFF622

# Add the debug symbol repository
debuggee$ codename=$(lsb_release -c | awk  '{print $2}')
debuggee$ sudo tee /etc/apt/sources.list.d/ddebs.list << EOF
deb ${codename} main restricted universe multiverse
deb ${codename}-security main restricted universe multiverse
deb ${codename}-updates main restricted universe multiverse
deb ${codename}-proposed main restricted universe multiverse

# Retrieve the list of available debug symbol packages
debuggee$ sudo apt update

# Install the debug symbols for the current kernel version
debuggee$ sudo apt install --yes linux-image-$(uname -r)-dbgsym

Get the kernel source code

Next, we’ll need the kernel source code. When both source code and debug symbols are loaded, the debugger can show the exact line of source code being executed.

Enable the source repository in the Software Center’s sources configuration. (It’s not straightforward to script, unfortunately.) Then download the source to ~/kernel/source:

debuggee$ sudo apt update
debuggee$ mkdir -p ~/kernel/source
debuggee$ cd ~/kernel/source
debuggee$ apt source $(dpkg-query '--showformat=${source:Package}=${source:Version}' --show linux-image-$(uname -r))
debuggee$ cd linux-*

We now have the kernel source.

Modern compilers optimize code. Loop unrolling is one such optimization: if a compiler can prove that a loop will execute exactly n times, it can replace the loop with the n copies of the loop’s body. This transformation preserves the visible effect of the loop while improving performance. However, these optimizations cause the source code to diverge from the executable code, which makes debugging more difficult. Therefore, we prefer to debug using artifacts built with compiler optimizations disabled (using the -O0 option to gcc, for example.) Unfortunately, that’s not possible with the Linux kernel. So we won’t attempt to recompile the kernel without optimizations.

Build the GDB scripts

The kernel includes a set of GDB scripts that make debugging the kernel much easier. For example, kernel functionality (including PF_KEYv2) is often contained in loadable modules. These modules are loaded only when their functionality is needed. But, because they’re loaded and unloaded at runtime, they’re more difficult to configure for debugging. Fortunately, the GDB scripts included with the kernel automatically configure module debugging for you. (You can do it yourself, but it’s a pain.)

These scripts have a small dependency on the kernel build process, though. We need to build just enough of the kernel to make these scripts functional. We don’t need to build and install our own kernel, because the generic Ubuntu kernel includes the right configuration settings already, so we can abort the build as soon as the scripts are prepared.

Start the kernel build:

debuggee$ sudo apt build-dep --yes linux-image-$(uname -r)
debuggee$ sudo apt install --yes fakeroot libncurses5-dev
debuggee$ fakeroot debian/rules clean
debuggee$ fakeroot debian/rules binary-headers binary-generic binary-perarch

You can terminate the build once debian/build/build-generic/scripts/gdb/linux/ is created. You don’t need to wait for the build to finish, as we’re not using a custom kernel; we’re just debugging the Ubuntu-supplied kernel binary. (If you do want to install your own kernel, continue following the Ubuntu wiki instructions.)

Disable KASLR on the debuggee machine

We also need to disable kernel address space layout randomization (KASLR). KASLR randomly perturbs the location of kernel code to make attacks more difficult. It’s a great exploit mitigation technology, but when it’s enabled the addresses in our symbol files won’t match those in the kernel, so our debugging tools will be useless. We’ll need to turn it off.

We can disable KASLR by changing the GRUB bootloader settings. As root, open /etc/default/grub (sudo nano /etc/default/grub) on the debuggee machine and look for GRUB_CMDLINE_LINUX_DEFAULT. You’ll see a line that looks like:


Add nokaslr to the end:

GRUB_CMDLINE_LINUX_DEFAULT="splash quiet nokaslr"

Then, apply the change:

debuggee$ sudo update-grub

Confirm that /boot/grub/grub.cfg was updated with the new option:

debuggee$ grep nokaslr /boot/grub/grub.cfg
        linux   /boot/vmlinuz-4.13.0-19-generic root=UUID=... ro  splash quiet nokaslr $vt_handoff
                linux   /boot/vmlinuz-4.13.0-19-generic root=UUID=... ro  splash quiet nokaslr $vt_handoff

The change will take effect on the next boot. (We’re about to shut down anyway to enable the VMware debug stub.)

Enable the VMware debug stub

We need to enable the VMware debug stub. This is a simple debugging interface included with VMware. It’s not a full debugger, but it speaks the GDB remote debugger protocol, so we can control it from GDB on the debugger machine.

There’s no UI to enable the debug stub; we’ll need to change the VM configuration by hand. Shut down the debuggee VM, and open its configuration file. The configuration file will be located in the VM data directory. If the VM is named debuggee, the configuration file will have the name debuggee.vmx. At the end of the file, add:

debugStub.listen.guest64 = 1

(For 32-bit guests, use debugStub.listen.guest32 = 1 instead.)

Then, start up the debuggee again. Confirm that the nokaslr option is present:

debuggee$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.13.0-19-generic root=UUID=... ro splash quiet nokaslr

Set up the environment on the debugger machine

We’ll copy the environment from the debuggee machine to the debugger machine. This way, we won’t need to use exactly the same OS on both machines.

Copy the the symbols for the running kernel, the kernel source code, and the GDB scripts. On the debugger machine, run:

# Change your paths and addresses as appropriate: below, the debuggee machine is

# We have to be careful when copying the gdb scripts from debian/build: it contains a symlink to its parent.
# We exclude it in the first transfer, and then copy only the scripts in the second step
debugger$ rsync --recursive --relative --copy-links --verbose \
  --exclude '/home/alambert/kernel/source/linux-4.13.0/debian/build' \ :/usr/lib/debug ~/kernel
debugger$ rsync --recursive --relative --copy-links --verbose \ \
  :/home/alambert/kernel/source/linux-4.13.0/debian/build/build-generic/scripts/gdb ~/kernel

Install GDB:

debugger$ sudo apt install --yes gdb

We need to fix up one last thing to use the GDB scripts: the scripts assume the kernel is in the current directory and has the name vmlinux. Make it so:

debugger$ cd ~/kernel
debugger$ ln -s usr/lib/debug/boot/vmlinux-4.13.0-19-generic vmlinux

Finally, start GDB. We’ll compare the address of init_uts_ns between the symbols and the running kernel to make sure everything is configured correctly:

debugger$ gdb

(gdb) file vmlinux
Reading symbols from vmlinux...done.

(gdb) info address init_uts_ns
Symbol "init_uts_ns" is static storage at address 0xffffffff81e10280.

Compare that with the address from the debuggee’s running kernel:

debuggee$ sudo grep ' D init_uts_ns' /proc/kallsyms
ffffffff81e10280 D init_uts_ns

The addresses should match exactly. If the address from /proc/kallsysms is 0000000000000000, ensure that you’re reading the file as root. (Hiding kernel addresses from unprivileged users is a security feature.) If they still don’t match, double-check that KASLR is disabled.

The kernel was built on a build server, so its embedded source paths differ from your local paths. Use info sources to determine the original source path; mine was /build/linux-tt6jd0/linux-4.13.0:

(gdb) info sources
Source files for which symbols have been read in:


Then remap that path to your local source path, so GDB can locate source code automatically:

(gdb) set substitute-path /build/linux-tt6jd0/linux-4.13.0 /home/alambert/kernel/home/alambert/kernel/source/linux-4.13.0

After you’ve done this, you should be able to summon the source code for a symbol:

(gdb) list init_uts_ns
25              struct new_utsname name;
26              struct user_namespace *user_ns;
27              struct ucounts *ucounts;
28              struct ns_common ns;
29      } __randomize_layout;
30      extern struct uts_namespace init_uts_ns;
32      #ifdef CONFIG_UTS_NS
33      static inline void get_uts_ns(struct uts_namespace *ns)
34      {
(gdb) info source
Current source file is /build/linux-tt6jd0/linux-4.13.0/include/linux/utsname.h
Compilation directory is /build/linux-tt6jd0/linux-4.13.0/debian/build/build-generic
Located in /home/alambert/kernel/home/alambert/kernel/source/linux-4.13.0/include/linux/utsname.h

Connect GDB to the debuggee

Finally, we’re ready to connect. VMware exposes its debug server on localhost:8864 (localhost:8832 for 32-bit guests), so instruct GDB to connect to that endpoint:

(gdb) target remote localhost:8864
Remote debugging using localhost:8864
native_safe_halt () at /build/linux-tt6jd0/linux-4.13.0/arch/x86/include/asm/irqflags.h:54
54      }

And we’re connected! We can see the source context:

(gdb) list
49      }
51      static inline __cpuidle void native_safe_halt(void)
52      {
53              asm volatile("sti; hlt": : :"memory");
54      }
56      static inline __cpuidle void native_halt(void)
57      {
58              asm volatile("hlt": : :"memory");
(gdb) info source
Current source file is /build/linux-tt6jd0/linux-4.13.0/arch/x86/include/asm/irqflags.h
Compilation directory is /build/linux-tt6jd0/linux-4.13.0/debian/build/build-generic
Located in /home/alambert/kernel/home/alambert/kernel/source/linux-4.13.0/arch/x86/include/asm/irqflags.h

And the backtrace:

(gdb) bt full
#0  native_safe_halt () at /build/linux-tt6jd0/linux-4.13.0/arch/x86/include/asm/irqflags.h:54
No locals.
#1  0xffffffff8190ddb0 in arch_safe_halt () at /build/linux-tt6jd0/linux-4.13.0/arch/x86/include/asm/paravirt.h:98
        __esi = <optimized out>
        __edx = <optimized out>
        __edi = <optimized out>
        __ecx = <optimized out>
        __sp = 0xffffffff81e03df0 <init_thread_union+15856>
#2  default_idle () at /build/linux-tt6jd0/linux-4.13.0/arch/x86/kernel/process.c:341

We won’t be able to see symbols from loaded kernel modules yet. We’ll load the helper script and then run lx-symbols, which will probe the loaded modules and configure GDB appropriately:

(gdb) source home/alambert/kernel/source/linux-4.13.0/debian/build/build-generic/
(gdb) lx-symbols
loading vmlinux
scanning for modules in /home/alambert/kernel
loading @0xffffffffc025a000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/fs/nls/nls_utf8.ko
loading @0xffffffffc0265000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/fs/isofs/isofs.ko
loading @0xffffffffc0255000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/arch/x86/crypto/crct10dif-pclmul.ko

Use c to continue execution, and Control-C to halt execution. Use n to step over, s to single-step, and finish to finish the current stack frame. Remember that we’re running a build with optimizations, so sometimes variables will be optimized out or the source line may change unexpectedly.

The script will automatically add new symbols as kernel modules are loaded. When I added an IPsec policy, several modules were loaded into the kernel and then automatically loaded into the debugger:

loading @0xffffffffc0271000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/net/xfrm/xfrm_algo.ko
loading @0xffffffffc0276000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/net/key/af_key.ko
loading @0xffffffffc0285000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/net/ipv4/xfrm4_mode_transport.ko
loading @0xffffffffc02ba000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/net/ipv4/esp4.ko
loading @0xffffffffc0280000: /home/alambert/kernel/usr/lib/debug/lib/modules/4.13.0-19-generic/kernel/crypto/authenc.ko

You can use Control-C to halt execution and set new breakpoints in these modules:

(gdb) b pfkey_add
Breakpoint 1 at 0xffffffffc027a290: file /build/linux-tt6jd0/linux-4.13.0/net/key/af_key.c, line 1485.

Thread 1 hit Breakpoint 1, pfkey_add (sk=0xffff88022ebf0400, skb=0xffff880227100b00, hdr=0xffff880231a9a800, ext_hdrs=0xffffc900039d7c50)
    at /build/linux-tt6jd0/linux-4.13.0/net/key/af_key.c:1485
1485    {
(gdb) c

Now you have a working kernel debugger!

Before quitting the debugger, use detach to disconnect. Otherwise, the VM will terminate when the debugger closes. Use quit to exit GDB.

Happy debugging!