COMP 630: Take-Home Final: Linux Kernel Extensions

Due on Weds, May 8, 2024, 11:59 PM
 Note: You may not use late hours on this take-home exam. You may complete this assignment in teams of up to 3, which do not necessarily have to be the same team as the rest of the semester.

Introduction

This take-home exam will introduce you to Linux kernel programming and OS security issues, by implmenting a few simple commands in a kernel, followed by a back-dorr that escalates privilege. This lab is designed to be a bit more open-ended.

The course staff have provided you with a starter code that provides some useful scaffolding.

Getting the Starter Code

You will need to click on this link to create a private repository for your code.

Once you have a repository, you will need to clone your private repository (see the URL under the green "Clone or Download" button, after selecting "Use SSH". For instance, if your private repo is called final-team-don:

walter% git clone git@github.com:comp630-s24/final-team-don.git

Development Environment

There are a number of ways to do kernel development, but we are going to follow a methodology similar to what we have done with JOS - run our test kernel under qemu, and attach using gdb.

For Your Safety

We assume you will work primarily on walter, where you don't have administrator privilege. If you find yourself wanting to work on another system, please note:

Modifying the OS kernel on your system can lose all data on the system! If you introduce a null pointer in a regular program, it crashes and loses all of its data; the same is true of an OS kernel. If you introduce a bug in the OS, it will crash. When an OS crashes, it can corrupt the file system and lose all of your data (but we hope it won't). Thus, it is essential that you do two things to protect yourself.

Push your code to another machine before testing. Before you install and test kernel code, be sure to use git to commit and push your code to another location (e.g., github). That way, if the file system is corrupted, you don't lose your work. If you don't want to inflict untested code on your teammates, create a branch in git.

Building a minimal Linux kernel

For this assignment, we will use Linux version 6.8.7. Start by downloading the source from here: 

wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.8.7.tar.xz
tar xf linux-6.8.7.tar.xz

Normally, you would configure the build options using make menuconfig. However, we will provide you with a build configuration file as part of the starter code, called qemu.config, which you should copy into the kernel source directory and then compile Linux:

cp qemu.config linux-6.8.7/.config
cd linux-6.8.7
make olddefconfig
make -j 24

At this point your uncompressed kernel is in vmlinux, and the compressed boot images is in arch/x86/boot/bzImage.

Building the starter code

In the lab4 directory, type make to build the 630hax.ko kernel module. The makefile assumes that the kernel is in ./linux-6.8.7 directory under the starter code; if you put the kernel source elsewhere, you can specify this with the environment variable KERNELDIR.

Disk Image

Download a disk image here, which is a file that qemu will treat like a disk. The disk image contains a simple Ubuntu file system.

You may also build the image from scratch, using ./build-img.sh, on your own system (this script requires root privilege); the result will be the same (and it will take longer).

We provide several scripts that will allow you to mount the image (mount-img.sh), unmount it (umount-img.sh), copy in the module (cp-module.sh), and boot a VM (startvm.sh). A typical workflow might look like this: 

make
./cp-module.sh
./startvm.sh

A window should pop up with the console. The root account on the image has no password. You will need an ssh connection that forwards the X11 protocol for this to work. To do this, use the ssh -Y walter command. This also requires you to install an X11 server on your laptop/desktop system. For Linux distributions, this will be there by default. For OS X, you can use XQuartzVcXsrvor PuTTY. You can also install Linux in a VM.

Further note: if you are off campus, you will need to install the UNC Cisco VPN client for this to work. There are reports that VcXsrv does not work off campus, but putty does.

Once you log in as root, you can load the kernel module by typing insmod ./630hax.ko; the module can be unloaded by typing rmmod 630hax. You can view all loaded modules by reading the output of the lsmod command. You can confirm that the 630hax module was loaded by checking the output of dmesg: 

[   20.467846] init: plymouth-upstart-bridge main process (821) killed by TERM signal
[  162.349918] init: tty1 main process ended, respawning
[  225.170703] init: tty1 main process ended, respawning
[  279.530944] init: tty1 main process ended, respawning
[ 5822.253745] Hello, comp 630 world

A note on VcXsrv:

For the Windows users, Rohan Wagle kindly shared the following notes on VcXsrv:

After installing, to start VcXsrv for some reason it's called XLaunch in Windows search. Search for XLaunch and launch it, then click the option for "Multiple Windows". Keep Display Number set to -1. Then press Next. Choose "Start no client" and press Next. Then keep all the checkboxes selected on the next screen, including "Disable access control". Press Next and then Finish. The VcXsrv window will disappear, but it will be in your Windows taskbar tray. You can kill it later by right-clicking its tray icon and clicking Exit.

Open Windows Terminal or Putty and ssh into the remote Linux machine, using the -X flag in the ssh command. Once you're logged into the Linux machine, set the DISPLAY environment variable to your Windows machine's ip address with ":0" at the end. For example, if my Windows machine's ip address on the network is 192.168.1.123, then I'd enter the following on the Linux machine ssh session: 

export DISPLAY="192.168.1.123:0"

Then you should be able to open an Xwindow application in Windows coming from the Linux machine.

Debugging the kernel module

Similar to JOS, we can use gdb to attach to and debug the kernel and our module. In one terminal instance, type:

$ ./startvm.sh -g
gdb is listening on port 6000

Note that your gdbport will vary and may not be 6000 (each student should get a unique one). But you will need to note it for use below:

In another terminal, type:

$ gdb ./linux-6.8.7/vmlinux
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
    .

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./linux-6.8.7/vmlinux...
(gdb) target remote localhost:6000
Remote debugging using localhost:6000
0x000000000000fff0 in exception_stacks ()
(gdb) c

Once you have the kernel running, you will need to load your module. However, you need to know the base address for the symbols, which is determined dynamically. You can get this information from /proc/modules, but you must read this as root: 

$insmod 630hax.ko
$cat /proc/modules
630hax   12288 0 - Live 0xffffffffc0118000 (0)

The value 0xffffffffc0118000 is the base address of the kernel module. This will change from run-to-run, unfortunately. So you will need to get this value, and teach gdb where to find your module source. In the gdb window:

(gdb) add-symbol-file 630hax.ko 0xffffffffc0118000
add symbol table from file "630hax.ko" at
	.text_addr = 0xffffffffc0118000
(y or n) Y
Reading symbols from 630hax.ko...
(gdb) b exit_630hax
Breakpoint 1 at 0xffffffffc0118280: exit_630hax.
(gdb) c
Continuing.

Once you unload the modulew with rmmod 630hax, you should get a breakpoint in gdb: 

Thread 1 hit Breakpoint 1, exit_630hax () at /home/porter/comp630-s24/assignments/final/630hax.c:144
144	{
(gdb) bt
#0  exit_630hax () at /home/porter/comp630-s24/assignments/final/630hax.c:144
#1  0xffffffff81139ea2 in __do_sys_delete_module (name_user=, flags=)
    at kernel/module/main.c:755
#2  0xffffffff81139f97 in __se_sys_delete_module (flags=, name_user=)
    at kernel/module/main.c:698
#3  __x64_sys_delete_module (regs=) at kernel/module/main.c:698
#4  0xffffffff81003c51 in x64_sys_call (regs=regs@entry=0xffffc900002f7f58, nr=)
    at ./arch/x86/include/generated/asm/syscalls_64.h:177
#5  0xffffffff81f3227b in do_syscall_x64 (nr=, regs=0xffffc900002f7f58) at arch/x86/entry/common.c:52
#6  do_syscall_64 (regs=0xffffc900002f7f58, nr=) at arch/x86/entry/common.c:83
#7  0xffffffff82000135 in entry_SYSCALL_64 () at arch/x86/entry/entry_64.S:121
#8  0x00005644116df480 in ?? ()
#9  0x00007ffe9290b89c in ?? ()
#10 0x00005644116de2a0 in ?? ()
#11 0x00007ffe9290a590 in ?? ()
#12 0x0000000000000000 in ?? ()
(gdb) c

And, from here, you can use gdb as with a user-space program --- setting break points, inspecting local variables, etc.

Understanding the Skeleton Code

All kernel modules provide an init and exit method, which are called when the module is loaded and unloaded, respectively. This is currently all the 630hax module includes; many modules will create devices or register other callbacks.

The 630hax module also provides examples of how to find kernel functions. Allowed functions are exported explicitly as symbols: a module can simply call these.

Note: You may not need the particular functions we provide or call in the starter code---don't feel like you are doing anything wrong if you don't use them. These are provided only as examples of how to find functions you may need.

Helpful Resources

The best resource to finding kernel function is the Linux Cross-Reference (LXR), located at http://elixir.bootlin.com/. This site includes a number of useful features that can help you find your way through the source code.

These books (available through the campus Safari Online subscription), also are a helpful reference in understanding Linux kernel code:

Core Assignment

The first exercise will be to write a simple /proc file. A proc file is a pseudo-file --- it is not persistent, and really just a file interface to some kernel variable or data structure. The proc file system can also be thought of as an alternative system call interface.

Exercise 1. (10 points) Create a file, /proc/running_total that implements a simple running total in the kernel. When a user writes a number into this file, it is added to the running total. When a user reads the file, they should see the sum of all previous numbers written to the file.
For example: 

# echo 3 > /proc/running_total
# echo 2 > /proc/running_total
# cat /proc/running_total
5
# echo 0 > /proc/running_total
# cat /proc/running_total
5
# echo -5 > /proc/running_total
# cat /proc/running_total
0

We provide some starter code. We suggest starting with the read hook, which will initially just return zero until the write hook is implemented.
Be sure to use a lock to properly synchronize accesses to the total. We suggest using a spin_lock, since one can update this structure without blocking.

The second exercise will be a bit more complicated - requiring you to allocate linked list nodes to store data, iterate over them, and add entries in sorted order.

Exercise 2. (15 points) Create a second file, /proc/sorted_list keeps a linked list of that implements a sorted list (ascending) of numbers written to the file. When one reads it, they get a line-by-line listing of the numbers that were previously entered, in sorted order.
For example: 

# echo 4 > /proc/sorted_list
# echo 0 > /proc/sorted_list
# echo -3 > /proc/sorted_list
# echo -3 > /proc/sorted_list
# echo -2938 > /proc/sorted_list
# echo 3934 > /proc/sorted_list
# cat /proc/sorted_list
-2938
-3
-3
0
4
3934

As before, be sure to use a lock to protect the list! You may use the same spin lock as Exercise 1. Also, be sure to free the list nodes when the module exits.

The final exercise will demonstrate the power of a kernel module in an unsafe language in one big address space: the ability to make unsanctioned changes to a data structure. Here, we will get the current process's task struct (i.e., thread control block) and modify the current process's id, or pid.

Exercise 3. (10 points) Create a third, write-only file, /proc/set_pid that allows the user to overwrite their pid(!!). In other words, if my process is initially pid 400, and we write 390 into the file, subsequent calls to getpid() from the process should return 390.
Because the shell forks heavily, one cannot easily test this with shell commands. We have provided a test utlity, called pid_test that prints the current pid, writes the new pid to the proc file for you, and prints the result of a subsequent getpid() call. 

# ./pid_test 400
Setting pid to 400
My initial pid is 169
My new pid is 400
Here, 400 is the desired pid. 169 is the original pid of this process, and will change from run to run.
Note that Linux has some complexity around namespaces and pid sharing for threads. We are also doing some things here that are not supported, in part by copying code from the kernel that is not exported as a module API. As a result the code has a few notes about how our style is not what you should do in real code. Specifically, we just need to leak pid structures, and we abuse rcu_dereference() to write.

This completes the take-home final. As usual, hand in your work to gradescope. Please be sure not to include the Linux source or kernel binary in your handin - we will use the same Linux kernel (6.8.7) and configuration file (qemu.config) for testing and grading. There is no auto-grader; these will be hand-graded.

Last updated: 2024-05-05 13:57:17 -0400 [validate xhtml]