Testing on Fedora

This module must run in Qemu. I've included a setup script, setup_vm.sh, that will quickly build and start a virtual machine. Among other things, the script will download a Fedora 43 (kernel 6.17 - 6.18) LTS live iso. Alternatively, you can provide a path to another iso if you already have one on your machine.

Note

All testing completed on an x86_64 system. I have not tested on ARM. Running x86 (32-bit) will either fail or give erroneous results; see the large comment block in the qemu_edu_lkm_read() function to understand why.

Testing Environment:

Linux Dist:         Fedora 43
Kernel Version:     6.18.12-200
System Arch:        x86_64

Running VM on MacOS

My script was designed to run on Linux, and I'm not interested in making this vm script work on all OS's. That said, if you're on mac, you'll likely on have to change a few things in the scripts.

1. In QEMU_ARGS, change accel=kvm to accel=tcg OR accel=hvf [see warning and #3]
2. Change -display vnc=:1 to -display cocoa
3. If you're on a Silicon mac and use HVF acceleration, you should consider using an ARM Fedora ISO.

Warning

I have not tested this solution on Mac because I want to focus on one architecture, x86_64. Silicon Macs are ARM ISA, so you cannot use hardware acceleration, hvf, provided by mac unless you use an ARM fedora ISO instead. Otherwise, you have to select tcg acceleration, which is full cpu-emulation; thus, very slow. In general, when working with Linux, it's preferred to use Linux host.

Update Linux host & Install packages

(For Fedora Linux Host)

sudo dnf -y update
sudo dnf -y upgrade 
sudo dnf -y install qemu

Cloning the Repo and Running a VM

As stated earlier, my host machine is running Fedora 43. However, I am connecting remotely to the VM graphical interface from my Macbook, so display is set to vnc=:1 in the QEMU_ARGS array. If you're working directly on a Linux host (or on Mac), run the following command to view your display options: qemu-system-x86_64 -display help. For mac, you'll likely use cocoa and for Linux, gtk.

git clone https://github.com/mgreenj/Qemu-driver.git
cd Qemu-driver
./setup_vm.sh [optional: /path/to/iso]

If you decided to connect to the VM graphical UI, you'll need a VNC client, such as ReanVNC or TigerVNC. I'm using the latter. After downloading the ISO, creating a disk, and booting the VM, the setup script will drop you into a QEMU shell. DO NOT EXIT this shell! Not only is it useful for advanced debugging and verification, it will also enable you to eject the cdrom iso from the VM, after the OS install completes, without requiring a reboot.

(qemu) `info block`
virtio0 (#block178): fedora_dev.qcow2 (qcow2)
    Attached to:      /machine/peripheral-anon/device[3]/virtio-backend
    Cache mode:       writeback

ide2-cd0 (#block358): Fedora-Server-dvd-x86_64-43-1.6.iso (raw, read-only)
    Attached to:      /machine/unattached/device[19]
    Removable device: not locked, tray open
    Cache mode:       writeback
(qemu) `eject ide2-cd0`
(qemu)

Mounting Local Files

The repository directory will be allowed to passthrough so that you can mount it. Unfortunately, I'm not aware of any methods for automatically running a startup script in a Qemu VM; however, the good news is that mounting is likely trivial for anyone following this guide.

# In the VM guest

mkdir -p ~/host_files
sudo mount -t 9p -o trans=virtio hostshare ~/host_files

Compiling & Loading the Module

# In the Repo directory

make
make load

Verify Module Load

To verify, you should check the /dev/ directory as well as dmesg output. The following commands should be run; example output for each is included below.

Verification Commands

dmesg | tail -n10
ls -l /dev/qemu_edu_driver

(Example) Verification Output

dmesg output

root@virtme-ng:/home/user/Qemu-driver# dmesg | tail -n10
[   54.993423] qemu_edu_driver: loading out-of-tree module taints kernel.
[   54.993437] qemu_edu_driver: module verification failed: signature and/or required key missing - tainting kernel
[   54.994524] qemu_edu_driver 0000:00:01.0: EDU Device Probed! Vendor: 0x1234, Device: 0x11e8
[   54.994782] qemu_edu_driver 0000:00:01.0: Misc device registered as /dev/qemu_edu_driver
[   54.994845] qemu_edu_driver 0000:00:01.0: enabling device (0000 -> 0002)
[   54.995960] ACPI: \_SB_.L001: Enabled at IRQ 17
[   54.996209] qemu_edu_driver 0000:00:01.0: MMIO mapped at virtual address: 0000000073f0388a
[   54.996223] qemu_edu_driver 0000:00:01.0: DMA Buffer: CPU-virtual=0000000094e71c55, Bus-physical=0x00000000026d7000
[   54.996229] qemu_edu_driver 0000:00:01.0: [qemu_edu_driver] Registering IRQ handler
[   54.997518] qemu_edu_driver 0000:00:01.0: Probe complete. Driver is ready.
root@virtme-ng:/home/user/Qemu-driver#

device file

root@virtme-ng:/home/user/Qemu-driver# ls -l /dev/qemu_edu_driver

crw------- 1 root root 10, 260 Feb 25 12:50 /dev/qemu_edu_driver
root@virtme-ng:/home/user/Qemu-driver#

Learning Point

You'll notice the addresses shown in the dmesg output above all begin with the 0x0000. Recall that virtual address space is split between User (or Process) Virtual Address Space (VAS) and Kernel Virtual Address Space (KVAS).

When Linux is running in 32-bit mode, the virtual addresses are 32 bits. However, when Linux is running in 64-bit mode, the virtual addresses are only 48 bits. The remaining 16 (MSB) bits act as signage.

• For virtual addresses belonging to User VAS, the 16 MSB bits are set to zero.
• For virtual addresses belonging to Kernel VAS (KVAS), the 16 MSB bits are set to one.

Virtual memory addresses beginning with 0x000 belong to user Virtual Address Space, and virtual addresses beginning with 0xFFFF belong with Kernel Virtual Address Space.

Verify DMA Read Operation

Verify Interrupt

Important

This test was executed before implementing a function, so zeros in hexdump is expected.

root@virtme-ng:/home/user/Qemu-driver# hexdump -C -n 32 /dev/qemu_edu_driver
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000020
root@virtme-ng:/home/user/Qemu-driver#
root@virtme-ng:/home/user/Qemu-driver# cat /proc/interrupts | grep qemu_edu
 53:          0          0          0          0          0          4          0          0  IO-APIC  17-fasteoi   qemu_edu_driver
root@virtme-ng:/home/user/Qemu-driver#
root@virtme-ng:/home/user/Qemu-driver# hexdump -C -n 32 /dev/qemu_edu_driver
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000020
root@virtme-ng:/home/user/Qemu-driver#
root@virtme-ng:/home/user/Qemu-driver# cat /proc/interrupts | grep qemu_edu
 53:          0          0          0          0          0          5          0          0  IO-APIC  17-fasteoi   qemu_edu_driver
root@virtme-ng:/home/user/Qemu-driver#

The output above shows the interrupt counter increasing on CPU5. Notice how the interrupts are all handled by threads on CPU5. The EDU device supports INTx (legacy) and MSI interrupts; however, INTx are enabled by default. While MSI is not as powerful as its successor, MSI-X, it should allow us to distribute interrupt handling for our module accross multiple CPUs.

root@virtme-ng:/home/mgre2324/Qemu-driver# lspci -v
00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
	Subsystem: Red Hat, Inc. Device 1100
	Flags: fast devsel

00:01.0 Unclassified device [00ff]: Device 1234:11e8 (rev 10)
	Subsystem: Red Hat, Inc. Device 1100
	Flags: bus master, fast devsel, latency 0, IRQ 53
	Memory at c0000000 (32-bit, non-prefetchable) [size=1M]
	Capabilities: [40] MSI: Enable- Count=1/1 Maskable- 64bit+
	Kernel driver in use: qemu_edu_driver