Using qemu to debug kernel (by quqi99)

Posted 2022-01-01 quqi99

作者：张华 发表于：2021-12-06
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明
( http://blog.csdn.net/quqi99 )

五年前做过类似的事，见: https://blog.csdn.net/quqi99/article/details/50640902

Prepare Kernel

sudo apt install qemu qemu-kvm bridge-utils virt-manager -y
sudo apt install bison flex libelf-dev libssl-dev build-essential libncurses-dev -y
sudo apt install libncurses5-dev libncursesw5-dev -y
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
cd linux-2.6
#git checkout -b v5.9 v5.9
make defconfig
#cp -v /boot/config-$(uname -r) .config
./scripts/config -e DEBUG_INFO -e GDB_SCRIPTS
make menuconfig
make -j $(nproc) bzImage
#sudo make modules_install
#sudo make install

./scripts/config -m E1000
make SUBDIRS=drivers/net/ethernet/intel/e1000 modules
ls drivers/net/ethernet/intel/e1000/e1000.ko

Prepare Root fs

# Way 1
#mv rootfs.img ./core/rootfs.cpio.gz 
#cd core
#gunzip rootfs.cpio.gz
#cpio -idmv < rootfs.cpio
#rm -rf rootfs.cpio
# Way 2
#sudo apt-get install debootstrap
#wget https://raw.githubusercontent.com/google/syzkaller/master/tools/create-image.sh -O create-image.sh
#chmod +x create-image.sh
#./create-image.sh
# Way 3
cd /bak/work/linux/rootfs
wget http://cdimage.ubuntu.com/cdimage/ubuntu-base/releases/20.04/release/ubuntu-base-20.04.1-base-amd64.tar.gz
dd if=/dev/zero of=rootfs.img bs=1024 count=1M
mkfs.ext4 -F -L linuxroot rootfs.img
sudo mkdir -p /mnt/tmpdir && sudo chown -R $USER /mnt/tmpdir
sudo mount -o loop rootfs.img /mnt/tmpdir/
sudo tar zxvf ubuntu-base-20.04.1-base-amd64.tar.gz -C /mnt/tmpdir/
sudo cp /etc/resolv.conf /mnt/tmpdir/etc/
sudo mount -t proc /proc /mnt/tmpdir/proc
sudo mount -t sysfs /sys /mnt/tmpdir/sys
sudo mount -o bind /dev /mnt/tmpdir/dev
sudo mount -o bind /dev/pts /mnt/tmpdir/dev/pts
sudo chroot /mnt/tmpdir
apt update
apt-get install language-pack-en-base sudo ssh net-tools ethtool wireless-tools ifupdown \\
  network-manager iputils-ping rsyslog htop vim xinit xorg alsa-utils --no-install-recommends
useradd ubuntu
passwd ubuntu
echo "debug" > /etc/hostname
cat << EOF | sudo tee -a /etc/hosts
127.0.0.1 localhost
127.0.0.1 debug
EOF
echo "allowed_users=anybody" > /etc/X11/Xwrapper.config
dpkg-reconfigure tzdata
exit
sudo umount /mnt/tmpdir/proc/
sudo umount /mnt/tmpdir/sys/
sudo umount /mnt/tmpdir/dev/pts/
sudo umount /mnt/tmpdir/dev/
sudo umount /mnt/tmpdir/

Debug Kernel

sudo qemu-system-x86_64 -s -S -m 1024M -smp 4 \\
  -kernel /bak/linux/linux-2.6/arch/x86/boot/bzImage \\
  -append "nokaslr root=/dev/sda console=ttyS0" \\
  -drive file=/bak/work/linux/rootfs/rootfs.img,format=raw,index=0,media=disk \\
  --enable-kvm -cpu host -nographic

cgdb /bak/linux/linux-2.6/vmlinux
(gdb) set architecture i8086
(gdb) target remote :1234
(gdb) hbreak start_kernel
(gdb) b start_kernel
(gdb) c
(gdb) bt
(gdb) list
# Inside VM, echo 'c' | sudo tee /proc/sysrq-trigger
(gdb) add-symbol-file vmlinux 0xffffffff81000000 #echo 0x$(sudo cat /proc/kallsyms | egrep -e "T _text$" | awk 'print $1')
(gdb) b sysrq_handle_crash

Use initramfs instead

cd /bak/work/linux/busybox
wget https://busybox.net/downloads/busybox-1.34.1.tar.bz2
tar -xf busybox-1.34.1.tar.bz2
cd busybox-1.34.1
make menuconfig
# Settings  ---> [*] Build static binary (no shared libs)
make -j$(nproc) && make install

cd _install/
mkdir proc sys
cat << EOF | tee init
#!/bin/sh
echo "==DBG== INIT SCRIPT"
mkdir /tmp
mount -t proc none /proc
mount -t sysfs none /sys
mount -t debugfs none /sys/kernel/debug
mount -t tmpfs none /tmp
mdev -s
echo -e "==DBG== Boot took $(cut -d' ' -f1 /proc/uptime) seconds"
setsid /bin/cttyhack setuidgid 1000 /bin/sh
EOF
chmod +x init
find . | cpio -o --format=newc > ./rootfs.img

qemu-system-x86_64 -kernel /bak/linux/linux-2.6/arch/x86/boot/bzImage \\
  -initrd  /bak/work/linux/busybox/busybox-1.34.1/_install/rootfs.img \\
  -append "nokaslr console=ttyS0" -nographic -s -S
cgdb /bak/linux/linux-2.6/vmlinux
(gdb) target remote :1234

Failed to use eclipse due to ‘Failed to execute MI command’

# download eclipse PlatformRuntime version
# https://archive.eclipse.org/eclipse/downloads/drops4/R-4.21-202109060500/#PlatformRuntime
# As of 2017, Goclipse is no longer actively maintained, We use GoSublime instead of eclipse for Go
# Install Pydev - http://www.pydev.org/updates
# Install CDT(https://www.eclipse.org/cdt/downloads.php) - https://download.eclipse.org/tools/cdt/releases/10.4

# use cdt to debug kernel
Import existing C/C++ Project -> Makefile Project with Existing Code, select /bak/linux/linux-2.6
Run -> Debug configurations -> C/C++ Attach to Application -> Right click to create,

in 'Debugger' TAB
Debugger: gdbserver
Type: TCP
Host name or IP address: node1
Port number: 1234

in 'Main' TAB
select '/bak/linux/linux-2.6/vmlinux' as C/C++ Application
select 'Disable auto build

Failed to execute MI command:
settings --> Debugger --> Main Tab --> Browse to gdb compiled for the same platform
/usr/bin/gdb

Reference

[1] https://www.jeanleo.com/2020/08/30/qemu%E8%B0%83%E8%AF%95%E5%86%85%E6%A0%B8%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/
[2] https://cloud.tencent.com/developer/article/1793157
[3] https://cloud.tencent.com/developer/article/1793157
[4] https://gist.github.com/adwait1-G/5b4ae85239619e6f94b01cfb1bdb7dd6
[5] http://nickdesaulniers.github.io/blog/2018/10/24/booting-a-custom-linux-kernel-in-qemu-and-debugging-it-with-gdb/