QEMU and GDB debugging a kernel module
Hi everyone, I am able to run qemu and get the *.iso files to boot fine. The qemu image works as well. But for the life of me I simply cannot get gdb to trigger when I set a break point in my kernel module ? I had trouble on my Debian 64bit, gdb complained saying "remote g packet too long" so I patched the remote.c file in gdb and things seemed okay but I am not sure if this did not introduce more bugs. And qemu wouldn't drop me to a shell but hung saying cannot find /bin/init.. So, to double check I ran Ubuntu-MATE 32 bit and all seemed well. But it still will not trigger gdb on my kernel module's break point. Has anyone ever used qemu and gdb ? If so has anyone successfully debugged a module ? I would be very thankful if someone can show me the 'specific' steps to follow to get this going. Documentation in the public domain is sparse and am yet to find something that gives a detailed breakdown of the steps and process. Thank's - Aruna
On Fri, Aug 21, 2015 at 12:39 AM, Aruna Hewapathirane <aruna.hewapathirane@gmail.com> wrote:
Has anyone ever used qemu and gdb ? If so has anyone successfully debugged a module ? I would be very thankful if someone can show me the 'specific' steps to follow to get this going.
I have done this once upon a time... checking the scripts I used: Step 1: launch qemu with -gdb like so: qemu-system-x86_64 -nographic -kernel /path/to/kernel -gdb tcp:10.0.0.1:1234 [all the other options] (In my case 10.0.0.1 is bridged with the host.) Step 2: Make a gdbinit file that looks like: set architecture i386:x86-64:intel target remote localhost:1234 file vmlinux # this part is deep magic - check /proc/modules for correct offsets add-symbol-file net/wireless/cfg80211.o 0xffffffffa00000000 # and so-on for whatever modules you care about continue Step 3: Launch gdb on the host YMMV -- I rarely have used an actual debugger for my own kernel work. I do frequently use gdb to do the same thing as addr2line by running gdb on the .ko and then "l *addr+0x123" -- to resolve symbols from oops messages, but you don't need qemu or remote gdb for that. -- Bob Copeland %% www.bobcopeland.com
On Fri, Aug 21, 2015 at 12:39 AM, Aruna Hewapathirane wrote:
Has anyone ever used qemu and gdb ? If so has anyone successfully debugged a module ?
I haven't done that. I have seldom used gdb directly. If I'm doing debugging I use gdb via ddd. The ddd program makes debugging a lot easier (for me) than the command line based gdb. -- Cheers! Kevin. http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick
On Tue, Sep 01, 2015 at 09:43:06PM -0400, Kevin Cozens wrote:
I haven't done that. I have seldom used gdb directly. If I'm doing debugging I use gdb via ddd. The ddd program makes debugging a lot easier (for me) than the command line based gdb.
Last I looked at it (a long time ago), ddd had to be one of the worst, most buggy, awful ui, pieces of crap I ever encountered. Even makes eclipse seem like a good idea in comparison. So bad I have never ever considered looking at it again. I assumed it had to have died out from being so bad. -- Len Sorensen
On Thu, Sep 3, 2015 at 11:41 AM, Lennart Sorensen < lsorense@csclub.uwaterloo.ca> wrote:
On Tue, Sep 01, 2015 at 09:43:06PM -0400, Kevin Cozens wrote:
I haven't done that. I have seldom used gdb directly. If I'm doing debugging I use gdb via ddd. The ddd program makes debugging a lot easier (for me) than the command line based gdb.
Last I looked at it (a long time ago), ddd had to be one of the worst, most buggy, awful ui, pieces of crap I ever encountered. Even makes eclipse seem like a good idea in comparison. So bad I have never ever considered looking at it again. I assumed it had to have died out from being so bad.
Never had the pleasure. But Kevin, don't fear the command line. A well-designed graphical debugger is a good and useful tool, but when you absolutely need to get into the plumbing; memory setup, ICE hardware protocols, etc. (ad nauseam) you almost inevitably go sideways from where the GUI designers intended. Kvm-qemu kernel debugging probably falls squarely in this category. All roads lead to gdb, it sometimes seems... Mike
On Thu, Sep 03, 2015 at 12:21:50PM -0400, Mike wrote:
Never had the pleasure. But Kevin, don't fear the command line. A well-designed graphical debugger is a good and useful tool, but when you absolutely need to get into the plumbing; memory setup, ICE hardware protocols, etc. (ad nauseam) you almost inevitably go sideways from where the GUI designers intended. Kvm-qemu kernel debugging probably falls squarely in this category. All roads lead to gdb, it sometimes seems...
gdb's command line is certainly overwhelming to many, but if I was really desparate for an interface to gdb, I would use emacs, and I am a vim user. -- Len Sorensen
On Fri, Sep 04, 2015 at 06:58:25AM -0400, Aruna Hewapathirane wrote:
<snip>
gdb's command line is certainly overwhelming to many, but if I was really desparate for an interface to gdb, I would use emacs, and I am a vim user. Len Sorensen <snip>
Len, try qemu it is a zillion times more overwhelming than gdb :)
Oh yes, the qemu command line is quite something. I do find the syntax more understandable than gdb though. -- Len Sorensen
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/09/15 09:33 AM, Lennart Sorensen wrote:
On Fri, Sep 04, 2015 at 06:58:25AM -0400, Aruna Hewapathirane wrote:
<snip>
gdb's command line is certainly overwhelming to many, but if I was really desparate for an interface to gdb, I would use emacs, and I am a vim user. Len Sorensen <snip>
Len, try qemu it is a zillion times more overwhelming than gdb :)
Oh yes, the qemu command line is quite something. I do find the syntax more understandable than gdb though.
That is probably because you have more gray matter ( gray matter in this case equates to long years of lived experience + testing + breaking things then wash+rinse+repeat till you finally fix things.. ) I still *cringe* when I have to fire up qemu.. hopefully someday I will feel comfortable but right now every single time I do this I actually do PRAY ! ( Seriously I kneel then PRAY ! yes dat bad.. ) Len, if I may please I am setting up a few servers for a good friend and they are running Zimbra Servers and Zimbra has it's own bundled Apache so I do not see the usual /var/www root and the log files are huge. Seriously huge. Any suggestions on how best to tweak for performance ? I also see free -m -l -t telling me it is hogging memory like anything. Is this a Zimbra issue or ? Once again how best to resolve ? The logs are purged daily so am guesing teh amount of transactions through Zimbra is causing the weight ? This is my first exposure to Zimbra and Servers actually so please have mercy and be gentle. Thanks - Aruna -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBAgAGBQJV6aV2AAoJEHIEJxfB21ta5f0P/0zc2Yu1qQXdgcxGaUWIGhi7 rCBiWkwrhZU3W8TTD6HBzGL471eU90vqkkJSpwSy+Fb9KAs7wIZz8jaVURrL6rkP 5X2lAIShk6kz58Xc1lmoI7cUN9urbqmV6TPdBxAhx/b/E9vzorzsIgVGMttuzvZz fuER+3Fm/Pmt3Kpeyx9JRpS6q8Dq6kQt5BJpY3bJTF05i5VERdH9CUFZS3PaTYv6 9V4gXLiAafj8d2kQtEcPkLJVbN2/UN2s1tMku3fW/MyLQtCxH/evX5U5RRiex472 xuNRzFvZ+yEfFwkSvV4gqxXjrt7AhBPaghKHiG0AIGVQDun5VJR9dd+DjB7MW9jJ yScU8N+m+0OEmlMhua8I9kRiW5GuapS8xTECp/bLeJw1//tGGsvPh4ZdwmKoq3HK 0cg2uSfpYdjJy2oa+aDhbCBLmIkOPatInwVVf50yuu4dL1DK6p6nBn38+EuwbfBk JoPSGTihENSRZlZvGrVz1lzXikdUpIn1tn3J0srW9UYrKoruWFmGAzK6ZH6HvSVL Wm+Jl+bF570a3RBYVDtCRPMpXKUGd0blWxi9AXPVhqvpVVkzdT/1qMzvEJawX608 Jo+Gg8igfThLaSz5TtBkiLwbyEI9aqGWXxe2lsdW5UM9HHvdBfOcRPPuuecdO1dc cbrc5Zj6tbmQSKK47BB6 =0WjX -----END PGP SIGNATURE-----
On Fri, Sep 04, 2015 at 10:06:46AM -0400, Aruna Hewapathirane wrote:
That is probably because you have more gray matter ( gray matter in this case equates to long years of lived experience + testing + breaking things then wash+rinse+repeat till you finally fix things.. )
I still *cringe* when I have to fire up qemu.. hopefully someday I will feel comfortable but right now every single time I do this I actually do PRAY ! ( Seriously I kneel then PRAY ! yes dat bad.. )
Len, if I may please I am setting up a few servers for a good friend and they are running Zimbra Servers and Zimbra has it's own bundled Apache so I do not see the usual /var/www root and the log files are huge. Seriously huge. Any suggestions on how best to tweak for performance ?
Never heard of Zimbra before, so no idea really.
I also see free -m -l -t telling me it is hogging memory like anything. Is this a Zimbra issue or ? Once again how best to resolve ?
The logs are purged daily so am guesing teh amount of transactions through Zimbra is causing the weight ?
This is my first exposure to Zimbra and Servers actually so please have mercy and be gentle.
Better to find someone that knows something about Zimbra. -- Len Sorensen
On 15-09-03 09:17 PM, Lennart Sorensen wrote:
gdb's command line is certainly overwhelming to many, but if I was really desparate for an interface to gdb, I would use emacs, and I am a vim user.
Another program that has an almost overwhelming command line is avrdude. Fortunately I knew of avrdude-gui as a way to tame it. It was old but I updated it for use with a current version of avrdude. One of these days I'll post the updated version of avrdude-gui to my website. -- Cheers! Kevin. http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick
On Thursday, September 03 2015, Lennart Sorensen wrote:
On Thu, Sep 03, 2015 at 12:21:50PM -0400, Mike wrote:
Never had the pleasure. But Kevin, don't fear the command line. A well-designed graphical debugger is a good and useful tool, but when you absolutely need to get into the plumbing; memory setup, ICE hardware protocols, etc. (ad nauseam) you almost inevitably go sideways from where the GUI designers intended. Kvm-qemu kernel debugging probably falls squarely in this category. All roads lead to gdb, it sometimes seems...
gdb's command line is certainly overwhelming to many, but if I was really desparate for an interface to gdb, I would use emacs, and I am a vim user.
Emacs has a good (or gud?) interface to GDB, but sometimes it lacks one feature or another. What I recommend to every GDB beginner (or even non-beginners, because they also don't know about this feature) is TUI, the Text User Interface inside GDB itself. It is ncurses-based, and after a long period of being unmaintained (and buggy), it has been receiving some TLC again lately. To enter TUI, press C-x a. To leave TUI, C-x a again. -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/
On 15-09-03 12:21 PM, Mike wrote:
But Kevin, don't fear the command line. A well-designed graphical debugger is a good and useful tool, but when you absolutely need to get into the plumbing; memory setup, ICE hardware protocols, etc. (ad nauseam) you almost inevitably go sideways from where the GUI designers intended.
I don't fear the command line. I just don't use gdb enough to know all its commands. I would need a cheat sheet in order to use it effectively from the command line. The extent of my command line use of gdb is limited to loading a core dump, loading the file with the symbol table and getting a backtrace. I find it convenient to be able to watch the contents of multiple variables and easily dig through structures in GUI front-ends to gdb. I haven't gotten in to some of the other types of programming/issues you mentioned so ddd has been ok for me so far. I read about a way to set up ddd for debugging of AVR microcontrollers using the AVR version of gdb but I haven't tried doing that yet. Using avr-gdb could get me into using gdb from the command line if I find I really need to use it. -- Cheers! Kevin. http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick
On 15-09-03 11:41 AM, Lennart Sorensen wrote:
Last I looked at it (a long time ago), ddd had to be one of the worst, most buggy, awful ui, pieces of crap I ever encountered.
I know what you mean. When I first tried ddd it would often crash in the middle of your debug session if you looked at it wrong, but it did recover. The UI hasn't changed a whole lot from the early days but it has gotten a bit less prone to crash. It usually works for me so I haven't bothered to look around and see if there is something better that can be used in place of ddd. -- Cheers! Kevin. http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick
On Thursday, September 03 2015, Kevin Cozens wrote:
On 15-09-03 11:41 AM, Lennart Sorensen wrote:
Last I looked at it (a long time ago), ddd had to be one of the worst, most buggy, awful ui, pieces of crap I ever encountered.
I know what you mean. When I first tried ddd it would often crash in the middle of your debug session if you looked at it wrong, but it did recover.
The UI hasn't changed a whole lot from the early days but it has gotten a bit less prone to crash. It usually works for me so I haven't bothered to look around and see if there is something better that can be used in place of ddd.
If you're a GNOME user you might give Nemiver a try, too. <https://wiki.gnome.org/Apps/Nemiver> -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/
On Tue, Sep 1, 2015 at 9:43 PM, Kevin Cozens <kevin@ve3syb.ca> wrote: On Fri, Aug 21, 2015 at 12:39 AM, Aruna Hewapathirane wrote: Has anyone ever used qemu and gdb ? If so has anyone successfully debugged a module ?
I haven't done that. I have seldom used gdb directly. If I'm doing debugging I use gdb via ddd. The ddd program makes debugging a lot easier (for me) than the command line based gdb.
Thanks Kevin I will take a look at ddd. Aruna
On 15-09-04 06:56 AM, Aruna Hewapathirane wrote:
Thanks Kevin I will take a look at ddd.
I did an internet search a day or two back. There are other alternatives to ddd for gdb front-ends and found a list of them at https://sourceware.org/gdb/wiki/GDB%20Front%20Ends One of them that looks interesting is Gede. It isn't available for the distro I'm running (Linux Mint) so it would have to be built from the source files. For now I'll stick with ddd. I'm used to it and I seldom need a source level debugger so I'm not particularly motivated to search for something else to use. -- Cheers! Kevin. http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick
Hi Bob, Thank you for all the pointers. I may keep bothering you if I still have issues eh ? Something you said " I have done this once upon a time..." made me type this: aruna@debian:~/linux-4.1.2$ find . -name "*.c" | xargs grep -i " bob copeland " and guess what find says: ./block/partitions/karma.c: * Copyright (C) 2006 Bob Copeland ( me@bobcopeland.com) ./fs/omfs/file.c: * Copyright (C) 2005 Bob Copeland < me@bobcopeland.com> ./fs/omfs/dir.c: * Copyright (C) 2005 Bob Copeland < me@bobcopeland.com> ./fs/omfs/inode.c: * Copyright (C) 2006 Bob Copeland < me@bobcopeland.com> ./drivers/net/wireless/ath/ath5k/led.c: * Copyright (c) 2009 Bob Copeland < me@bobcopeland.com> ./drivers/net/wireless/ti/wl1251/sdio.c: * Copyright (C) 2009 Bob Copeland ( me@bobcopeland.com) ./drivers/usb/storage/karma.c: * (c) 2006 Bob Copeland < me@bobcopeland.com> Holy Wow ! file system, wireless, usb ? holy wow again.. ( proceeds to fall off my chair ..) Yeah 2005 was a long time ago I was still using Windows 3.1 and had no idea what Linux was :) Privileged to meet you Mr.Copeland ( So what is it with folks who's names begin with Bob ? All the Bob's I know are possibly related to Einstein ? Including our own Mr.Jonkman ) Seriously many thanks for taking the time to reply it IS much appreciated ! Aruna
On Fri, Sep 4, 2015 at 6:46 AM, Aruna Hewapathirane <aruna.hewapathirane@gmail.com> wrote:
Hi Bob,
Thank you for all the pointers. I may keep bothering you if I still have issues eh ?
Sure, feel free to CC me as well in case I seem unresponsive, sometimes I don't read the TLUG list so closely.
Holy Wow ! file system, wireless, usb ? holy wow again.. ( proceeds to fall off my chair ..)
As I found then, a great way to learn about the kernel is to write a file system :) I now do a small amount of ongoing development in linux wireless for work, particularly in mesh. It's good fun. -- Bob Copeland %% www.bobcopeland.com
participants (6)
-
Aruna Hewapathirane -
Bob Copeland -
Kevin Cozens -
Lennart Sorensen -
Mike -
Sergio Durigan Junior