By Welcor and Jamdog, 2010
A common question asked by MUD owners is "My MUD keeps crashing, how do I use GDB to debug my MUD and find the problem". I hope to be able to answer that question with my quick guide below.
This is not a comprehensive tutorial that will teach you everything there is to know about GDB, but it will give you enough basic knowledge to hopefully root out the cause of any MUD crash.
What is GDB
GDB is a programmers debugging tool. It allows you to run the program within a controlled environment which will 'catch' a crash, and let you examine the status of the program at the time of the crash. It does much more than just this (like letting you pause your running program, and step through it one line at a time, or even examine your crashed program days after a crash, even when it wasn't running within GDB).
Running your MUD in Debug Mode
GDB should be run on your MUD from it's root folder (not in the bin subfolder as you might expect).
Next, type
gdb bin/circle to start GDB, and tell it to use your MUD executable. You will now be given the gdb prompt instead of your normal unix-shell one.
Next, type
run <port> to run your mud (obviously entering a port number to use, not typing "<port>"). You should see all your normal syslog info appear on your screen, and your MUD will eventually give a message saying "No connections, going to sleep.".
You should then be able to connect to your MUD on the specified port in the normal way. Your next step is to crash the MUD. Obviously, if you have a crash bug, you might have some idea of what's causing it. When you manage this, the MUD will appear to pause, and you will get a message in GDB, and your gdb prompt will return. You can now start debugging.
Normally, the first command you type will be
bt (for 'backtrace') which will show you the function tree, with the lowest level at the top. This shows the files each function call is in, and the line number it's called from. Below is an example of how this could look:
Program received signal SIGSEGV, Segmentation fault.
load_char (name=0x22c540 "Jamdog", ch=0x10ebcb48) at players.c:248
248 GET_SKILL(ch, i) = 0;
(gdb) bt
#0 load_char (name=0x22c540 "Jamdog", ch=0x10ebcb48) at players.c:248
#1 0x0045f967 in nanny (d=0x10eb62f0, arg=0x22ca50 "Jamdog") at interpreter.c:1417
#2 0x00440f80 in game_loop (mother_desc=3) at comm.c:903
#3 0x004410e0 in init_game (port=4000) at comm.c:531
#4 0x004414f6 in main (argc=1, argv=0x10011c80) at comm.c:378
Now, this may look scary and confusing, but it's relatively simple. The top line (beginning #0) is the point of the crash - it's the line that was being executed when the crash happened. Here is the breakdown of that line:
#0 - indicates the function depth. As you can see above, #0 is called from within the function at #1, and #1 is called from within the function at #2. The 'main' function will always be the last in the list.
load_char - This is the name of the function where the crash happened. Sometimes it is a function in the C core code, and not the MUD code, so look for the first function you recognize in the list.
(name=0x22c540 "Jamdog", ch=0x10ebcb48) - This shows the variables that were passed to the function. In this case, a string at memory address 0x22c540 and a char_data struct at memory address 0x10ebcb48. GDB handily shows the string for you, which was 'Jamdog'. You can see the contents of ch with the print command (see below)
at players.c:248 - This shows the file and line number where the crash happened.
You can now use the following commands to debug most problems:
info local - display the status of 'local' variables, variables created in the current function.
list - list the source code. Shows 3 or 4 lines above and below the current location.
print - display the staus of the specified variable (does not need to be local, and can view within structs this way - e.g. print ch->name or print *ch).
up - move 'up' the function tree (shown by bt) to the previous function.
kill - stop the current MUD program.
quit - exit from gdb (prompts to kill a running program)
Here is an example of these in use:
info local
id = 0
i = 2
fl = (FILE *) 0x10011f74
fname = "plrfiles/F-J/jamdog.plr"
buf = "\204b\026aPÄ\"\000\000\000\000\000ÿ\001", '\0' <repeats 14 times>, "\001"
buf2 = "X\006\000\000ðÃ\"\000\001\000\000\000$Ä\"\000(Ä\"", '\0' <repeats 17 times>, "PE\02"
line = "ðé$\000í"...
tag = "þÿ\000\000\2268"
ch = (struct char_data *) 0x10ebcb48
(gdb) print *ch
$1 = {pfilepos = 50, nr = 65535, in_room = 791, was_in_room = 65535, wait = 1, player = {passwd = "Ja3pvo1SmcUB6", name = 0x8b117a0 "Jamdog", short_descr = 0x0, long_descr = 0x0, description = 0x0, bio = 0x0, title = 0x8b16308 "the Omnipresence", sex = 1 '\001', chclass = 4 '\004', level = 200, adm_level = 4, time = { birth = 1243951538, logon = 1288030404, played = 2148763}, weight = 146 '\222', height = 174 } }
(gdb) print ch->in_room
$2 = 791
(gdb) list
243 /* character initializations */
244 /* initializations necessary to keep some things straight */
245 ch->affected = NULL;
246 mudlog(NRM, LVL_GOD, TRUE, "Setting skills to zero");
247 for (i = 1; i <= MAX_SKILLS; i++)
248 GET_SKILL(ch, i) = 0;
249 mudlog(NRM, LVL_GOD, TRUE, "Setting sex");
250 GET_SEX(ch) = PFDEF_SEX;
251 GET_CLASS(ch) = PFDEF_CLASS;
252 GET_LEVEL(ch) = PFDEF_LEVEL;
(gdb) up
#1 0x0045f967 in nanny (d=0x10eb62f0, arg=0x22ca50 "Jamdog")
at interpreter.c:1417
1417 if ((player_i = load_char(tmp_name, d->character)) > -1) {
(gdb)
Running GDB on your MUD After a Crash
Obviously, you can't keep your MUD in debug mode all the time. If it crashed while you were asleep, it would 'pause' the MUD until you found it in the morning.
For that reason, modern tbaMUD releases do a 'core dump' in the event of a crash. This is basically a snapshot, or photograph of the MUD at crash time, and this can still be used by GDB to trace the problem.
Core dump files are saved in your lib folder and should be checked regularly. Because core dumps are commonly 10-15Mb you should delete them once you are done troubleshooting.
To load the core dump into GDB, you need to be in the root folder of your MUD, and type:
gdb bin/circle lib/core.# GDB will load in the normal way, but you don't need to run the MUD. You can immediately type
bt to see the backtrace from crash-time, and you can use all the above commands to debug what happened. Because the MUD isn't running however, you won't be able to step-through the program (which usually isn't necessary anyway, and why I didn't add it to this guide).
Here is another typical debugging session. This is what you need to post to the forums when you need help:
gdb bin/circle
<gdb output>
gdb> run
<mud log - it's usually helpful to include the last couple of lines.>
Program received SIGSEV in some_function(someparamaters) somefile.c:1223
gdb> bt
#0 0x234235 some_function(someparamaters) somefile.c:1223
#1 0x343353 foo(bar) foo.c:123
#2 0x12495b baz(0x0000000) baz.c:3
gdb> list
1219
1220 foobar = something_interesting();
1221
1222 foobar = NULL;
1223 free(foobar);
1224 }
1225
1226
1227 next_function_in_somefile_c(void)
gdb> info local
foobar = NULL
gdb> up
frame #1 0x343353 foo(bar) foo.c:123
gdb> list
<similar output to the above, but different file>
gdb> info local
<similar output to the above, but different variables>
gdb> up
frame #2 0x12495b baz(0x0000000) baz.c:3
gdb> list
<similar output to the above, but different file>
gdb> info local
<similar output to the above, but different variables>
gdb> up
You are already at the top frame.
