Alter.Org.UA
 << Back Home UK uk   Donate Donate

WinDbg basics
HOWTO for QA engineers

Index

Abstract

There is such debugger - WinDbg. It is developed by Microsoft and can be downloaded from this location:
http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx.

In general there are 2 possible ways of debugging: with single machine and with 2 machines. When we have single machine we can store debug messages (logs) to file and if crash happens analyze automatically created memory dump (crashdump). After reboot, of course. When we use 2 machines, one of them (host) monitors another (target). The plus of two-machines way is that memory dump and latest logs can always be obtained against single machine method.

Setup

Common settings, Target machine
  1. install latest version of DbgPrintLog with following command line: 
    update.bat
DbgPrintLog.exe --drv:inst B --drv:opt DoNotPassMessagesDown 1 --drv:opt BufferSize 16384
    This will allow to extract debug messages preceding system crash from memory dump and significantly reduce traffic over communication channel if dual machine debugging is used. If you are going to debug boot-driver (e.g. for disk controller), use the following command line switch:
    --drv:inst 1
  2. Setup crashdump:
    • Make sure that pagefile (virtual memory) is set to be greater than physical memory and is located on the same volume where running copy Windows is installed.
    • Make sure if there is enough disk space on the disk on the volume where running copy of Windows is installed. Memory dump includes snapshot of entire physical memory and some additional information.
    • My Computer -> Properties -> Advanced -> Startup and recovery
    • Enable memory dump
    • set dump type to Kernel Dump or Full Memory Dump.
      Attention: do not set Minidump/Small dump option, since it is absolutely useless for saving logs.
  3. Setup verifier. It is Windows built-in driver validation tool, helps to detect problems early.
    • run verifier.exe. Since WinXP it is built-in, under 2000 it comes with DDK.
    • chose "Custom settings" - second option. Next
    • chose "Select from list" - bottom radio-button. Next
    • check everything, probably except "Low Resources Simulation". Next
      Attention: "Low Resources Simulation" - is intended to check driver's error handling in awfull stress conditions. It just shall not to crash. Never use this option for testing in normal operation mode.
    • chose "Select Driver from list" - last option. Next
    • check required driver(s). If target driver is not loaded at this moment, use "Add not loaded drivers to list" button under list. Drivers are usually located in WINDOWS\System32\drivers\ directory.
    • OK, do not reboot now
  4. reboot machine to make sure all settings are saved to disk before installing application and/or driver to be debugged.

When single machine debugging is used, no more preparation is required. You can continue reading from Debugging section.

Target machine settings for remote debugging (dual machine)
NT3.51/NT4/2000/XP/2003
  1. connect machines with IEEE1394 (FireWire) or NULL-modem cable. It is recommended to use cable with full or partial handshaking.
    Note: IEEE1394 is supported since Windows XP only. Windows 95/98/ME, Windows NT4 and Windows 2000 cannot be used for remote debugging over IEEE1394.
  2. do not forget to make common settings (see above)
  3. open with some editor BOOT.INI file in the root directory of your boot volume.
  4. find there record corresponding to your OS, smth. like this: 
       multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional"
  5. make a copy of this record in the next line
  6. add to the end of copied string the following:
    • if IEEE-1394 cable is used: 
             /debug /debugport=1394 /CHANNEL=3
    • if NULL-modem cable connected to COM2 is used: 
             /debug /debugport=COM2 /baudrate=115200
    final string shall look like this (without line break): 
    multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP Pro" /noexecute=optin /fastdetect /debug
    /debugport=1394 /CHANNEL=3
  7. make sure that TIMEOUT value in BOOT.INI has non-zero value
  8. now BOOT.INI shall looks like this: 
    [Boot Loader]
Timeout=5
Default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[Operating Systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP Pro" 
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP Pro DBG" /noexecute=optin /fastdetect /debug
    /debugport=1394 /CHANNEL=3
  9. disable FireWire (IEEE1394) card in Device Manager:
    My Computer -> Manage -> Device Manager
Target Vista machine settings for remote debugging (dual machine)
Vista
  1. connect machines with IEEE1394 (FireWire) or NULL-modem cable.
    Note: IEEE1394 is supported since Windows XP only. Windows 95/98/ME, Windows NT4 and Windows 2000 cannot be used for remote debugging over IEEE1394.
  2. do not forget to make common settings (see above)
  3. run cmd.exe with administratice rights.
    Note: It is not the same sa run it under administrative account. This is special action available via context menu.
  4. execute the following comamnds:
  5. if IEEE-1394 cable is used: 
       bcdedit /dbgsettings 1394 CHANNEL:3
  6. if NULL-modem cable connected to COM2 is used: 
       bcdedit /dbgsettings serial debugport:2 baudrate:115200
  7. if USB-link is used with session name 'debugging': 
       bcdedit /dbgsettings USB targetname:debugging
  8. enable debugging: 
       bcdedit /debug on
Target machine settings for remote debugging (dual machine)
on system setup stage or in
Windows PE environment
  1. connect machines with IEEE1394 (FireWire) or NULL-modem cable.
    Note: IEEE1394 is supported since Windows XP only.
  2. find and open with some editor txtsetup.sif file
  3. find there: 
    OsLoadOptions ="/fastdetect /minint"
  4. and replace it with 
    OsLoadOptions ="/fastdetect /minint /debug /debugport=COM2 /baudrate=115200"
    or 
    OsLoadOptions ="/fastdetect /minint /debug  /debugport=1394 /CHANNEL=3"
Host machine
  1. do not forget to connect machines with cable!
  2. install WinDbg
  3. put dbgprn.dll from latest version of DbgPrintLog to 'dbgext' subdirectory of WinDbg install directory. (e.g. C:\Program Files\WinDbg\dbgext)
  4. Define environment variable _NT_SYMBOL_PATH, pointing to local path to debug symbol files (uually this is 'sym' subdirectory of WinDbg install directory (e.g. C:\Program Files\WinDbg\sym) and URL of internet symbol server (e.g. http://msdl.microsoft.com/download/symbols). Short filenames are recommended (DOS, 8.3). 
    _NT_SYMBOL_PATH=srv*C:\Progra~1\WinDbg\sym*http://msdl.microsoft.com/download/symbols
  5. Download debug symbol pack from Microsoft and place it to the same directory (pointed by _NT_SYMBOL_PATH, see above). It worth setting NTFS Compressed attribute on this diectory. You may use symchk.exe from WinDbg package to load sumbols for specified executables individually.
  6. Run WinDbg
  7. follow menu items: File -> Kernel Debug ->
    • 1394, chose channel 3, OK, Answer 'Save' if prompted for workspace saving. If you run WinDbg for the 1st time, error message "Unable to open file" may appear. This means, that system didn't detect remote FireWire port on target machine yet. In this case do the following:
      • keep WinDbg open
      • reboot target machine and chose OS item marked with [Debug] or [Debugger Enabled] (or smth. like this) in boot menu
      • host machine shall display New Hardware Found message and install drivers for Debug Port. You may be prompted for Windows installation disk.
      • in some cases you may need to reboot host machine and repeat procedure once again.
      • if properly connected and configured as described above machines still cannot establish debug connection, I don't know what else can help. May be try to replace your FireWire PCI cards with different ones, I saw some cards, those seemed to be good, but didn't work properly on data transfers.
    • COM, just tell to which COM-port of host machine NULL-modem cable is connected and set speed to 115200.

Debugging

Single machine (local) debugging
  1. run DbgPrintLog for capturing logs: 
    DbgPrintLog.exe --full --cpu -l 10
    This will start console, where help message and current log file name are displayed. Do not close it. Logs will be stored in files DbgPrintXXX.log in same directory. XXX in filename is a sequential number of file. The very first file has name DbgPrint.log. Previously created files are not overwritten and numbering is continued. -l 10 instructs logger to keep only 10 latest log files.
  2. save debug info files .PDB from the build you are about to test to some separate location. If problem happens only with Release build, please configure your project to generate debug info for your binaries. If debug info is generated, .PDB files shall appear near compiled .EXE, .SYS, .DLL. When tuning Debug Info options, do not chose 'Separate Types' option.
  3. mark saved .PDBs somehow. If system crashes, .PDBs generated for the binaries were installed would be required. Ideally, after each build binaries and their .PDBs should be stored together.
  4. run dsync.exe from freboot package to synchronize system cache with on-disk data. This improves File System consistency if crash happens. You can download it from the following address:
    http://alter.org.ua/soft/win/fastreboot
  5. now install your drivers, services and application.
  6. make your tests until something goes wrong:
    • Unexpected behavior
      1. switch to logger console and press 'Esc' to stop logging
      2. pack *.LOG files from the folder where DbgPrintLog.exe was run to archive.
      3. also, it would be nice to put this archive together with binaries and .PDBs used in given test session.
    • System crash, BSOD
      1. memory dump shall be created in the directory, specified during preparation. Usually it is created in WINDOWS directory and is named MEMORY.DMP.
      2. after reboot put copy of MEMORY.DMP together with binaries and .PDBs for possible future use.
        Note: verify creation time of this file. It may happen so, that you see dump of previous crash. In this case this file and further steps are useless.
      3. run WinDbg
      4. open MEMORY.DMP:
        File -> Open Crash Dump
      5. in the command prompt of WinDbg execute the following commands: 
        !analyze -v
!dbgprn.save -a -e --full --cpu -T R --file <full path to file>
        if !dbgprn.save says "Cannot find DbgPrnHk!DbgPrnHk_State export", try the following command: 
        !dbgprn.lsig
!dbgprn.save -a -e --full --cpu -T R --file <full path to file>
      6. Log will be saved to file <full path to file>. Pack obtained log-file to archive.
      7. also, it would be nice to put this archive together with binaries and .PDBs used in given test session.
      8. select entire text from log window of WinDbg and copy-paste it to some file, place this file together with copy of memory dump, archived logs and snapshot of binaries and .PDBs
    • System gets frozen
      1. In most cases this is actually BSOD. But system cannot generate memory dump for some reason. Usually, if you switch to dusl-machine debugging, you will see this BSOD in WinDbg and can make memory dump. Set it up :)
  7. send archive(s) with logs to developer and tell what did you do before problem appeared.
Remote debugging (two machines)
  1. do not forget to connect machines with cable!
  2. Run WinDbg
  3. follow menu items: File -> Kernel Debug ->
    • 1394 chose channel 3, OK, Answer 'Save' if prompted for workspace saving.
    • COM just tell to which COM-port of host machine NULL-modem cable is connected and set speed to 115200.
  4. reboot target machine and chose OS item marked with [Debug] or [Debugger Enabled] (or smth. like this) in boot menu.
  5. WinDbg on host machine shall display information about establishing debug connection and print some information about target system.
  6. when target system is up, run there DbgPrintLog.exe for capturing logs: 
    start DbgPrintLog.exe --full --cpu -l 10
    This will start console, where help message and current log file name are displayed. Do not close it. Logs will be stored in files DbgPrintXXX.log in same directory. XXX in filename is a sequential number of file. The very first file has name DbgPrint.log. Previously created files are not overwritten and numbering is continued. -l 10 instructs logger to keep only 10 latest log files.
  7. save debug info files .PDB from the build you are about to test to some separate location. If problem happens only with Release build, please configure your project to generate debug info for your binaries. If debug info is generated, .PDB files shall appear near compiled .EXE, .SYS, .DLL. When tuning Debug Info options, do not chose 'Separate Types' option.
  8. mark saved .PDBs somehow. If system crashes, .PDBs generated for the binaries were installed would be required. Ideally, after each build binaries and their .PDBs should be stored together.
  9. run dsync.exe from freboot package to synchronize system cache with on-disk data. This improves File System consistency if crash happens. You can download it from the following address:
    http://alter.org.ua/soft/win/fastreboot
  10. now install your drivers, services and application.
  11. make your tests until something goes wrong:
    • Unexpected behavior
      1. switch to logger console and press 'Esc' to stop logging
      2. pack *.LOG files from the folder where DbgPrintLog.exe was run to archive.
      3. also, it would be nice to put this archive together with binaries and .PDBs used in given test session.
    • System crash, BSOD
      1. message about system stop shall appear in WinDbg window on host machine. Target machine will look frozen at this moment.
      2. in the command prompt on WinDbg execute the following commands: 
        .dump /f <DumpFileName>
!analyze -v
!dbgprn.save -a --full --cpu -T R --file <full path to file>
        if !dbgprn.save says "Cannot find DbgPrnHk!DbgPrnHk_State export", try the following command: 
        !dbgprn.lsig
!dbgprn.save -a --full --cpu -T R --file <full path to file>
      3. Log will be saved to file <full path to file>. Memory dump shall be stored in <DumpFileName> file. Pack obtained files to archive.
      4. also, it would be nice to put this archive together with binaries and .PDBs used in given test session.
      5. select entire text from log window of WinDbg and copy-paste it to some file, place this file together with copy of memory dump, archived logs and snapshot of binaries and .PDBs
      6. if you see magic word "KeBugCheckEx" in WinDbg output - reboot target machine with power or reset button, since it is unusable in curent state. But if you see "first chance exception" - you can try continue execution with the following WinDbg command: 
        g
  12. send archive(s) with logs to developer and tell what did you do before problem appeared.

See also:

FB or mail alterX@alter.org.ua (remove X)   Share
<< Back designed by Alter aka Alexander A. Telyatnikov powered by Apache+PHP under FBSD © 2002-2024