Detecting & Defeating the Debuggers

Debuggers are the main tool used in reverse engineering. It is used by serial crackers to break the software protection or to uncover the algorithm used in the proprietary applications. On the other hand it is also used by researchers to analyze the malwares.

Detecting the presence of debuggers is an important step in this direction. Here I will discuss about both user land and kernel level debugger detection techniques. Also I will throw some light on how one can defeat these techniques. Its always good to know both sides of the coin even though you always sit on one side.

In user land

Detecting debuggers in user land (ring 3) is simple. Windows provides API IsDebuggerPresent() which indicates if the application is being debugged. In such a case application may decide to terminate or may take different path just to evade the crackers.

There is a better method than one mentioned above. This involves directly reading ‘beingDebugged’ flag of PEB of the process. It is more stealthier than directly using the function since the function entry is clearly visible in the import table. In fact the IsDebuggerPresent() function internally does the same thing of reading the flag from PEB.

Here is the disassembly of IsDebuggerPresent Function

mov eax, dword ptr fs:[18]
mov eax, dword ptr ds:[eax+30] ; eax now points to PEB
movzx eax, byte ptr ds:[eax+2] ; retrieves PEB->beingDebugged value

Bypassing the above detection is simple as well.You can just attach debugger and modify the return value of IsDebuggerPresent(). You can also directly modify the ‘beingDebugged’ value in PEB. OllyDbg has several plugins which does this automatically.

This technique of detecting debuggers is pretty old, but it still helps in evading casual crackers. Now there are most customized methods specific to debuggers such as OllyDbg, IDAPro, Softice etc.

You can find some very good techniques at OpenRCE.

Inside the Kernel

There are very less resouces available online when it comes to kernel as very few people have dared to enter ring 0. However windows provides support for detecting and defeating the debuggers inside kernel. You can use exported variable KdDebuggerEnabled of ntoskrnl to detect if the machine is being debugged by kernel debugger. The good place to perform this check in the DriverEntry routine of your driver.

Once the debugger is detected, you can either terminate execution of your driver or disable the debugger itself. To stop the debugger, you can use another exported function KdDisableDebugger on NT based machines.

This same trick is used by IceSword (anti rootkit tool) to prevent reversers from knowing its internals.Here is the code snippet from IceSword driver Isdrv120.sys which does this check and then disables the debugger.

However inside the ring 0 also its not rare to find debugger specific checks. For example, you can test for the presence of SoftIce by checking if its driver is loaded or not.

– Nagareshwar

.