Why debugging LSP is different?

When debugging your own application you just press F5 and presto, you’re in business but in the world of LSPs it’s not that easy (other wise I wouldn’t write this article). The challenge with debugging LSPs is that each LSP is running at the space of the process which runs it which means that for each process that is using the network – the LSP will be loaded. For example if we have 30 processes which uses the network then we will have 30 loaded LSP, each in the context of the loading process.

How to debug the LSP

In order to debug the LSP we must attach to an application that loaded it but we shouldn’t attach to any application because when we usually debug the LSP it’s because of an error that occured for a specific application so we must attach to that application. For example if we want to debug the LSP inside Firefox we would attach to FireFox.exe

Personally I prefer to start an application in debugger, this allows me to follow all the steps in the LSP, and also I know that the debugger has all the symbols (sometimes when attaching you don’t get all the information).

Internet Explorer

The latest version of Internet Explorer ( version 8 ) is openning number of processes for one browsing window – A minimum of two, one is rendering and the other handles networking which means that you need to know which process to attach to, and if you start in a debugger, you get the rendering process and not the networking process.

To overcome this obsticle you need to tell IE to use a single process and not multiple processes, to do that you need to change the registry: Add the value TabProcGrowth to the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Main, the value needs to be a DWORD with the value 0. This change will allow you to easily debug Internet Explorer 8 LSP problems.

Debugging release LSPs

Release LSP has no symbols which means it’s much harder to debug then debug LSPs, there are two ways to go with this: Replicate – Find a way to replicate the problem and then use debug LSP to debug it. Logs – The LSP comes with a function which it writes to the debug of Windows, you can enable it for release and save it to a file (I prefer the %temp% directory, no complaints from users about unknown files, and you can write to it from any integrity level).

Common LSP bugs

These are the most common LSP bugs you can find:

• Memory corruption – Memory corruption will crash the process and in the crash information you will see your LSP as the cause
• Threading issue – Not handling data structures correctly (LSP is multithreaded in nature) will result in memory corruption do to race conditions, this mostly seen as a memory corruption or CRT asserts.
• Not intercepting the right method – This part is when the LSP doesn’t intercept what you wanted it to, this usually occurs because a failure to intercept a key method, bad handling of overlapped operation or failure to follow the guidelines for Windows Vista/7 LSP development.
• LSP not working – This can be because of incorrect LSP registration or existance of another LSP which causes a conflict.
• LSP registration fails – This can be because of using the default installer or the presence of a security software.

Final thoughts

Needless to say that most of these bugs are found when going from default Winsock LSP sample to a commercial product. Using any of Komodia’s products saves valuable time from the R&D team which means that they don’t have to develop, debug and QA their LSP and can focus on the product. Komodia is working on commercial product for three years now (from 2007) so we had our share of LSP debugging to stand behind this paragraph.