This page covers all challenges relating development of network interception product, some are shared across redirection technologies, some are specific to them.

LSP

Bugs in LSP sample

The default Microsoft sample everyone is using (including Komodia) and has built in bugs that must be fixed (just to clarify, Komodia had fixed all the bugs mentioned here):

• Doesn’t work with Safari 5.1 and iTunes 10.5
• World of Warcraft sometimes hangs on startup.
• Firefox 6.0 and above may hang on high volume traffic.
• LSP will not intercept FF 4.0 and above on Win7.
• Will hang IPSec on Windows XP/2003.
• General bugs with overlapped operations.

Other challenges with LSP development

Also when developing your LSP you need to take care of the following issues:

• Problem with apps that use SetFileCompletionNotificationModes on Windows Vista and above, causing problems connecting to MSSQL, you can read more about it here.
• Support overlap operations for browsers like Chrome and other applications
• Know how to handle Admin and UAC privileges under Vista and above
• Know how to communicate with your LSP, some methods will not work across all OS or 32/64bit difference
• Remote debugging, once you do have a bug, how can you remotely know why, you need to know what to log in order to resolve it quickly

If you decide to develop your own solution we recommend that you start with supporting Chrome, Safari and FF on Windows 7, because they will pose most problems. If you start on IE and after three months comes with a solution and then later struggle to make it work for the other browsers, you just wasted three months.

LSP conflicts and OS compatibility

There are LSP conflicts that will occur with other LSPs, so you need to make sure your installer knows how to deal with 3rd party LSP and put your LSP in the right place, some LSPs will require you to be first in the chain, some last, you can only know that after investigating those LSPs manualy, keep in mind that the default installer doesn’t support too much out of the box, you will need to add at least 5000 lines of code to make it support all the scenarios you want (layering before or after a provider, doing a provider bypass, layering over base providers only, supporting Windows in various languages)

Once you are done, you need to make sure your LSP works with Anti Viruses, you need to check 40 brands against 5 browsers across 20 flavors of Windows.

Komodia’s LSP solution

Our Komodia’s Redirector already have all of these solved, and each month we get at least one item new to add because of the speed which the software world change, overall it took us one and a half year to get something stable that works also outside of the VM.

NDIS

NDIS is a kernel driver and works on packets level, therefore most of the work must be done on packet parsing, and you will need to code these features:

• Reconstruct the TCP/IP stream, remove duplicate packets and acks.
• If you plan to modify data, you need to adjust ID and seq numbers.
• Make sure your driver supports various VPNs.
• Make sure your driver supports 3G networks.
• Doesn’t work with localhost.

Komodia’s solution

In Komodia’s Redirector we chose not to used NDIS mainly because of the limitation of LocalHost and VPN/3G, some products do use it for workstation but we believe it’s more of a server technology.

WFP

WFP can work either at packet level or stream level, which one to choose is up to your implementation, the challenges with WFP are:

• Some functionality will not work on Vista (depending on the SP)
• You need to install it correctly so it will load on startup
• It’s quite new, so it’s hard to find information about it
• You need different code for Windows Vista, Windows 7, Windows 8
• WFP on Windows 8 requires to implement a special chain of redirection so you’d be able to work with other WFP drivers

Komodia’s solution

In Komodia’s Redirector we have an option to use WFP, it took us three months to develop it on top of our pre-existing solution and another three months to make it Windows 8 ready.

DLL injection

DLL injection is used when you don’t want to require the user to have admin privileges, the challenges with DLL injection are:

• Which hooking library to use, some are very high prices (like MS detours, remember it’s free only for free apps, and doesn’t come with the 64bit version for the free version), some are free but are identified as a Virus by most majors AV providers.
• Some games will detect DLL injection as an attempt to hack, and will result in banning the players account, make sure your app knows how to handle it.
• You need to make sure you know how to handle 32bit and 64bit processes the hooking is different, also it’s very hard to do an injection from a 64bit service to a 32bit process.
• You can inject into Windows Metro, but can’t redirect Winsock traffic to localhost.

Komodia’s solution

In Komodia’s Redirector we use a LSP, we believe this is the way to go. Like previously said, the only situation where DLL injection is needed is that if you don’t want to require admin privileges.

HTTP Parsing

After you sorted out the redirection component you need to make sure you can read the HTTP traffic correctly, while parsing the data you need to make sure you can handle the following:

• Some servers doesn’t use \r\n for line termination.
• Know how to decode gzip.
• Know how to decode inflate and handle differences between web servers.
• Know how to decode SDCH encoding.
• Know how to parse chunked transfer.
• Know how to handle HTTP 100 continue.
• Know how to redirect a web page correctly.
• Know how to handle variations across browsers.
• Know how to handle POST correctly.
• Know how to aggregate some content while not intercepting other based on rules.

Komodia’s solution

In Komodia’s Redirector the component development and QA in the field took about six months, it’s a very complex component, but for our clients the API is very simple.

SSL decoding

If you want to inspect SSL traffic you need to know to decrypt SSL, the method is very complex to implement, we wrote a short article on how: SSL Decoding works.

Challenges when implementing this method are:

• Know how to install CA cert for Windows store.
• Know how to install CA cert for Firefox store.
• Know how to install Ca cert for Opera (there’s no API, you have to reverse engineer their store structure).
• Know how to clone certificates correctly.
• Know how to detect if a certificate is valid or not.
• SSL has many features, some sites require that you enable or disable SSL features in order to work, some sites will not accept TLS (modern SSL) and you will have to fall back to SSLv3.
• On Windows 7 know to how to download root/intermediate certificates in real time for validation.

Komodia’s solution

In Komodia’s Redirector it’s the most complex component, overall with all the field QA and bugfixes it took one year to make the component stable as it is know.

Intercepting DNS

There are number of ways to do it, and it depends on your needs, basically there are a few ways to do it, the challenges are:

• Knowing which method is best for your specific scenario.
• Once you know which scenario you want, you need to choose the best technology for the job, a wrong choice may mean starting over, for example it’s important to decide up front which OS you want to support, not all technologies support all OSs, and connection types.

Komodia’s solution

In Komodia’s Redirector we have a DNS hijacking module, it is used mostly for anonymizing and it is using hooks to control the app DNS methods, you can read about how it works.

Field QA

What’s field QA, it doesn’t matter how much tests you’ll do in house, in the field you will have strange variations you couldn’t possibly think of, for example a system with: firewall, corporate firewall, AV solution and a proxy (just an example), you can’t cover all bases that’s why QA takes more then development because all of these field QA which is more common then you may thing.