Tool review: Microsoft Network Monitor 3.3

I have used Wireshark for packet sniffing and analysis for a number of years, starting back when it was called Ethereal. A little while ago I was using it to look at broadcast packets that our clients send out, and decided that it would be great if Wireshark could interpret our wire-level protocol and display meaningful information about the packets. After a bit of searching, I found that you can add plug-ins to Wireshark, allowing you to do whatever you want with the packet data. I found some detailed instructions on how to do this, beginning with:

• Install a version of the Microsoft C/C++ compiler
• Install a particular platform SDK
• Install Cygwin
• Install Python
• Install Subversion
• Get the Wireshark source
• Configure the source
• Build Wireshark

Once you're done all that, you can start looking at building your plug-in in C. I set up a Windows XP VM and spent a day or two doing all of this, but never got to the point of actually creating the plug-in. A few days later we had a team status meeting, during which I mentioned this project. A colleague, Peter, asked if I had looked at Microsoft NetMon, saying that he believed it allowed you to add your own parsers as well. I downloaded it and took a look. Thank you Peter, for saving me days, if not weeks of development time. In less time than it took me to set up the VM in preparation for writing a Wireshark protocol analyzer, I had analyzers written for the majority of both our UDP and our TCP protocols.

Writing parsers

As a packet sniffer, NetMon is not really much different from Wireshark, though I find the interface a little more intuitive. This might be because I'm running on Windows, and Wireshark has always looked to me like a Unix program that has been ported to Windows rather than an application written for Windows. They both support both capture and display filters. NetMon has colour filters as well – particular packets or conversations can be coloured based on the filter results. You can view packets as they are captured, save them to a file, and load them back in again later.

But writing a parser is orders of magnitude easier than writing a Wireshark plug-in. You simply tell it what ports your protocol uses and what the protocol looks like in a proprietary language (called NPL – Network Monitor Parser Language) that's vaguely C-like but very simple. Some properties of this language:

• it handles bitfields, ASCII and Unicode text, and binary data, as well as various types of numeric values (8, 16, 32, or 64 bits, integer or floating-point, signed or unsigned, big- or little-endian)
• you can define your own data types
• there are a number of special data types built-in; if your packet contains a 32-bit IP address, for example, you can just specify it as IPv4Address and it will get interpreted and displayed as expected
• you can make structs which group pieces of the data together, and arrays which hold collections of the same type of data
• you use while loops and switch statements to modify behaviour. For example, your protocol might have a byte that indicates the type of packet, and then the structure of the packet depends on the value of that byte. No problem.
• you can indicate both storage format and display format, so if you have a byte that's 0 for a request and 1 for a response, you can display the words "request" and "response" rather than just 0 or 1. The rest of the code can reference this value by name and get 0 or 1. The display string can be as complicated as you want, even referencing other pieces of the packet by name.
• it supports conversations, and there are variables that have global, conversation, packet, or local scope

The help file installed with the app describes each of the language features, and I found a document that describes an example protocol in great detail.

Drawbacks

The biggest drawback of this tool is the parser editor. It's not very powerful – it makes notepad look feature-rich. I use Ctrl-Backspace (delete previous word) and Ctrl-Del (delete next word) a lot, since it's supported in Windows Live Writer, Word, and emacs, but support is spotty – sometimes it works, sometimes it deletes the wrong word.

The main feature it's missing is undo. It doesn't even have a single-level undo. If you hit backspace one too many times, you'd better remember what that last character was because it's gone. An editor that doesn't support undo is pretty much unacceptable in this day and age, and I lost data more than once because of it. Once you realize that you can't undo mistakes, you end up clicking Save a lot more often, and do things like copy the file to a backup file before you make big changes. I checked my files into source control and started checking them in periodically, which is a good idea anyway, but if the parser stuff wasn't so damn cool, the lack of an undo feature might be a showstopper. Emacs supports Ctrl-A and Ctrl-E to get to the beginning and end of the current line respectively, and sometimes I instinctively use those keystrokes in editors where they're not supported, like this one. Unfortunately, Ctrl-A here means "select all", so doing that and then typing something is disastrous because there's no undo, so you just lost your entire file. You need to quit the file (do not save!) and then reload it, losing whatever changes you had made. Even a single-level undo would save you from that.

The compiler has some problems as well – there were a number of times where I got compilation errors that were badly written or vague enough that I didn't know what the problem was. It would point to what looked like a valid statement and say that it was unrecognized or invalid, and it turned out to be because of a missing (or extra) semi-colon on a different line, or a language rule that wasn't obvious.

Once you've made the changes to your parser, you have to save it and then click "Reload Parsers", which reloads all 370+ parser files it knows about. Surely there could be a way to just reload the one that I changed? Now, there are dependencies between files, so changing one file might require that a different file be reloaded, so reloading them all is the safest but it's slow. Ideally, the tool should be able to figure out the dependency tree and only reload files that depend on the ones changed. And the tool should prompt me to save if I have an unsaved file and I click "Reload Parsers".

If anyone from the NetMon dev team reads this, here's a bug report: If I load a parser file, then paste some code into the file, it's not marked as "dirty" until I actually type something. Also, if I load a display or capture filter from a file, this generally means "replace what's in the textbox with the contents of the file", not "insert the contents of the file into the textbox at the current cursor position". I can see how that feature might be useful in combining filters, but it should not be the default.

As powerful as the NPL language is, there are things it simply can't do. In my case, some of our packets can be encrypted or compressed, but the NPL language can't decrypt or decompress them. It would be nice to be able to write a small plug-in that could do these types of things, but it's not supported. The Wireshark approach would work for that.

Experts

For those analysis needs that are not satisfied by parsers, NetMon supports things called "experts", which are external programs that can read the data from a capture file and analyze it in whatever way it wants. It sounds similar to a parser except that it's written in C or C++ (or C#, I think) and has the limitation that it only works on saved files, so you can't look at the results in real-time as you can with a parser. I've stared to write one of these to solve the decompression/decryption problem I mentioned above. There doesn't seem to be a way to decrypt the data and write it out into a new capture file, but I can at least decrypt, parse, and display the data. I can reuse the parser code I've already written, since the program is dealing with pre-parsed information, but I have to grab each field individually and display it, so I essentially have to rewrite all the display code in C.

Summary

Overall, this is a very cool utility and has replaced Wireshark as my packet sniffer of choice. The documentation is pretty thorough and it includes some good examples. If all else fails, the parser code for all the other supported protocols is right there. There is a help forum to which I've posted a couple of questions and gotten quick and helpful responses. I wrote a while ago about how cool Windows Live Writer is, so kudos to Microsoft for yet another cool utility.