Building Fiddler Importers

Learn how to build your own Fiddler importers with this simple guide.

When reproducing a bug on a website or web service, traffic captures are invaluable because they allow developers and testers to easily see exactly what’s going on at the network level, even without having access to the reproduction environment. Over the years, Fiddler’s Session Archive Zip (SAZ) file format has become the gold standard for such captures because SAZ files are easily captured (with Fiddler, Fiddler Everywhere, FiddlerCap, or a Fiddlercore-based tool) and are easily reviewed with the powerful Fiddler desktop client.

However, in some cases, you may wish to debug network traffic that was originally captured using another tool and exported in a format like a Wireshark PCAP or the browser-standard HTTP Archive Format (HAR). Fiddler already includes importers for those formats, but what if you want to support a different format that Fiddler doesn’t yet support?

Fortunately, Fiddler is extremely extensible, and this extends to the import/export system. We’ve previously published an example exporter, and today I’d like to talk about how you can build an importer.

I’m now an engineer on the Microsoft Edge browser team, and our preview builds are now being tested by users around the world. When those users encounter problems, sometimes the best path for debugging starts with capturing a network traffic log. For some of our more technical users, that might involve collecting a SAZ file using one of the existing Fiddler-* products. But for other users (especially our upcoming MacOS users), collecting a SAZ file involves more overhead. In obscure cases, collecting a SAZ file might cause a repro to disappear.

Fortunately, the new Microsoft Edge browser is built atop the Chromium open-source browser engine, and Chromium includes a built-in network logging feature. To collect a network log in Edge, simply navigate a tab to edge://net-export (in Google Chrome, you’d visit chrome://net-export instead).

Configure the options using the radio buttons and click Start Logging to Disk. Open a new tab and reproduce the problem. After you’ve completed the repro, click Stop Logging. At the end, you’ll have a NetLog .json file containing all of the network events. The output JSON file can be viewed using a web-based viewer:

However, in many cases, it would be more desirable to view the file using Fiddler. Unlike the web based viewer (which tends to show content as a base64-encoded string of goo), Fiddler includes built-in formatters for common web formats (e.g. images, JSON, etc), and it makes it easier to export response bodies as files, resend captured requests using the Composer, and replay captured responses using the AutoResponder.

Fiddler does not include a NetLog importer by default, so let’s build one.

First, we start by looking at the Chromium documentation for the relatively straightforward file format. NetLog files are JSON-serialized streams of network events. The low-level events (tracking DNS lookups, TCP/IP connections, HTTPS certificate verifications, etc) are associated to a request identifier, which represents one or more network requests (what Fiddler calls a “Web Session”). So, the task of our importer is to:

1. Parse the JSON file into a list of events
2. Bucket the events by the request identifiers
3. Generate one or more Fiddler Web Sessions from each bucket

The C# sourcecode for the FiddlerImportNetlog extension is available on Github. Beyond all of the typical project files, the extension itself consists of just two files, Importer.cs, FiddlerInterface.cs, and Properties\AssemblyInfo.cs.

The AssemblyInfo.cs contains just one line of interest:

Fiddler requires this attribute to load the assembly as an extension; it specifies the minimal version of Fiddler you’ve tested the extension to run in.

The FiddlerInterface.cs file contains a simple class which implements the ISessionImporter interface. The ProfferFormat attribute on the class specifies the format type and a descriptive string to show in the import dialog:

The importer interface exposes a single method, ImportSessions, which accepts a string specifying the selected format, a dictionary of options, and an optional event handler to call back with import progress events. The function returns an array of Sessions created from the imported data.

The bulk of the extension’s logic is found in Importer.cs. The caller provides a StreamReader from which the imported file’s text is read and then parsed using the JSON.JsonDecode method made available by the using Fiddler.WebFormats statement at the top of the file. WebFormats.JSON is a fast, simple parser that reads the JSON into Hashtables (for JS objects), ArrayLists (for JS arrays), and primitive types (booleans, doubles, strings, etc).

After parsing, the importer looks up the mappings from named event types (e.g. URL_REQUEST_START_JOB) to the integer identifiers recorded inside the event entries, then parses the list of event entries, bucketizing those with a source of URL_REQUEST by the request id. Then, the ParseSessionsFromBucket method loops over each URL_REQUEST’s events to collect the data (e.g. request headers, response headers, response bodies) needed to generate a Fiddler Web Session. Two notable details:

1. The NetLog format may have multiple request/response pairs in a single URL_REQUEST (e.g. if the request resulted in a redirect or an automatic authentication challenge/response)
2. The NetLog format does not presently store (anywhere) the request body content for POST requests. In contrast, response body bytes may or may not be present based on the options chosen by the user when starting the logging process

After generating the list of extracted Web Sessions, the importer generates a few “mock” Sessions that allow the user to peruse the raw data captured in the file (e.g. the browser configuration, list of enabled extensions, etc).

After an hour of quick-and-dirty coding, Fiddler can now import NetLog json files:

If you’re just interested in using this importer without building it yourself, you can install it from my website. If you find any bugs, please file them!

Hopefully, you’ve found this to be a useful introduction to how easy it is to build importers to get your data into Fiddler!