Earlier today, Google blogged that their Chrome browser had detected a site presenting an unauthorized (fake) certificate for Google.com’s HTTPS site. The certificate had been generated by an intermediate Certificate Authority certificate that was inadvertently issued by a trusted Certificate Authority. Google blocked the Intermediate Certificate and notified other browser makers who promptly blocked the certificate as well.

Chrome was able to detect the forgery because that browser includes a technology called Certificate Pinning, whereby the client HTTPS stack has a fixed mapping of hostnames to certificates. If the client sees an unexpected certificate during the HTTPS handshake, it knows that either an intermediary is decrypting the traffic (a la Fiddler) or a Certificate Authority was fooled into supplying an unauthorized certificate. Some Twitter clients, the iOS AppStore app, and a small number of other clients use a similar technique to reject unrecognized certificates.

Unfortunately, the pinning feature has shortcomings—it requires that the client maintain and update the mapping of hostnames to certificates, an expensive proposition which eliminates a significant portion of the flexibility and value of using HTTPS’ PKI-based system.

Various proposals have been announced to address the threat of bogus certificates; for instance, Certificate Transparency proposes that all Certificate Authorities must publish a log of every certificate that they issue, and those logs are audited by site owners to ensure that no bogus certificates have been issued for their sites. However, any comprehensive solution is likely years away.

Introducing Convergence

In this post, I’ll demonstrate a proposal called Convergence, which attempts to detect targeted attacks against HTTPS traffic—for instance, from within a specific locale or country.

Here’s how Convergence works: when the client connects to a HTTPS site, it obtains its certificate. It then consults a set of servers around the world, called notaries, and checks to see whether the site in question is sending the same certificate to all of those servers as well. If so, then there’s good reason to believe that the certificate received by the client is the one intended by the server.

Convergence is an interesting idea, and a Firefox plugin is available that implements the protocol. However, this plugin only works in Firefox. What if you wanted to try out this technology in another browser—or even in a non-browser HTTPS client? You’d need to be able to inject your code into the certificate validation logic in every client application, a herculean task which would prove quite complicated.

Validating Certificates in Fiddler

Here’s where Fiddler comes into play. As a local proxy which establishes HTTPS connections on client applications’ behalf, Fiddler is in a position to evaluate every certificate used to secure every HTTPS connection made by your computer. As such, Fiddler exposes a great vantage point from which notary validation checks can be made. If the notaries affirm that the server-presented certificate is valid, the connection proceeds as normal. On the other hand, if the notaries indicate that the certificate doesn’t match what they see, Fiddler can securely abort the handshake and warn the user.

Fiddler and FiddlerCore already expose a simple event which can be used to evaluate HTTPS servers’ certificates. From your Fiddler extension or FiddlerCore-hosting application’s code, simply attach an event handling delegate:

FiddlerApplication.OnValidateServerCertificate += CheckCert;

The event handler is provided information about the server’s certificate using the properties of the ValidateServerCertificateEventArgs argument:

The handler can then control whether the certificate is deemed valid using the ValidityState property, which can be set to any of the values in the CertificateValidity enumeration:

/// <summary>

/// Enumeration of possible responses specified by the ValidateServerCertificateEventArgs as modified by FiddlerApplication's <see cref="FiddlerApplication.OnValidateServerCertificate">OnValidateServerCertificate event</see>  

/// </summary>

public enum CertificateValidity

{

 /// <summary>

 /// The certificate will be considered valid if CertificatePolicyErrors == SslPolicyErrors.None, otherwise the certificate will be invalid unless the user manually allows the certificate.

 /// </summary>

    Default,

 /// <summary>

 /// The certificate will be confirmed with the user even if CertificatePolicyErrors == SslPolicyErrors.None.

 /// Note: FiddlerCore does not support user-prompting and will always treat this status as ForceInvalid.

 /// </summary>

    ConfirmWithUser,

 /// <summary>

 /// Force the certificate to be considered Valid, regardless of the value of CertificatePolicyErrors.

 /// </summary>

    ForceInvalid,

 /// <summary>

 /// Force the certificate to be considered Invalid, regardless of the value of CertificatePolicyErrors.

 /// </summary>

    ForceValid

}

If the certificate presented had any intrinsic errors (e.g. it was expired), the extension’s CheckCert handler will leave ValidityState untouched to let Fiddler perform its default handling for invalid certificates.

Otherwise, the notaries are consulted over HTTPS. The extension could simply use a HTTPWebRequest object to retrieve the Notaries’ JSON-based responses… but considering that Fiddler itself includes a full HTTP(S)-implementation, we’ll simply use that instead:

byte[] arrBody = Encoding.ASCII.GetBytes("fingerprint=" + oNRI.sServerCertificateHash);

HTTPRequestHeaders oHeaders = Parser.ParseRequest(

    String.Format("POST https://{0}/target/{1}+{2} HTTP/1.1\r\nUser-Agent: {4}\r\nHost: {0}:443\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length:{3}\r\n\r\n", 

        oNRI.sNotaryHost, oNRI.sHostname, oNRI.iPort, arrBody.Length, _sUAString));

Session oVote = FiddlerApplication.oProxy.SendRequest(oHeaders, arrBody, oNewFlags);

ManualResetEvent oWaitForDone = new ManualResetEvent(false);

oVote.OnStateChanged += (sender, eA) =>

{

 if (SessionStates.Aborted == eA.newState)

    {

        FiddlerApplication.Log.LogString("Signaling aborted notary request");

        oWaitForDone.Set();

    }

 else

 if (SessionStates.Done == eA.newState)

    {

        FiddlerApplication.Log.LogString("Signaling completed notary request");

        oWaitForDone.Set();

    }

};

if (!oWaitForDone.WaitOne(GetNotaryTimeout(), false))

{

    FiddlerApplication.Log.LogFormat("Timed out waiting for '{0}'s opinion of '{1}'", oNRI.sNotaryHost, oNRI.sHostname);

 return NotaryResult.NoOpinion;

}

NotaryResult oNR;

switch (oVote.responseCode)

{

 case 200:

 // A reply with status code 200 indicates the notary correctly verified the fingerprint you sent it. In the body of the reply the notary will respond with a JSON encoded list of fingerprints and timestamps the client can cache.

    oNR = NotaryResult.LooksGood;

//...

After each of the notaries returns its opinion, the event handler determines their consensus opinion. If the notaries confirm the certificate, our extension’s event handler will return CertificateValidity.ForceValid. If any suggest that it’s a forgery, the method will return CertificateValidity.ForceInvalid.

To improve performance, the extension caches notaries’ opinions for the lifetime of the current Fiddler instance. The extension also permits the user to exempt certain servers’ certificates from analysis. This is useful because some sites won’t be reachable by notaries (e.g. the intranet site https://mail can’t possibly be reached by notaries) and sometimes notaries have problems with servers that are geographically load-balanced around the world (I commonly get back warnings for ssl.google-analytics.com, for instance).

Limitations and Next Steps

It’s worth explicitly mentioning that this project was designed as a demonstration of Fiddler’s extension model rather than a deployable Convergence solution. If you wanted to enhance the extension, you’d probably want to take on a few tasks:

• Offer a Preference-based configuration model to manage the Notary servers’ addresses and public keys(currently hardcoded).
• Cache the JSON responses containing all “known valid” certificates seen by the Notaries.
• Serialize notary checks so that if two parallel HTTPS connections are established and return the same certificates, the extension only performs one Notary census.
• Add a visualization for the notary consultations.

Of course, you’re not limited to implementing the Convergence protocol. You could write your own certificate validation extension that implements certificate pinning, one that rejects certificates from “Banana-republic CAs,” or implements any other validity criteria you think are interesting.

Try it out

The code from this article can be found here. You can build the code using Visual Studio 2010 and later. If you simply want to run my version, simply copy HTTPSNotary.dll to %userprofile%\documents\fiddler2\scripts\ and restart Fiddler. Enable the extension using the HTTPSNotary menu.