Fiddler, websockets, and iOS

[Andy]

I'm trying to use Fiddler to proxy an application that uses both HTTP and web sockets.  The web sockets are actually BLIP.

When the application runs, I see all the http traffic just fine in Fiddler, however I see no indication of the web sockets in Fiddler.  In the application logs I can see the web sockets timing out.  In this particular scenario the application waits for the data on the web sockets and no longer progresses as the data is required.  This only happens when using a fiddler proxy.

Is there something special I need to do in order to get fiddler to proxy the web sockets?  If there's a limitation, say due to not supporting BLIP, is there a way to configure fiddler to not proxy any web sockets but still proxy http?

TIA

[Nick Iliev]

Hello Andy,

 

While searching for more information on the BLIP protocol, we noticed the following technical description from the linked source.

The BLIP protocol runs over a bidirectional network connection and allows the peers on either end to send messages back and forth. The two types of messages are called requests and responses. Either peer may send a request at any time, to which the other peer will send back a response (unless the request has a special flag that indicates that it doesn't need a response.)

Messages are multiplexed in transit, so any number may be sent (or received) simultaneously. This differs from protocols like HTTP 1.x and WebSockets where a message blocks the stream until it's completely sent.

So while there are not many technical details on how exactly the protocol is transporting its messages, it sounds like this is now a protocol based only partially on HTTP and WebSockets and uses a different implementation. A bit later, the specification goes as follows:

BLIP is layered atop a message-based protocol called the transport, typically WebSocket

....

A BLIP client opening a connection MUST request the WebSocket subprotocol BLIP_3. A server supporting BLIP also MUST advertise support for this subprotocol. If one side supports BLIP but the other doesn't, BLIP messages cannot be sent; either side can close the connection or downgrade to some other WebSocket based schema.

BLIP messages are sent in binary WebSocket messages; text messages are not used, and receiving one is a fatal connection error.

Both BLIP and WebSocket use the terminology "messages" and "frames", where messages can be broken into sequences of frames. Try not to get them confused! A BLIP frame corresponds to (is sent as) a WebSocket message.

The BLIP transport is loosely based on the WebSocket protocol, but it differs in implementation (especially in how BLIP sends and uses messages as frames). It looks like that BLIP is a subprotocol with core differences compared to the WebSocket protocol. The specification suggests that if the client does not support BLIP (in your case, FIddler is a client to the server), then both should negotiate and use WebSocket instead. However, this is not happening, which suggests that the server enforces the usage of BLIP or that the negotiation fails for some other reason. 

Fiddler supports capturing WebSocket out-of-the-box (no additional configuration is needed - you can immediately capture WebSocket through an online test site like https://www.piesocket.com/websocket-tester) but does not officially support BLIP. Note that the BLIP owner also archived the official repository in March 2022. There has been no activity in the BLIP repositories for several years, so there is limited support for that protocol.

Still, if possible, please share a demo application or endpoint that reproduces the BLIP issue so we can investigate further. One possible thing to check is if you can see a completed TLS handshake - if not, you probably need to set your BLIP application to fully trust the Fiddler CA so that the TLS handshake does not fail.

 

 

Regards,
Nick Iliev
Progress Telerik

[Andy]

Thank you Nick for the very detailed response.

One other piece of pertinent information is the application is a Xamarin application running on iOS.  So, I'm using fiddler as a proxy for a physical iOS device.

While the official repository was archived, BLIP has been consumed, and I believe maintained, in the couchbase lite core project, which is still active.  Couchbase is the product we're using where proxying with Fiddler becomes an issue when synchronizing the database.

I toggled all the cert flags I could find in Couchbase's api with no luck.  I've also turned on all the Couchbase logging I could find, but nothing indicates it's a certificate trust issue, though I could be wrong.  I'm simply receiving messages that the connection timed out.

The best sample application I could provide is the one provided by Couchbase as a sample, but it has a decently involved pre-reqs (which appear to be detailed in the tutorial) as you have to run their server in a docker container.  https://developer.couchbase.com/tutorial-quickstart-xamarin-forms-sync

At this point, I'd be happy to simply have fiddler pass-thru the the Couchbase traffic so my application can continue and I can still inspect the http requests.

[Nick Iliev]

Hey Andy,

 

While looking into the issue and testing different WebSocket endpoints, it appears that there is currently a bug in the recent iOS versions that, as a result, are causing WebSocket requests (ws:// and wss://) never to use the set system proxy and to always go through the default gateway. Different users have reported the issue multiple times, but so far, Apple has released no official statement on when the problem will be resolved. I've also confirmed the iOS bug with Safari, where the WSS connection to https://www.piesocket.com/websocket-tester never goes through the proxy (when the connection goes through the iOS Safari) but works as expected if opened from another client (that respects the Fiddler proxy for HTTP/HTTPS).

That said, BLIP builds its transport upon the WebSocket protocol. You might hit the same issue as your Xamarin application executes the request through iOS. If that is the case, you should be able to capture other HTTP(S) requests but not the BLIP/WebSocket request made from the iOS device.

However, the issue might differ if you cannot capture any HTTP(S) request (like a  request to https://example.com) from your iOS application.

If you own the Xamarin application, you could try explicitly setting its requests through the Fiddler proxy (see this blog post for a possible approach). Of course, that would work if you have access to the application's codebase, as the change requires an application rebuild. Setting the iOS library that executes the HTTP/HTTPS request to explicitly use Fiddler as a proxy should (in theory) solve both the case where you are not able to capture any HTTP(S) request and where you are not able to capture WebSocket requests due to the Apple bug described above.

Update: While investigating what might cause the WebSocket traffic to not go through the proxy, I've also stumbled upon this issue logged in the Xamarin repositories:

https://github.com/xamarin/xamarin-macios/issues/5527 

https://github.com/mono/mono/issues/12758 

... the web socket is not being recorded by Charles (or Fiddler).

Actual Behavior
The ClientWebSocket connects and sends all the data without taking into account the proxy.

The issue describes yet another critical bug (this time in ClientWebSocket) that prevents the WSS from going through a predefined proxy. Instead, all WSS connections are using the default gateway.