grpc - incomplete Message

[Black]

Hey,

currently trying the grpc Feature of Fiddler Everywhere. I have no Protofile (not sure if it is required at all).
But when I running Fiddler as Interceptor (I using Proxifer to reditrect traffic to Fiddler Everywhere)

it tells me: Incomplete Message on the Response.

When I run the App without Fiddler it works, so I assume Fiddler can't handle the grpc yet. Or is the Protofile required for him to handle  + forward the message?

[Nick Iliev]

Hello,

 

The ProtoBuf file is needed to decrypt the received message successfully. Even without a .proto file, the message should be received as expected, given that the gRPC connection is successful (Fiddler will try to decrypt the message, and even if that is not possible, it will show the original encrypted content).

The above said the issue on your side is most likely related to one of the following:

- Issue related to a specific gRPC implementation used within the client or server application (for example, you can successfully capture any other gRPC traffic while using Proxifier alongside Fiddler Everywhere).

- The issue is related to how the Proxifier operates (for example, you can't capture any other gRPC traffic while using Proxifier alongside Fiddler Everywhere).

In both cases, we will provide as much information as possible and, if applicable, a demo gRPC client & server application that we can use to reproduce the issue. You can also share the following information:

- What concrete technology is used to create the gRPC client and server applications?

- Are you experiencing the same issues with other gRPC services (e.g., the ones from grpc.io or other public resources)?

The issue could be related to how the Proxifier tool (or the tool alongside the specific gRPC server app) handles the CONNECT Tunnels (for gRPC sessions). By default, Proxifier uses only IP addresses when creating a CONNECT, while your server might use SNI. While Fiddler Everywhere is handling the transition from IP to hostnames, this could be an issue when traffic goes through Proxifier. You can try to test your application without Proxifier (directly through Fiddler) - this way, you will determine if the issue is strictly related to Proxifier or the concrete gRPC client/server implementation and Fiddler Everywhere.

 

Regards,
Nick Iliev
Progress Telerik

[Black]

Hey Nick!

The GRPC Client is a Game,  which is using GRPC. It uses the default Google Protobuff Integration (Unity).
Messages are just JSON Content, same as answers, so no real magic.

We just created the required Proto Files, so no issue on that end anymore.

Here are my Testcases so far:

Target URL is an IP : 127.0.0.1:55555
Fiddler + Proxifier + Game = Incomplete Message for the GRPC URL
Fiddler + Proxifier + Postman = Incomplete Message for the GRPC URL

See Fiddler.png

grpc-dumper + Proxifier + Game = Working, can read the Traffic
grpc-dumper + Proxifier + Postman = Working, can read the Traffic

GRPC-Dumper dont use any certificate, since its not encrypted (yet)

See: RequestGRPC.png

My assumption was, that the issue relies somewhere in Fiddler Everywhere. But I can be wrong ofc.
Not sure if Fiddler is supposed to edit messages in grpc - of this is a planned feature (which would be great I guess)

* grpc-dumper
https://github.com/bradleyjkemp/grpc-tools/releases/tag/v0.2.6

[Black]

Small Update:

I tested various tools now Mediator, etc. and they all work when running via Proxifier. 
Its only Fiddler who "breaks" the GRPC Requests.

PS: Is it planned to intercept / modify GRPC Requests?

[Nick Iliev]

Based on the details you provided, it appears that the issue may be related to how Proxifier and Fiddler Everywhere interpret the CONNECT Tunnels and subsequent SNI resolution. To address this, you can try creating a rule in Fiddler Everywhere that explicitly replaces the IP address with the hostname for all gRPC sessions (except the CONNECT Tunnels).

For example, if you have a gRPC service running on 127.0.0.1:4000, the corresponding hostname would be ozone.bg you can create the following rule:

[Black]

Hey, thanks for the reply!

- just in case when talking about 127.0.0.1 we actually mean a real IP so nothing in local network -

So I added a rule as you shown (just with my IP)
I noticed it does not modify the GRPC Request, so I tried to use the "Marking" Tool, to confirm. And it in fact not Marking the Session. (It still says 1 match found in current tab, but he ignoring the modification on it)

But the Handshake looks alright anyway already on GRPC.  The Handshake shows:

:authority : 127.0.0.1:4533

I don't know how fiddler work under the hood and I am prob. way less skilled enough, but I was curious, because all the tools I tested so far working fine, showing the content and allow the connection. (Mediatore, grpc-dumper and so on)

Just fiddler Everywhere seems to "break" the Connection.
Also I am not sure where I saw it, but I think in charles Proxy it said PNI. (Charles also wasn't able to decode it)

Also attached, is an Image of the Name Resoultion Setting of Proxifier.

[Nick Iliev]

Hey,

 

As an update related to the gRPC capturing with Fiddler, the team has just released a fix for a major issue in the gRPC capturing feature. So, it's recommended that you update to the latest version of Fiddler Everywhere (currently 5.7.1) to make use of this fix.

We are currently investigating a possible incompatibility between the name resolution settings in Proxifier and the default behavior in Fiddler Everywhere. If you're experiencing issues, it's a good idea to try capturing the gRPC traffic with and without the Proxifier application running while using the latest version of Fiddler Everywhere (version 5.7.1) where the mentioned fix is included. 

[Black]

Hey Nick.

I want to let you know, that 5.7.1 works fine, GRPC is tracked and decompiled with Protofiles.

Is there any plan to add Interceptors / Modify traffic in between? Using the Proto Files maybe? (Byte modification could prob. work too, but thats a lot of messing)

[Nick Iliev]

Thank you for confirming that the problem related to the "Incomplete Message" error in Fiddler Everywhere while the gRPC traffic was passing has been resolved in version 5.7.1. Your suggestion of adding the ability to modify gRPC messages in Fiddler sounds like an excellent feature. I have created a feature request for it in the following GitHub thread, which you can use as a reference:

https://github.com/telerik/fiddler-everywhere/issues/11 

 

[Nick Iliev]

Hello Everyone,

 

As an update related to the reported issue, we can confirm that the fix included in version 5.7.1 resolves the issue of being unable to properly capture gRPC traffic. To resolve the issue, update your Fiddler application through the Help > Check for Updates in-app option or by installing the latest version from https://www.telerik.com/download/fiddler-everywhere 

 

Regards,
Nick Iliev
Progress Telerik