.Net 7 - Kestrel - WebApi - HttpClient

[Philipp]

Hi, we are hosting web api controllers using kestrel (.net 7, c#, self hosted (not within IIS))

Within a controller we use a HttpClient to make a simple get request (https), but we are not seeing the request in fiddler everywhere (Http2 is activated in the settings and also the certificate stuff is activated).

The request to the controller (https & http2) is captured by fiddler, but not the httpclient call (the problem occurs on Win Server 2016), we are using the server name to call the controller (not localhost).

However if I create a console app, add the HttpClient call and start the console, fiddler does capture the call as expected.

I also tried netsh winhttp set proxy 127.0.0.1:8866 as admin, but that also doesn't work.

Any ideas?

 

Br

[Nick Iliev]

Hello Phillipp,

 

While we may not fully comprehend how the Kestrel server works, there are steps you can take on your end to address the HTTPS capturing problem. One option is to explicitly configure the .NET framework to use Fiddler as a proxy, which you can do by following the instructions provided in the documentation sections linked below:

https://docs.telerik.com/fiddler-everywhere/knowledge-base/capturing-net-traffic#capturing-net-traffic-through-system-capturing 

Another solution is to instruct your HttpClient to use Fiddler as a proxy.

using System.Net;

var proxy = new WebProxy
{
    Address = new Uri($"http://127.0.0.1:8866"), // or the IP address of the Fiddler Everywhere host machine. In case you are using a remote host, ensure that "Allow remote computers to connect" is enabled in FIddler's settings
    BypassProxyOnLocal = false,
    UseDefaultCredentials = false,
};

// Create a client handler that uses the proxy
var httpClientHandler = new HttpClientHandler
{
    Proxy = proxy,
};

// Disable SSL verification
httpClientHandler.ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;

// Finally, create the HTTP client object
var client = new HttpClient(handler: httpClientHandler, disposeHandler: true);

var result = await client.GetStringAsync("https://docs.telerik.com/fiddler-everywhere");

 

Regards,
Nick Iliev
Progress Telerik

[Philipp]

@Nick Iliev

 

Only using a web proxy works, but that means we have to create a solution to add this web proxy on demand (only when we want to use fiddler), that's quite cumbersome and takes some time (and money) for us to implement, is there no other solution, we own a Pro Edition of Fiddler Everywhere, if there is no other way than this should be listed as known issue/limitation!

 

Br

 

var proxy = new WebProxy
{
    Address = new Uri($"http://127.0.0.1:8866"), // or the IP address of the Fiddler Everywhere host machine. In case you are using a remote host, ensure that "Allow remote computers to connect" is enabled in FIddler's settings
    BypassProxyOnLocal = false,
    UseDefaultCredentials = false,
};

[Nick Iliev]

The HttpClient library can be used alongside Fiddler Everywhere (for example, through a simple console application), so it doesn't seem like there is a specific limitation - it is a library that normally respects the global proxy settings but also has the option to be configured explicitly. On the other hand, Fiddler Everywhere is just a forward proxy and will capture any traffic from any application configured to utilize the Fiddler proxy address and port (including HttpClient). That said, it looks like there is a specific configuration issue related to your application architecture and the Kestrel server - meaning that some root process is probably not respecting your proxy settings  (if Fiddler acts as such through the system capturing) or needs additional configuration (to point to the Fiddler Everywhere address and port explicitly).

You can try to start your process through the independent Fiddler's terminal or through the independent Fiddler's browser (which is preconfigured to respect the proxy)  - this way, you can test if the application (that uses HttpClient) is actually respecting the proxy settings or need an additional configuration. Alternatively, you can try to set the http_proxy and https_proxy environment variables so that the app (that uses HttpClient) can be proxied through Fiddler (refer to this documentation section for details).

 

 

[Philipp]

@Nick IIiev thx for the detailed information!

[Philipp]

@Nick the HTTP Client is only not captured when we start our application as Windows Service, as soon as we start it e.q in a command line (service user and user which started the command line are the same).

 

 

Do you have any ideas what could cause this?

 

IT SEEMS THAT Windows Service can't read the proxy settings or don't have one at all! (its a windows service restriction)

[Nick Iliev]

You can also explicitly set your NET services to use the FIddler Everywhere proxy (by reconfiguring the NET application configuration files).

https://docs.telerik.com/fiddler-everywhere/knowledge-base/capturing-net-traffic#configuring-proxy-in-net-application 

 

Additionally, some applications running as Windows services won't respect the proxy settings, so that said these must be passed explicitly through the described flows by Microsoft. Refer to the following articles for detailed information on how to handle similar scenarios:

https://www.telerik.com/blogs/using-fiddler-with-winhttp - This article was originally written for Fiddler Classic, which uses port 8888, but is also valid for Fiddler Everywhere, which uses port 8866.

https://learn.microsoft.com/en-us/windows/win32/winhttp/setting-wininet-proxy-configurations-in-winhttp#the-application-is-running-as-a-service 

https://rmm.datto.com/help/en/Content/Troubleshooting/TechnicalInformation/KB360021805932.htm