FIddlerCore blocks web pages redirection

[ADEL]

Hi,

 

I am using FiddlerCore with PuppeteerSharp browser automation tests in order to get performance metrics (thanks to FiddlerCore) like page loading time, DOM loading time, TTFB, HTTPs handshake time, ns lookup...etc.

I am facing a problem trying to access some website like: https://outlook.office.com which redirects to to Microsoft login portal. I have the impression that I am missing some configuration in my code but I cannot figure out what the problem is. The code I am use is the following: 

internal class FiddlerProxy : IDisposable
{
    public string Uri { get; private set; }
    public Proxy ProxyEndpoint { get; private set; }
    public int SessionCount { get; private set; }

    internal FiddlerProxy(string uri)
    {
        Uri = uri;
        ProxyEndpoint = null;
        SessionCount = 1;
    }

    public void Start()
    {
        // Register for the FiddlerCore events
        FiddlerApplication.AfterSessionComplete += OnAfterSessionComplete;
        FiddlerApplication.BeforeRequest += OnBeforeRequest;
        FiddlerApplication.OnNotification += delegate (object sender, NotificationEventArgs oNEA) { Console.WriteLine("** NotifyUser: " + oNEA.NotifyString); };
        FiddlerApplication.Log.OnLogString += delegate (object sender, LogEventArgs oLEA) { Console.WriteLine("** LogString: " + oLEA.LogString); };

        // Create an HTTPs certificate and Start the proxy
        if (!InstallCertificate())
            throw new InvalidOperationException("Creating a certiicate failed. Https traffic cannot be captured.");

        if (FiddlerApplication.IsStarted())
            Stop();

        FiddlerApplication.Startup(0, FiddlerCoreStartupFlags.Default & ~FiddlerCoreStartupFlags.RegisterAsSystemProxy);
        ProxyEndpoint = FiddlerApplication.CreateProxyEndpoint(FiddlerApplication.oProxy.ListenPort, true, "localhost");
    }

    public static bool InstallCertificate()
    {
        if (!CertMaker.rootCertExists())
        {
            if (!CertMaker.createRootCert())
                return false;

            if (!CertMaker.trustRootCert())
                return false;
        }

        return true;
    }

    private void OnBeforeRequest(Session session)
    {
        if (!ShouldProcess(session))
            return;
        else
        {
            // Do something
        }
    }

    private void OnAfterSessionComplete(Session session)
    {
        if (!ShouldProcess(session))
            return;

        Console.WriteLine($"Session no. {SessionCount++} coming from :{session.fullUrl}:");
        Console.WriteLine(session.Timers.ToString(true));
    }

    private bool ShouldProcess(Session session)
    {
        if (session.RequestMethod.Equals("CONNECT", StringComparison.InvariantCultureIgnoreCase))
            return false;

        if (session.fullUrl.Contains(Uri))
            return true;
        else
            return false;
    }

    public void Stop()
    {
        if (ProxyEndpoint != null)
        {
            ProxyEndpoint.Detach();
            ProxyEndpoint.Dispose();
        }

        FiddlerApplication.AfterSessionComplete -= OnAfterSessionComplete;
        if (FiddlerApplication.IsStarted())
        {
            FiddlerApplication.Shutdown();
        }
    }

    public void Dispose()
    {
        Stop();
    }
}

Then I use FiddlerProxy class in Main() method in Program.cs as follows:

static void Main(string[] args)
{
    var uriToMonitor = "https://outlook.office.com";
    using (var fiddlerProxy = new FiddlerProxy(uriToMonitor))
    {
        fiddlerProxy.Start();
        LoadPageUsingPuppeteerSharp(uriToMonitor, fiddlerProxy.ProxyEndpoint.ListenPort).Wait();
    }
    Console.ReadLine();
}

public static async Task LoadPageUsingPuppeteerSharp(string uri, int proxyPort)
{
    var launchOptions = new LaunchOptions
    {
        ExecutablePath = @".\chrome-win\chrome.exe",
        Headless = false,
        IgnoreHTTPSErrors = true,
        Args = new[] {
            $"--proxy-server=127.0.0.1:{proxyPort}",
            "--no-sandbox",
            "--disable-infobars",
            "--disable-setuid-sandbox",
            "--ignore-certificate-errors"
        }
    };

    using (var browser = await Puppeteer.LaunchAsync(launchOptions))
    {
        var context = await browser.CreateIncognitoBrowserContextAsync();
        using (var page = await context.NewPageAsync())
        {
            await page.SetUserAgentAsync("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3563.0 Safari/537.36");
            await page.SetViewportAsync(new ViewPortOptions { Width = 1500, Height = 800 });

            await page.GoToAsync(uri);
        }
    }
}

 

Attached are the console output that I get when running my program, and two screenshots of my chrome browser.

One last question, is it possible to configure an HTTP Proxy (some proxy that we have in our organisation) through which FiddlerCore can send requests?

 

Thank you in advance for your help.

Adel

[Eric R | Senior Technical Support Engineer]

Hi ADEL,

It appears that in this specific case related to Office 365, the application needs to process the Authentication Flow. The login redirect expects to process some information like, user id/password, application id/password. Unfortunately, this would need to be built into the application and is not related to FiddlerCore.

If inspecting Office 365 traffic is the goal, I recommend using the Desktop Fiddler version with the Office 365 Fiddler Extension. Note that this extension is unsupported by Fiddler and any issues with the extension would need to be submitted to their Issues Repository. 

I hope this helps. Please let me know if you need any additional information. Thank you for building with FiddlerCore.

Regards,

Eric R | Senior Technical Support Engineer
Progress Telerik

[ADEL]

Hi Eric,

Exactly, I am trying to inspect Office365 traffic. I understand the problem with modern authentication and FiddlerCore. I will try to find another solution.

Thank you so much.

Best,
Adel