Unable to capture Python Requests after Latest Update

[humaxuyohi]

Cannot sniff python script from command-line anymore after new update.

the script is using python-requests to make a get or post requests.

Last seen working on v5.0.20182.28034

 

[Kammen]

Hello,

Do you get any error messages, for example in the logs, that you can share in order to further investigate this?

Regards,
Kammen
Progress Telerik

[rocky]

I'm getting the same problem.  Where would I look for log files that I can send you?  Yesterday, I used Fiddler (sorry, I don't know what version) to capture http traffic from a python script.  Today, Fiddler updated to "v5.0.20194.41348 for .NET 4.6.1".  Now, it doesn't capture http traffic from the same python script. 

[Eric R | Senior Technical Support Engineer]

Hi Rocky,

There are a couple of ways to get information about web sessions. The Fiddler Log Tab is the first location that provides more information about how Fiddler is operating. This located in the Right-hand Side of the GUI. See the below screen shot for a visual reference.

Although, additional information is important for understanding the scenario. Are you able to provide the following information?

1. Are you seeing anything in the Session Output? If so, can you select it and Export the Session as an Archive?

2. What Operating System are you using Fiddler with?

3. Is the Python Script only sending and receiving HTTP/HTTPS Traffic? 

4. Are there any Python errors that are being written to the Console?

Once I have this information I will be able to provide a more thorough response. Thank you for using the Fiddler Forums.

Regards,

Eric R | Technical Support Engineer
Progress Telerik

[rocky]

1. Are you seeing anything in the Session Output? If so, can you select it and Export the Session as an Archive?

No.  There's nothing in the session output.
2. What Operating System are you using Fiddler with?

Windows 10 Enterprise
3. Is the Python Script only sending and receiving HTTP/HTTPS Traffic? 

Yes.  Here's a sample python script:

import requests

r = requests.get("http://www.google.com")

4. Are there any Python errors that are being written to the Console?

No

The log tab is this:

-= Fiddler Event Log =-
See http://fiddler2.com/r/?FiddlerLog for details.

14:10:32:7981 Fiddler Running...
14:10:32:7991 AutoProxy failed. Disabling for this network.
14:10:32:8998 Windows 8+ AppContainer isolation feature detected.
14:10:35:7941 Assembly 'C:\Users\rocky\AppData\Local\Programs\Fiddler\CertMaker.dll' was not found. Using default Certificate Generator.

If I use Chrome to browse the web, I see plenty of sessions in Fiddler.  The call to the requests library in python to get www.google.com however does not show up in Fiddler.  Python is running in a cmd window.

[Eric R | Senior Technical Support Engineer]

Hi Rocky,

Thank you for providing the additional information. Although, looking at the script, I can see that this will return a blob of unreadable information. I have also tested this as shown in the below screenshot. 

 

If you're not seeing any requests in the Sessions Pane, I believe that this may be a different issue. Can you confirm if you are able to capture any requests? Thank you.

Regards,

Eric R | Technical Support Engineer
Progress Telerik

[rocky]

I can.

 

 

[Eric R | Senior Technical Support Engineer]

Hi Rocky,

That's great! It looks like you're still capturing other requests. Is it possible that you can perform the same test as the screenshot I previously provided? See the below steps to perform the test.

1. Open Fiddler

2. Open a Command Prompt

3. Enter the Python REPL

4. Execute the requests.get('http://www.google.com') command

5. Take a Screenshot of Fiddler and the Command Prompt overlay

6. If you do see an entry in the Session Pane, select it and export it.

Additionally, if you're note seeing requests and are using HTTPS, this could be due to the Fiddler Root Certificate not being honored. See the Running Fiddler to Capture Python Requests forum post for more information.

I hope this helps. Please let me know if you are able to perform the above test. Thank you and I look forward to your reply.

Regards,

Eric R | Technical Support Engineer
Progress Telerik

[rocky]

Here's the screen shot of the test.  As you can see, there is no entry in the Session Pane from the python process.

[Eric R | Senior Technical Support Engineer]

Hi Rocky,

Perfect! This shows that the AutoProxy is failing which would result in requests not being captured. This can happen in a Domain Joined Machine with Group Policy Settings, if VPN is in use or if Firewall Software is blocking the AutoProxy. These are usually outside of Fiddler.

Alternatively, other user's have reported that removing the Fiddler Root Certificate from the Windows Certificate Store and then re-enabling Decryption has worked. To clear the Installed Certificates follow the below steps.

1. Navigate to Tools -> Options

2. In the HTTPS Tab of the Options Dialog choose Remove all Interception certificates

3. Perform the Decrypt Traffic process again.

Please give this a try and let me know if you need any additional information. Thank you.

Regards,

Eric R | Technical Support Engineer
Progress Telerik

[rocky]

I followed the steps you gave but I still don't see anything captured.  I think you're correct that the AutoProxy is failing.  I'll research that and post back if I fix it.  Thanks