Original request reworked by Fiddler

[idia]

Hello,

I have a very strange behavior, I build on a Java program a request to query a site. For your information, here is the code, but it's not the root of the problem:

conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(SimpleCrawler.timeout);
		      conn.setReadTimeout(SimpleCrawler.timeout);
		      conn.setRequestProperty( "Host", url.getAuthority() );
		      conn.setRequestProperty( "User-Agent", userAgent );
		      conn.setRequestProperty( "Accept", SimpleCrawler.acceptPageType);//text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
		      conn.setRequestProperty( "Accept-Language", SimpleCrawler.acceptLanguage );
		      conn.setRequestProperty( "Accept-Encoding", "gzip, deflate, br" );
InputStream input = conn.getInputStream();
			  byte[] buffer = new byte[4096];
			  int n;

			  OutputStream output = new FileOutputStream( chemin );
			  while ((n = input.read(buffer)) != -1) 
			  {
			      output.write(buffer, 0, n);
			  }
			  output.close();

And the request downloads a corrupted file. Now if I go through fiddler with this code :

 Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("127.0.0.1", 8888));
		    	  conn = (HttpURLConnection) url.openConnection(proxy);

 

I see the request go through Fiddler and the file downloads correctly.

It's as if Fiddler modified the return or the request by reorganizing something to correct a problem that was there before. But I can't figure out what it is, if I knew I'd fix my problem.

Do you have any idea why I am seeing this behavior?

Thanks

[Nick Iliev]

Hello Idia,

 

If you're experiencing issues with certain scenarios appearing broken, Fiddler's applications, such as Fiddler Classic, have the ability to "fix" them. In your situation, one possible solution is to test the streaming option in Fiddler by enabling or disabling it.

Additionally, you may want to consult this blog post for other potential causes of varied behavior: https://www.telerik.com/blogs/help!-running-fiddler-fixes-my-app.

 

Regards,
Nick Iliev
Progress Telerik

[idia]

Hello, after some research, it's a problem concerning both a forgotten close at the end of the request and, above all, a problem with the GZIP encoding in the output (Fiddler automatically decodes requests in my configuration, so the output encoding is clear because the GZIP decoder is in the Fiddler layer).

I suggest you add this last point to your documentation, thank you for your help.