It's very strange that there's literally no
way for you to set the proxy, because this means that the Java program in question could never be sold for use in an enterprise, as most use proxy servers.
However, you can likely get this working anyway. There are two issues:
1> You have to get Java to trust Fiddler's root certificate.
2> You have to ensure Fiddler is returning the right certificate.
Step #1 is straightforward and it sounds like perhaps you've already done this as you don't have any problem with the application you wrote yourself?
For Step #2, the problem is that Fiddler is deciding what certificate to return based on the target of the CONNECT request. However, because you've "proxified" the application, the CONNECT request contains the IP address of the target server rather than the expected hostname. When Fiddler dutifully returns a certificate for the target IP address, the client application (which doesn't know that it was proxified) rejects that certificate because the SubjectCN=<ipAddress> doesn't match the expected SubjectCN=<hostname> that the client expects.
Fortunately, this problem arose often with older Android devices and so Fiddler offers two ways to workaround the issue.
One way is to force Fiddler to connect to the server, then return a certificate that matches the subject CN from the server; see https://groups.google.com/forum/#!searchin/httpfiddler/android$20SNI/httpfiddler/hvsDR14j1Lg/wDVxZFlS5PgJ
for info on the X-UseCertCNFromServer
The other way is to set the SubjectCN based on the TLS ClientHello SNI extension which is sent if the client supports it. To use that approach, set the preference fiddler.network.https.SetCNFromSNI to true. This will cause Fiddler to use the hostname specified inside the TLS ClientHello handshake to generate the target site's certificate. https://groups.google.com/forum/#!searchin/httpfiddler/fiddler.network.https.SetCNFromSNI$20/httpfiddler/iJk7yctHdFg/LcSficgvg8IJ
With any luck, one of these two approaches will work for you.
Check out the Telerik Platform - the only platform that combines a rich set of UI tools with powerful cloud services to develop web, hybrid and native mobile apps.