Did you configure Internet Explorer according to this article? In order your browser to work properly with Test Studio it needs to be configured properly. One of the settings required is to have tabs opened in a new window, instead of a new tab.
Test Studio will connect to any new window (no tab) of Internet Explorer.
If you haven't done so already, please calibrate your browser as seen in this article.
If I am missing something please elaborate a bit more on your scenario.
Please record a video which is demonstrating the issue. I suggest using Jing due to it's intuitive interface but any video capture software will work.
Hope to hear from you soon.