You are correct; your scenario is slightly different and poses an automation challenge. I have reproduced this locally with a Microsoft demo site and will file a bug for it.
In the meantime you can try a work-around. Insert a "Navigate to" step after the last test step, but before the "Handle OnBeforeUnload dialog" step. It can navigate anywhere, like google.com for instance. This way, the "Are you sure you want to navigate away from this page" alert will be prompted by navigating to another page and not by closing the browser. Test Studio remains engaged for an additional step, properly handles the dialog, and navigates to Google.
See the attached screen shot for an example of the step order.
All the best,
the Telerik team
for a live 'What's New in Test Studio R1 2011 SP2' event on Tuesday, July 19 at 2pm EST!
Have you looked at the new Online User Guide
for Telerik Test Studio?