Your issue stems from the fact that your content is actually loaded inside an iframe, which means that we cannot take the modal overlay outside of that iframe, as it creates a separate window object and you cannot take anything out of it.
The general approach is shown in the following help article: http://www.telerik.com/help/aspnet-ajax/window-programming-opening-from-within-dialog.html
. On the different way s to open a RadWindow please examine this one: http://www.telerik.com/help/aspnet-ajax/window-programming-opening.html
the Telerik team