Unfortunately the Z-order of the RadWindows is only for the RadWindows and they cannot be mixed with other popups (or at least this cannot be controlled easily). The RadWindow currently brings to the front by closing and opening its popup, but this is because of a bug in Silverlight and might be changed.
As you correctly noticed you could achieve this by changing the control template of the RadWindow
and showing a modal window.
the Telerik team