I would suggest to check the following demos - I believe they will be of help:
Returning Values from a Dialog
Communication between RadWindows
Using RadWindow URL to supply server arguments
I hope this information will get you started. You can find more details about the used approaches in the documentation as well.
If you still experience problems, I recommend to achieve the desired functionality not by using RadWIndow but a standard browser's popup (window.open). Since RadWindow is intended to replace the browser's popup, once you have your logic working as expected with a popup, it will work with RadWindow as well.
the Telerik team