I'm not sure of setting that using JavaScript but you can get this done by setting the Behavior property of the RadWindow to Close on the aspx side as shown below in the code snippet.
This functionality is controlled by the Behaviors property of the RadWindow control. For example if you want to have only the close button in the toolbar, but you also need this window to be moveable and resizeable, you need to set Behaviors="Close,Move,Resize".
If you need to set this property on the client: