I have inspected the issue by putting RadTreeView in a ContentTemplate of a RadWindow to the CheckBox demo with the code below but the checked nodes were still returned properly:
<telerik:RadTreeView ID="RadTreeView2" runat="server" CheckBoxes="True" Height="280px"
Hope this will be helpful.
the Telerik team
If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to their blog feed