This behavior is due to a bug in Firefox that occurs when you have an IFRAME in a table - which is basically RadWindow's structure. If you set a very small height (about 150px), the table shrinks as expected, however the IFRAME element doesn't.
So far we were not able to find an appropriate cross-browser workaround for this problem but if we manage to fix it, we will implement the fix right away. For the time being, when you want to display something in a very small container, I would suggest to use RadDock or RadToolTip.
the Telerik team
Check out Telerik Trainer
, the state of the art learning tool for Telerik products.