The problem is that the Toggle Button in the Expander header is ~30px in height. This is limiting the min height of the RadExpander. In our Q2 release we will add a property of type Style for the toggle button, which will give you the ability to modify this behavior.
For the moment the only way to achieve smaller header height is to edit the template.
In the attached example we demonstrated two expanders: first with whole Style replaced and second only with TemplateDown initialized. Please note the initialization of the HeaderTemplate property will not helps you because it will not apply for the whole header but only for its content.