This is a little bit out of the box and you need to use a DataTemplate to insert a Button in the RadPane header as shown in this
help article. So the tricky part is that you need to implement the logic on button click and take in consideration if the click comes from a floating Pane or from a docked one. I am attaching a simple scenario using one button for maximizing and restore. The logic uses the previous PaneGroup to restore a docked Pane and in some cases this might not work (depending on your scenario), but if you want to make sure that it can always be restore, you just need to have a static (never to be moved) PaneGroup and restore the Panes there.
If you have more questions, please don't hesitate to contact us.
All the best,
the Telerik team