You can edit the ControlTemplate of the RadTabitem in the Expression Blend and use the SelectionVisual and NormalVisual bordersin order to style the tab items when they are selected or unselected accordingly. The described approach is used in the attached solution. You can give it a try and let us know if it satisfies you.
Feel free to ask if you need more info. We would be glad to assist you further.
Regards,
Petar Mladenov
the Telerik team
Browse the videos here>> to help you get started with RadControls for WPF