RadFormDecorator creates a table in older browsers that do not support CSS 3(e.g. IE7/IE8), in order to simulate the rounded corners, while in new browsers it uses CSS 3. That is why there is some inconsistency in the padding of the elements that are inside of the FieldSet under older and newer browsers. You can set padding of the elements that are inside of the FieldSet, so that they fit in older browsers as well.
Regarding the FieldSet issue under IE8 - this may result due to the set of a global CSS rule applied to TD/TR tag. I suggest that you set TD/TR rules for the particular table not the whole page.
Regarding the inconsistency of the legend color - I was not able to reproduce it. Could you test our online demos
and then tell us what is the problematic one and the applied skin. Otherwise, if the issue is related to RadFormDecorator, you can open a support ticket and send us a runnable project, so we can do an investigation locally.
I also advise that you make sure your browse is not zoomed, because this can also break pages.
All the best,
the Telerik team
Explore the entire set of ASP.NET AJAX controls we offer here
and browse the myriad online demos
to learn more about the components and the features they incorporate.