Just by looking at the snippet, I can tell it's TabStrip.css located in the Telerik.Web.UI\Skins folder.
For the second part of your question, I am not 100% certain what you mean by "failing compatibility and accessibility tests".
If the styles are wrong, you can easily override them by using a stronger selector like (the highlighted part is what you add to have a stronger selector):
* html div.RadTreeView .rtMinus{ /*correct styles*/ }
If you mean that the styles are failing CSS validation, you should not worry, because it [the validation] doesn't mean anything in this particular case: we are using CSS hacks to target only IE 6 browsers.
Regards,
Ivan Zhekov
Telerik
If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to the blog feed now.