This feature is not supported out-of-the box and there is not a way to achieve it with the current editor version.
You can only restrict the styles that are loaded as items in the Apply Class using the class
tags in the ToolsFile.xml file:
class" value="a.link" />
class" value=".img" />
The tags above will load in the dropdown only the two classes a.link and .img if they are included in the page with the editor or imported through the CssFiles property of RadEditor.
the Telerik team