You can inspect the HTML elements that you want to style with IE developer toolbar and Firefox. Note that you should delete the browser cache and restart the browser to see the applied changes to the dialogs because the browser caches the CSS files.
If you want to get updates on new releases, tips and tricks and sneak peeks at our product labs directly from the developers working on the RadControls for ASP.NET AJAX, subscribe to their blog feed now.