I reported the issue below back in September 2017. It was scheduled for a fix in R1 2018 (here). However, it appears that the problem persists. Did it get missed or is there a setting/option that I'm missing. I think it is a rather major issue for anyone trying to put a DPI aware app together.
This is the problem:
In a DPI aware app, using the docking tool window auto-hide dramatically increases the size of any controls on the window. This can be demonstrated with a new application. Add a RadDock to the form, add a couple of tool windows and a document window, and add a few controls to either of the tool windows (label, button, treeview, property grid, etc.). Run the app as DPI aware at 150 scale. Hide a tool window. Open the tool window. When the tool window slides open all the font and control sizes have greatly increased.