I'm having a similar problem. I created the project prior to owning or installing telerik controls. I'm using Visual Studio 2017. I have a Windows Forms Application project and a Windows Forms Control Library project. All of the controls and forms are in the Control Library. All projects are using .NET Framework 4.7.2.
When I select the Control Library project, I am able to select Telerik > Telerik UI for WinForms > Convert to Telerik Controls and the App Converter wizard displays. I select "UI for WinForms, v2018.3.1016.40 (Dev) [InstallFolder]" for the version but neither convert whole solution, nor convert specific files works. Convert whole solution causes Visual Studio to close and reopen with no changes to the project or its files, and convert specific files continues to display "Loading files. Please wait..."
I tried adding a new project, both a Telerik WinForms project and also a normal WinForms project and copied over one control and one form to each of those new projects. When selecting either of the new projects, I experienced the same problem as previously described; but, I had an additional Telerik > Telerik UI for WinForms > Upgrade Wizard for both of the new projects. The wizard completes successfully and shows 0 items processed.
(Side note, there are two typos in the Convert to Telerik Controls wizard's dialogue)