Indeed, you have turned in the right direction. The toolbar-dropdownbutton and the regular DropDownList component are two different components. Thus their features aren't shared. Thus, as you have inferred, the way to add a custom layout to the popup of a responsive Toolbar is to implement a Custom Control Type:
The last example on the referenced article above demonstrates how to add a Kendo DropDownList Component as a custom tool to a responsive ToolBar by using both the toolbarTemplate and popupTemplate. I am pasting it here for reference:
Please let me know in case further assistance is required for this case. Thank you.
Progress is here for your business, like always. Read more
about the measures we are taking to ensure business continuity and help fight the COVID-19 pandemic.
Our thoughts here at Progress are with those affected by the outbreak.