The radprompt tempalte allows simple HTML only - you cannot put ASP.NET / ASP.NET AJAX controls in it which means that RadSpell cannot be inside that template.
What I can suggest in your scenario is not to open a radprompt dialog but a RadWindow set up with a content page that will look like a prompt window. On that content page, you can put the textbox and the radspell control and have the spellcheck that you need.
More information on how to use RadWindow as a dialog is available here:
the Telerik team
Check out Telerik Trainer
, the state of the art learning tool for Telerik products.