We have already replied in the support ticket about this issue, but here is again the answer in case anyone else has similar questions:
"To disperse concerns about integrating our ASP.NET AJAX controls in DNN 4.5+ and 5.x, I can say that the Telerik.Web.UI assembly(based on ASP.NET AJAX) is fully compatible with DNN 4.5 and onward versions. There is a nice blog on DotNetNuke 4.5 and ASP.NET AJAX which you can refer to.
We have a chapter in the help for Integrating RadControls in DNN with how-tos, api reference and download and installation instructions. With Q1 2009, most topics from the DNN chapter will be updated and some new topics will be added.
The SkinObjects and modules that we ship with the Telerik.DNN.SkinObjects_[version].zip and Telerik.DNN.Modules_[version].zip packages respectively can be used in DNN versions 4.5 or higher. For DNN 5.0 you need to download the latest version - Q3 2008 SP2 (2008.3.1314) from your account. Please, see the attached screenshot.
If you have satisfied the version requirements listed above, you can safely use the Telerik.Web.UI controls in a dnn web site in the same way as you would use them in a regular asp.net ajax web site. The tutorial on How to create a DNN module with RadControls for ASP.NET AJAX will help you get started. "
the Telerik team
Check out Telerik Trainer, the state of the art learning tool for Telerik products.
I figured it out. The DNN module from Telerik comes with the Telerik.Web.UI.DLL for ASP.NET Version 2.0. Manually copy the telerik.web.ui.dll from the ~\telerik\radcontrols for aspnet ajax 3q 2008\bin35 to your bin directory on the website.
then it works!