Yes you have to add our controls manually to the VS toolbox. Currently we do not support automatically adding them. We will consider this for VS 2010 as we have some issues with the automatically adding for VS 2008. Basically you should follow this instructions.
Note that they are for Silverlight but it is all the same just the tab you are using is WPF instead of SL.
Basically you can choose all controls marked with Telerik dll, but you can also chose from them only the one you will use as there are a lot of classes that you won't use at all and will just encumber the toolbox.
the Telerik team