I think your problem will be fixed when you set the NewLineBr property to false in the editor's configuration file (in the case of Wikis this is ListConfigFile.xml). Here is a link with more information about the property: http://www.telerik.com/help/aspnet/editor/UsingTheNewLineBrProperty.html
Basically, when you set it to false, the editor will insert new paragraphs (<p>) each time you press the Enter key. If you want to insert just a break (<br/>), you will need to press Shift+Enter.
the Telerik team