What you have described is a common problem with various frameworks that allow customization through third party products. Another example of the same scenario is DotNetNuke and its module system.
As you already pointed out, you will need to deploy the controls assemblies to the GAC, so multiple versions of the same assembly can coexist. However, if another solution is retracted and it uses the same control version, the control will be removed from the GAC (we have tested this with our RadEditor control). In this case, you need to instruct the SharePoint admin to re-deploy your solution in order to restore the missing GAC assemblies (or alternatively, copy them in the GAC manually).
As for ILMerge - it is possible to use it, but indeed it does not work with embedded resources. You seem to have figured out how to use the RadControlsDir instead.
I don't the think the problem you are experiencing is caused by the merging, though. It is probably a CSS problem - MOSS tends to have a lot of CSS styles that are too generic (e.g. affecting all DIVs) and tend to interfere with third-party CSS styles quite often. Can you please try switching to another MasterPage, for example the default.master? Or you can try using a clean page with no MOSS styles. If this fixes the problem, it's definitely a CSS problem.
Let us know how it goes.
the Telerik team