OnUnload() does not seem to be called when exiting the silverlight application in this manner, and we suspect this is the root of this issue. Setting IsVisible to Collapsed and calling UpdateLayout() did not seem to work from Application_Exit(). However, we were able to come up with a solution...
We implemented logic in Application_Exit() to search the root visual's descendants for all RadHtmlPlaceholder controls. We then removed each child HtmlElement in the RadHtmlPlaceholder control's HtmlPresenter as follows:
var htmlChildren = htmlPlaceholder.HtmlPresenter.Children;
Hopefully this solution will help someone else...