I'm not sure what the version you upgraded from is, but the RadMaskedTextBox has been removed from the Insert Hyperlink dialog in 2011. In fact, the control was just recently removed from the Silverlight suite altogether, so if you are using it somewhere else in your application you would need to migrate to RadMaskedInput
In the place of the masked input in the dialog there is now a TextBox control, so you can attach your context menu to it. Since it's not the only TextBox in the dialog you might want to try a different approach for determining which control exactly you need, for example:
TextBox txtText = (dialog.Content
I hope this is useful.
Check out Telerik Analytics, the service which allows developers to discover app usage patterns, analyze user data, log exceptions, solve problems and profile application performance at run time. Watch the videos
and start improving your app based on facts, not hunches.