I've tried to reproduce the issue, but to no avail - the correct document is returned. The GetRootDocument() method can return null if the Annotation's AnnotationMarkerLayoutBox is null or the RadDocument is not measured and arranged.
Please try to call RadDocument's MeasureAndArrangeInDefaultSize() method instead of UpdateEditorLayout(). If the problem still exists, please try to isolate it in a sample project and send it to us.
Looking forward to your reply.
Check out the Telerik Platform - the only platform that combines a rich set of UI tools with powerful cloud services to develop web, hybrid and native mobile apps.