I am not quite sure what you are trying to achieve, but generally the approach you are using is not recommended and may cause different issues.
What I mean is that a Table element, by design, cannot be the first or last block in a document and there cannot be adjacent tables in a document. Moreover, the last block in a table cell has to be a paragraph (not a table). Thus, when inserting the inner table, the
by design a mechanism prevents it from being the last block. You should probably make sure to insert empty paragraphs around such tables to prevent future issues:
Moreover, I am not sure how you expect the table to look - do you expect for both borders to overlap or to have a little space between them? Generally, inserting an inner table should look like in the attached image and you can verify that be nesting tables in a RadRichTextBox using the UI.
I hope the provided information helps! Let us know if you have other comments or questions.