The error is casused by not removing items from LeftList first.
Try removing items from LeftList before adding them to RightList.
Also, create a temporary list of the items first ... since removing items from LeftList while iterating through the selected items will also cause problems when there is more than one item selected.
The solution looks like this:
Private Sub SwapItems()
Dim items As List(Of RadListBoxItem)
items = New List(Of RadListBoxItem) ' temporary list
For i As Integer = 0 To LeftList.SelectedItems.Count - 1
For Each item As RadListBoxItem In items