The databinding does not invoke the property setter on every letter typed when a mask is for instance like "A5". In this case the value is updated on every 5th symbol entered. This is how the control is designed and this behavior is expected. Using mask starting with small letter like "a5" however will invoke the setter on each input.
As for the question about the dynamic change of the mask the scenario is complex and it is not supported out of the box as there are a lot of events and methods called after value is entered and in this specific scenario the Mask is being changed in a moment when not all calculations are made. In order to workaround this, a Dispatcher can be used after attaching to the ValueChanged event. This is approach is shown in the attached sample project.
private void tbClid_ValueChanged(object sender, Telerik.Windows.RadRoutedEventArgs e)
var value = (sender as RadMaskedTextInput).Value;
else if (value.ToString().StartsWith("5"))
private void ChangeMaskDispatched(string newMASK)
Dispatcher.BeginInvoke(new Action(() => tbClid.Mask = newMASK));