I have managed to reproduce the described issue. The up and down buttons in the NumericUpDown control are native WPF RepeatButtons and it appears that this is framework behavior as if you test the same scenario with a single RepeatButton you will observe the same behavior. It seems that the continuous touch is handled by the framework in order to perform scroll operation if you swipe up or down afterwards. In this case, I'm afraid, we cannot suggest you any workaround for that behavior.
If you have any other questions let us know.
Check out Telerik Analytics, the service which allows developers to discover app usage patterns, analyze user data, log exceptions, solve problems and profile application performance at run time. Watch the videos
and start improving your app based on facts, not hunches.