Test Studio provides the UnexpectedDialogAction feature. Currently it's only available when running your tests as part of a Test List. Please read this documentation article
on "Avoiding Dialog Handling Problems with the UnexpectedDialogAction Feature" to see how to use it.
If you want to use this feature when running tests from quick execution, you have to do it in coded step with code like this:
Settings.Current.UnexpectedDialogAction = UnexpectedDialogAction.HandleAndContinue;
You can add this coded step at the beginning of each test, in which you expect to appear an unexpected dialog.
Unfortunately, this feature will not work only for a native Windows dialogs. So, if it doesn't work for you please send us a screenshot of what your popups look like in order to assist you further in this issue.
Hope to here from you soon!
All the best,
the Telerik team