Test Studio cannot constantly monitor for HTTP errors during the functional test.
Since YSOD screens are regular web pages with a DOM tree you can use a standard verification step to verify them where you expect to appear (i.e. if the page contains HTTP error 404).
I recorded a short video for a demonstration.
Hope this helps.
All the best,
the Telerik team