VirtualQueryableCollectionView is designed to provide a number of items for a virtualized ItemsControl, as soon as it requests them. In both of the listed scenarios, ListBox tries to operate with certain items. If this data is not present, VQCV tries to retrieve it. Control over the requested items is handled by the ItemsControl. Trying to interfere with this logic may cause erroneous behavior.
Check out Telerik Analytics, the service which allows developers to discover app usage patterns, analyze user data, log exceptions, solve problems and profile application performance at run time. Watch the videos
and start improving your app based on facts, not hunches.