Hello!
I have a very poor performance when EnableRowVirtualization="True" and no rowHeight is set.
If I set e.g. Row.Height = 90 then everything is okay.
It sound like you may understand why its happening, but I'll just share my experience. We had the gridview in a row of height "Auto" so the gridview thought it was much larger and therefore took up a huge amount of memory (it thinks the whole thing is on screen). We put the gridview into a row with height "*" and the performance problem was fixed because row virtualization was able to actually work.