Thta's what I was guessing too, because I noticed that the vectors sometimes move in a strange way on the screen, mostly when the application is loaded. Then my question is why do you feel the need to make those calculations instead of simple transformations that would support BitmapCached data.
I'm comparing this performance with some functionality that exists in flash map controls like openscales.
http://openscales.org/demo/index.html
The effect here is very clear and simple, no need to even look at the code. They handle a bitmap between zooms, that leaves some blank areas when zooming out but overall it is a nice looking map.
Also i'm comparing with some other silverlight map controls, like azukimap.
http://farm.azukimap.jp/case/miyake/
it does not have smooth zoom but it is still fast for a large number of vector features because of the cache. I used that thing in a project a year ago and the speed was quite nice.
Problem with the flash version is that, well, I hate flash with a great burning passion and with the azukimap that the application is nice, but sometimes horrible to control and debug.
You have a really nice line of products here with great integration and it would be a shame not to give it a full potential. Of course this is self serving too but I would really love to finally see a silverlight map control that has it all. You already started having custom projections, you don't have wms but it probably can be implemented by users by using singleimageproviders and these plus the nice integration sure make me tempted to use it. We already have bought all the controls but mostly for the alphnumeric part, although all our products have maps too.
So. Why do you do those calculatioins? At some point some developer was creating a map in a project and he did a lot of extra calculation to move the vectors to coincide with the raster without thinking that it was easier to move the raster to fit the vector because there was a lot less math to do. Anyway, just my 2 cents.