ShapeFile taking too long to load

[vishal]

Hi There

I am reading multiple shapefiles in a loop and for each shapefiles I'm creating VirtualizationLayer object and then I'm adding each framework element present in that shapefile to VirtualizationLayer , as shown in below code.

layer = new VirtualizationLayer();
layer.ShapeFill = new MapShapeFill();

foreach (FrameworkElement shape in shapes)
{
         layer.Items.Add(shape);
}

1. The issue is few shapefiles are big files and have some 60K+ framework elements and are loading too late.Can't we add shapefile directly to VirtualizationLayer object or any workaround to reduce this time.Is this looping required?Can't we add shapefile directly to layer.In an another open source application we are loading same shapefile without looping the way we are doing in RadMap.

2. For few shapefiles , I have to set shapefill property differently, i.e. ShapeFile.Fill for some shapefiles has to be transparent and for few it has to be filled with color value,similarly ShapeFile.Stroke has to be set differently based on shapefile.The types of shapefiles i have are MapPolyLine etc, The question here is how to know the shapetype of shapefiles, i.e. MapPolyLine. Pointed into Issue no:2 in screenshot.

Cheers
Vishal

[Pavel R. Pavlov]

Hi Vishal,

Let me get straight to your issues. With such number of shapes it is expected that the performance of the RadMap will drop. If you need to to load such number of shapes into the map it will be better to use the virtualization feature of the layer. It will allow you to load the shapes in chunks depending on the user action. This will increase the overall performance of the control.

As for your second issue, you can take a look at our GraphColorizer. It allows different colors to be applied to different shapes on the map. Please evaluate it and let me know if you need any further assistance.

Regards,
Pavel R. Pavlov
Telerik

 

 

[vishal]

Hi Pavel 

Thanks for the direction.Appreciate if you can share any link for WPF example or can send me any WPF project using Visualization feature you mentioned.

Also if you notice that I'm using VirtualizationLayer instead of InformationLayer to gain perfromance, as below

layer = new VirtualizationLayer();
layer.ShapeFill = new MapShapeFill();
foreach (FrameworkElement shape in shapes)
{
             layer.Items.Add(shape);
}

So what is difference between VirtualizationLayer and Virtualization Feature of the Layer, as you suggested.

Thanks In Anticipation
Vishal

[Martin Ivanov]

Hi Vishal,

The RadMap control supports 2 visualization engine packages. The first one consists of 3 layers for visualization of the map objects: InformationalLayer, DynamicLayer and VirtualizationLayer. Note that this is also the old package. RadMap provides another (new) package that have a single layer which replaces all 3 layers from the old one.

I recommend you to take a look at the UI Virtualization feature of the new Visualization layer as Pavel suggested and see if it will work for you. I also attached a simple project demonstrating the setup of visualization layer with virtualization, based on the virtualization help article. I hope it will be a good starting point for your requirement.

In addition you can take a look at the Hotel example that demonstrates loading of multiple shape files, and also the UI Virtualization demo. The examples are in Silverlight but they share most of its code with the WPF version.

Regards,
Martin
Telerik

 

 

[vishal]

Hi There

Thanks for the WPF example. it works wonder.

However there is performance degrade as like below;
a. Responsiveness is slow.
b. Zooom In-Zoom Out is slow
 
I have attached the project, kindly have look. The performance was slightly better while loading shape-files manually.
The heaviest shapefile is : RoadResidential ,this file have 60K elements,rest don't have many framework elements.

1. The Silverlight demo you shared [http://demos.telerik.com/silverlight/#Map/DynamicLayer], is using BingMap not shapefile,so I'm not sure whether it will response in same way if I load shapefile. Also can we use NokiaHERE Maps?

2. From Hotel example, i used the code to load multiple shapefiles asynchronously.

Link to download life: https://www.dropbox.com/s/nahnbxtl79dfm60/Async_ShapeFile_RadMapExample.rar?dl=0

Also we have purchased the Telerik License now, so i might use different email id for login.

Thanks In Anticipation
 

[Martin Ivanov]

Hello Vishal,

I will need some more time to investigate your project and shape files and I will contact you as soon as I have more information on the matter. 

Regards,
Martin
Telerik

 

 

[Martin Ivanov]

Hi Vishal,

Let me start with the set up of your project. In order for the UI virtualization of the VisualizationLayer's shape to be enabled you will need to define a ZoomLevelGridList and use a MapShapeDataVirtualizationSource, then set it as a VirtualizationSource of the VisualizationLayer.

<telerik:VisualizationLayer.VirtualizationSource>
    <telerik:MapShapeDataVirtualizationSource x:Name="mapShapeDataVirtualizationSource" ClearCache="False" >
        <telerik:MapShapeDataVirtualizationSource.Reader>
                 .....
        </telerik:MapShapeDataVirtualizationSource.Reader>
    </telerik:MapShapeDataVirtualizationSource>
</telerik:VisualizationLayer.VirtualizationSource>

The virtualization source does not allow multiple shape files by default. If you want to load them you should set the ClearCache property of the source to False.

<telerik:MapShapeDataVirtualizationSource x:Name="mapShapeDataVirtualizationSource" ClearCache="False" >
    <telerik:MapShapeDataVirtualizationSource.Reader>
            <telerik:AsyncShapeFileReader x:Name="mapShapeDataReader">
                <telerik:AsyncReaderSource Source="/WpfApplication11;component/Resources/coast.shp" />
                <telerik:AsyncReaderSource Source="/WpfApplication11;component/Resources/BuildupGeneral.shp" />
                <telerik:AsyncReaderSource Source="/WpfApplication11;component/Resources/Motorway2014.shp" />
                <telerik:AsyncReaderSource Source="/WpfApplication11;component/Resources/MountsPeaks.shp" />
                <telerik:AsyncReaderSource Source="/WpfApplication11;component/Resources/Rabout.shp" />
                <telerik:AsyncReaderSource Source="/WpfApplication11;component/Resources/RoadResidential.shp" />
            </telerik:AsyncShapeFileReader>
    </telerik:MapShapeDataVirtualizationSource.Reader>
</telerik:MapShapeDataVirtualizationSource>

As for the slow performance - the loaded shape files are quite big and it takes a time to create map shape data objects from them and perform some initial calculations. The files contain very detailed polygons with large number of points and the WPF rendering engine is very sensitive when it comes to the number of points it should render. When a geometry is created for the first time (when the shape appears in the view port for the current zoom level) it takes time to prepare it. RadMaps' engine performs some optimizations for the number of points but this optimization makes sense for the zoom levels when low shape details are visible. To increase the performance of the control you can try to simplify the initial shape files, if this is possible. 

I can also give you several tips that concerns the performance of RadMap. However, probably they won't significantly improve the performance in your scenario where the shape files contain 60k+ shapes with big amount of points.

• RadMap is using Bitmap caching which is turned on by default in order to speed up rendering of the point-type objects. But for the layer which show map shape data objects the bitmap cache should be turned off. To do so you can set the VisualizationLayer's UseBitmapCache property to False.
  

  <telerik:VisualizationLayer UseBitmapCache="False">

• You can also try different CellHeight and CellHeight values for the different zoom level grids. Specifying the size of cell for the zoom level affects to a number of requests to a Virtualization Source which is depended on the viewport size of the map.
• You can set the Resolution property of the of the MapShapeDataVirtualizationSource. The property gets/sets the minimum pixels size of the drawn object. If the size of the object is less then this value then the object it is not drawn in the VisualizationLayer

Note that RadMap does not provide NokiaHere Maps provider. However, if the HERE API allows, you should be able to use its services and wrap them into custom provider.

Regards,
Martin
Telerik

 

 

[Sanjay]

Hi Martin,

Thanks for resolution.I'm replying via our paid account.

I am trying to create VirtualizatioLayer at runtime, setting AsynShapeFile source to shapefile etc. Below is the code which doesn't work for line 

vLayer.VirtualizationSource = map;

where map is MapShapeDataVirtualizationSource object. Full code is below.
  

//Manually creating virtualization layer

           VirtualizationLayer vLayer = new VirtualizationLayer()

           {

               ShapeFill = new MapShapeFill()

               {

                   Stroke = new SolidColorBrush(Colors.Blue),

                   StrokeThickness = 2,

                   Fill = Brushes.Yellow

               }

           };

 

           //Set ZoomLevel for this Layer

           vLayer.ZoomLevelGridList.Add(new ZoomLevelGrid(8));

           vLayer.ZoomLevelGridList.Add(new ZoomLevelGrid(10));

           vLayer.ZoomLevelGridList.Add(new ZoomLevelGrid(13));

 

           //Create a AsyncShapeFileReader object and load shapefile

           MapShapeDataVirtualizationSource map = new MapShapeDataVirtualizationSource();

           //AsyncShapeFileReader reader = map.Reader as AsyncShapeFileReader;

           AsyncShapeFileReader reader = new AsyncShapeFileReader();

           reader.Source = new Uri("/TelerikRadMaps;component/ShapeFiles/coast.shp", UriKind.Relative);

           map.Reader = reader;

           map.ReadAsync();

           map.ClearCache = false;

           vLayer.VirtualizationSource = map;//But MapShapeDataVirtualizationSource class implements IMapVirtualizationSource.

           vLayer.UseBitmapCache = false;

           this.radMap.Items.Add(vLayer);


XAML

<telerik:RadMap x:Name="radMap"  Width="{Binding}" Height="{Binding}"  Center="-19.981681155272,57.3084997231693" >
           <telerik:RadMap.Provider>
               <telerik:EmptyProvider ></telerik:EmptyProvider>
           </telerik:RadMap.Provider>
       </telerik:RadMap>

Download Sample Project

1. Can you point out what wrong I'm doing.
2. Setting useBitmapCache to VirtualizationLayer at runtime gives compile time error.

vLayer.UseBitmapCache = false;

3. What will happen if I set ZoomLevelGridList to have low level zoom value i.e. less than 8,in terms of visibility of data and rendering of shapefiles.

vLayer.ZoomLevelGridList.Add(new ZoomLevelGrid(5));

4.Each shapefiles are different because each have different type of shape data, so I'm loading each shapefiles in individual layers.Is this approach right.

Thanks in Anticipation

[Martin Ivanov]

Hi Vishal,

Let me get straight to your questions:

• I am trying to create VirtualizatioLayer at runtime, setting AsynShapeFile source to shapefile etc. Below is the code which doesn't work for line 
  

  vLayer.VirtualizationSource = map;

  Can you point out what wrong I'm doing.
  Setting the VirtualizationSource does not work in your project because you are creating a VirtualizationLayer instead of VisualizationLayer. The MapShapeDataVirtualizationSource is part of the VisualizationLayer and it can be used only with its VirtualizationSource property. Note that in my last reply I described the ViSualizationLayer of RadMap and its virtualization feature.
  
  Since the MapShapeDataVirtualizationSource implements the IMapItemsVirtualizationSource, not the IMapVirtualizationSource interface. a compile time error occurs.
  
• Setting useBitmapCache to VirtualizationLayer at runtime gives compile time error.
  For your question about setting the UseBitmapChache property, keep in mind this property is exposed only by the VisualizationLayer. The VirtualizationLayer does not provide such functionality.
• What will happen if I set ZoomLevelGridList to have low level zoom value i.e. less than 8,in terms of visibility of data and rendering of shapefiles.
  The ZoomLevelGridList divides the map on zoom regions. When you enter one of those regions an items request is executed through the virtualization source and the get items are loaded in the layer. In other words, the bigger is the range between the zoom levels, the less requests will be send to the source. 
• Each shapefiles are different because each have different type of shape data, so I'm loading each shapefiles in individual layers.Is this approach right.
  You can do that, but this could decrease the performance of the map control, since when you add a new layer, a new visual element is added and maintained in the visual tree. 

Please let me know if this information helps.

Regards,
Martin
Telerik

 

 

[Sanjay]

Hi Martin 

Thanks for Reply.Have few follow up questions

A. ZoomLevelGridList : 
1. For given example below.

<telerik:VisualizationLayer.ZoomLevelGridList>
           <telerik:ZoomLevelGrid MinZoom="8"  CellHeight="100" CellWidth="100" />
          <telerik:ZoomLevelGrid MinZoom="14"  CellHeight="200" CellWidth="200"/>
          <telerik:ZoomLevelGrid MinZoom="20"  CellHeight="300" CellWidth="300"/>
</telerik:VisualizationLayer.ZoomLevelGridList>

1.1 What is the region, is this the visible part of the map.How many regions would be created for above code.
1.2 I understand that bigger the range between zoom level, the less request would be made which implies better performance.So if I have 5 shapefiles in one layer, the zoomlevel values would be applied to all shapefiles,which mean request would be made for all shapefiles as the zoomlevel value is met.Can i have different zoomlevel for different shapefiles in one layer to avoid sending request for all shapefiles.
1.3 When the request is made, does it send request for items only of the visible part of map (i.e region) , or whole map.I'm afraid in my case its sending request for all shapefiles.
1.4 How can i show current zoomlevel on map.
1.5 How the value of CellWidth /CellHeight should be set, i.e should we keep it smaller for low zoom level and increase it as zoom level increases, or opposite way.Is CellWidth /CellHeight defines the regions?
1.6 In above xaml code, i have zoom level at 8 , 14 and 20, so does it mean that request will be only send at zoom level 8 , 14 and 20 , and no request would be send for any value other than these? How many zoom level should we have for performance gain.

Thanks in Anticipation

[Martin Ivanov]

Hi Sanjay,

I will need some time to take a look at your questions. I will contact you again with more information in several hours.

Regards,
Martin
Telerik

 

 

[Martin Ivanov]

Hello Sanjay ,

Here are the answers to your questions:

1. What is the region, is this the visible part of the map.How many regions would be created for above code.
   The zoom level grids divide the diagram into logical regions that fit into a zoom range. For example, for your current definition with MinZoom(s) 8, 14 and 20 the map will create three regions - one for the zoom levels between 8 and 13, another for those between 14 and 19 and third for 20. If you zoom only between 14 and 19, just the regions for this range will be created and the others won't exist. Once the zoom is out of this range, another region will be activated (for another zoom range).
2.  I understand that bigger the range between zoom level, the less request would be made which implies better performance.
   This is not necessary. Note that the overall performance of the map consists of three main factors - initial load performance, zoom performance and pan performance. There is no general rule how to keep optimal performance. The type (size, points, etc.) of the visualized shapes and the cell size is also considered.
   
   So if I have 5 shapefiles in one layer, the zoomlevel values would be applied to all shapefiles,which mean request would be made for all shapefiles as the zoomlevel value is met.Can i have different zoomlevel for different shapefiles in one layer to avoid sending request for all shapefiles.
   The map does not support different zoom levels for the different shapes, however, you should be able to implement logic that handles such scenario with a custom map virtualization source which implements the IMapVirtualizationSource interface.
   
   What you can do is basically to initially load the different shapes in different collections in your implementation of the source. When the MapItemsRequest() method is called to check the zoom level in its code. If the zoom level matches the desired level for the shape you can call the CompleteItemsRequest() method with the shapes you want to display. Here is a very simplified example for this:
   

   public class MyVirtualizationSource : IMapItemsVirtualizationSource
   {
       private List<MyShape> shapeFile1Shapes;
       private List<MyShape> shapeFile2Shapes;
   
       public MyVirtualizationSource()
       {
           //this.shapeFile1Shapes = GetData(shapeFile1Path);
           //this.shapeFile2Shapes = GetData(shapeFile2Path);
       }
   
       public void MapItemsRequest(object sender, MapItemsRequestEventArgs eventArgs)
       {
           // if the eventArgs.MinZoom corresponds to the zoom level I want to apply for the first shape file
           // {
           //      eventArgs.CompleteItemsRequest(shapeFile1Shapes);
           // }
       }
   }

   
   You can take a look at the Items Virtualization for more information on how  to implement custom virtualization source.
3. When the request is made, does it send request for items only of the visible part of map (i.e region) , or whole map.I'm afraid in my case its sending request for all shapefiles.
   Requests are made for every cell which intersects with the current viewport. Cell's size and positions are calculated when the zoom level grid is created. When the OnMapItemRequests() is called the shapes that are read from your shape files and are located in the current cell, will be added to the layer. However, you can control this process with implementing custom map virtualization source in which you can filter the shapes before add them to the layer.
4. How can i show current zoomlevel on map.
   You can show the current ZoomLevel on the map through the MapNavigation control. This control is used for navigation through the map and it contains a slider that controls the zoom level. Another approach to show the level is to create an UI element to visualize the zoom level and bind its content to the ZoomLevel property of RadMap.
5. How the value of CellWidth /CellHeight should be set, i.e should we keep it smaller for low zoom level and increase it as zoom level increases, or opposite way.Is CellWidth /CellHeight defines the regions?
   This also strongly depends on the specific scenario and the type of the displayed shapes. Basically, the bigger the cell size is, the more outer points will be created, which means there will be less requests and less calculations. The other case is, the smaller the cell is, the less outer points will be created, which you might not need to see, and more requests and calculations will be made. I recommend you to test different cell sizes in your scenario and see which one fits best.
6. In above xaml code, i have zoom level at 8 , 14 and 20, so does it mean that request will be only send at zoom level 8 , 14 and 20 , and no request would be send for any value other than these? How many zoom level should we have for performance gain.
   Generally it is good to limit the zoom levels in which you send requests. What I mean is that you can adjust them so that they load only the shapes which are necessary for a specific zoom level. For example if you have very small shapes that can be recognized only with a big zoom level, there is no point to load them in a small zoom level.

In summary, when it comes to performance you can customize the map so that it fits best in you case, based on the information that you display, the shapes and their points, and the expected user interaction (pan, zoom)

As a side note, there are CLR binding errors when you run the map control in debug mode (F5) which take time to be displayed in the Output of Visual Studio and decrease the control's performance. You can test the actual performance of the map by running your project without debugging (Release mode) with Ctrl+F5

I hope this helps.

Regards,
Martin
Telerik

 

[Sanjay]

HI Martin 

Thanks for detailed reply and patience.We are trying out your suggestions in our project.

Have 2 follow up questions.

1. Do RadMap engine reads .shx files? Shx files keeps the index of elements in shape file and act as bridge between .shp and .dbf files.I have not seen RadMap using .shx files. ESRI shape file reader use .shx file and rendering , zooming and panning is very fast in ESRI reader.Also i have noticed in few demo examples that RadMap do not read .dbf file as datasource object,it just reads .shp file.So how does RadMap engine use .shx file.

2. Do RadMap support raster(bitmap) files directly and not vector file?. So when we load shapefile which is vector files, do it convert it to raster file and read it.This might be one reason for ESRI reader performance.

Cheers

[Martin Ivanov]

Hello Sanjay,

The RadMap control displays shp-file or shp/dbf pair without using the shx-file. As for the examples that do not read the .dbf files, I am not sure which are those, but keep in mind that the .dbf file it is not necessary for the reader to get the shapes from a shape file. Also, if the name of the .dbf file matches the .shp file's name, the reader automatically will get the dbf file without setting it as a DataSource.

About your second question, when you load the shape file, the map control converts its data into map shape objects and it renders the shapes as Path elements. Since the WPF framework uses vector based rendering, the paths will be drawn as vectors. If you want to use raster images in the map you can take a look at the UriImageProvider help article or you can try to place image elements with the ItemTemplate of the map's layer and populate it with objects that contains image tiles of the map's regions.

Regards,
Martin
Telerik

 

 

[Matthes]

[quote]Martin said:

1. When the request is made, does it send request for items only of the visible part of map (i.e region) , or whole map.I'm afraid in my case its sending request for all shapefiles.
   Requests are made for every cell which intersects with the current viewport. Cell's size and positions are calculated when the zoom level grid is created. When the OnMapItemRequests() is called the shapes that are read from your shape files and are located in the current cell, will be added to the layer. However, you can control this process with implementing custom map virtualization source in which you can filter the shapes before add them to the layer.
2. [/quote]

Hello,

this thread was also very helpfull for me. However I have a question regarding the demo. In the stores demo, the DataServiceGetStoresCompleted() method explicitly filters for items that are in the cell. Is this required, because from your answer it seems that the shapes are filtered automatically ? Why should I filter it then ?

 

[Martin Ivanov]

Hello Matthes,

I slightly mislead you with my statement, please excuse me for that. We provide a default VirtualizationSource ("MapShapeDataVirtualizationSource") which internally filters the shapes based on the zoom level and the generated cell's bounds. If you create custom virtualization source you will need to filter the data on your own. The stores example demonstrates how you can implement custom virtualization source for the map's VisualizationLayer and filter your data manually based on specific conditions. This is why the DataServiceGetStoresCompleted() method filters the items explicitly.

Please let me know if you have further questions.

Regards,
Martin
Telerik

[Matthes]

Thank you! Just one additional question:

When binding the layers Itemssource to a collection of PolygonData (or by providing them in the Virtualizationsource) the layer automatically creates the Polygon Shapes on the map. Is it possible to display additional non-polygon items on the same layer via binding ? In my scenario I have a collection of Polygons and in their center I want to place a label pin. Center-location, polygondata and label string are properties of the viewmodels in the collection.

Now, if I would use a DataTemplate for this,I do not know how to display a polygon.  At the moment I use two layers to achieve this. One for the polygons and one for the center label pins. I thought it might be good to combine this to one layer for the entities of the type.

[Martin Ivanov]

Hello Matthes,

Yes you can use PolygonData and other elements in a single layer - you can even place FrameworkElements in the Items collection of the layer along with the MapShapeData objects. However, in your data binding scenario you can use the ItemsTemplate property of the layer define a template for the label objects. For your convenience I prepared a sample project demonstrating this approach. Please take a look at it and let me know if this is what you are looking for.

Regards,
Martin
Telerik

[Matthes]

Hey Martin,

thanks alot!