Radmap and mbtiles as a TiledMapSource

[brian]

I have been trying to write a TiledMapSource that reads from the mbtiles format https://github.com/mapbox/mbtiles-spec/blob/master/1.1/spec.md

 

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace RadMapMbtilesDemo
{
    using System.Data.SQLite;
    using System.Diagnostics;
    using System.IO;
    using System.Security.AccessControl;
    using System.Windows.Media.Imaging;

    using Telerik.Windows.Controls.Map;

    public class MbTileSource : TiledMapSource
    {
        private readonly string mbtilefilepath;

        /// <summary>
        /// Initializes a new instance of the MyMapSource class.
        /// </summary>
        public MbTileSource(string mbtilefilepath, int minZoom, int maxZoom)
              : base(minZoom, maxZoom, 256, 256)
        {
            this.mbtilefilepath = mbtilefilepath;
        }

        /// <summary>
        /// Initialize provider.
        /// </summary>
        public override void Initialize()
        {
            // Raise provider initialized event.
            this.RaiseIntializeCompleted();
        }
        /// <summary>
        /// Gets the image URI.
        /// </summary>
        /// <param name="tileLevel">Tile level.</param>
        /// <param name="tilePositionX">Tile X.</param>
        /// <param name="tilePositionY">Tile Y.</param>
        /// <returns>URI of image.</returns>
        protected override Uri GetTile(int tileLevel, int tilePositionX, int tilePositionY)
        {
            return null;
        }
        protected override Stream GetCachedTile(int tileLevel, int tilePositionX, int tilePositionY)
        {
            var zoomLevel = this.ConvertTileToZoomLevel(tileLevel);

            try
            {
                using (var conn = new SQLiteConnection(String.Format("Data Source={0};Version=3;", this.mbtilefilepath)))
                {
                    conn.Open();
                    var sqlquery = String.Format("SELECT tile_data FROM tiles WHERE tile_column = {0} and tile_row = {1} and zoom_level = {2};", tilePositionX, tilePositionY, zoomLevel);
                    Debug.WriteLine(sqlquery);
                    using (var cmd = new SQLiteCommand() { Connection = conn, CommandText = sqlquery })
                    {
                        var reader = cmd.ExecuteReader();
                        if (reader.Read())
                        {
                            var bytes = reader["tile_data"] as byte[];
                            if (bytes != null)
                            {
                                return new MemoryStream(bytes);
                            }
                        }
                    }
                }
            }
            catch
            {
                return null;
            }

            return null;
        }


    }
}

This works and reads the files from the mbtiles (sqlite underneath the mbtiles spec) but the map is requesting the wrong tiles and I cannot see why. My understanding is:

1) Radmap is requesting tiles in a TMS format

2) MbTiles are stored in a TMS format

So the XYZ values passed into GetCachedTile(int tileLevel, int tilePositionX, int tilePositionY) should be the same I use in my sqlite query. Things I have checked:

1) I wondered if my mbtiles files was actually using OSMs style XYZ and tried the code to flip the Y value, this still gives incorrect tile values

2) I checked my mbtile file with https://viswaug.wordpress.com/2011/06/28/mbtilesviewer/ and it works fine

 So I can only assume that Radmap and mbtiles are using a different format or different projection hence the issues lining the two up. Can anyone help?

[brian]

From reading further it appears the MBTile format uses an SRS of OSGEO:41001 which is another name for EPSG:900913 (http://wiki.openstreetmap.org/wiki/EPSG:3857) so I tried changing the TileProvider to use EPSG900913Projection instead of the mercator projection but the tile values being requested still dont appear to be correct

[brian]

Got it working using:

 

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace RadMapMbtilesDemo
{
    using System.Data.SQLite;
    using System.Diagnostics;
    using System.IO;
    using System.Security.AccessControl;
    using System.Windows.Media.Imaging;

    using Telerik.Windows.Controls.Map;

    public class MbTileSource : TiledMapSource
    {
        private readonly string mbtilefilepath;

        /// <summary>
        /// Initializes a new instance of the MyMapSource class.
        /// </summary>
        public MbTileSource(string mbtilefilepath, int minZoom, int maxZoom)
              : base(minZoom, maxZoom, 256, 256)
        {
            this.mbtilefilepath = mbtilefilepath;
        }

        /// <summary>
        /// Initialize provider.
        /// </summary>
        public override void Initialize()
        {
            // Raise provider initialized event.
            this.RaiseIntializeCompleted();
        }
        /// <summary>
        /// Gets the image URI.
        /// </summary>
        /// <param name="tileLevel">Tile level.</param>
        /// <param name="tilePositionX">Tile X.</param>
        /// <param name="tilePositionY">Tile Y.</param>
        /// <returns>URI of image.</returns>
        protected override Uri GetTile(int tileLevel, int tilePositionX, int tilePositionY)
        {
            return null;
        }
        protected override Stream GetCachedTile(int tileLevel, int tilePositionX, int tilePositionY)
        {
            var zoomLevel = this.ConvertTileToZoomLevel(tileLevel);
            var newy = Math.Pow(2, (double) zoomLevel) - tilePositionY - 1;
            try
            {
                using (var conn = new SQLiteConnection(String.Format("Data Source={0};Version=3;", this.mbtilefilepath)))
                {
                    conn.Open();
                    var sqlquery = String.Format("SELECT tile_data FROM tiles WHERE tile_column = {0} and tile_row = {1} and zoom_level = {2};", tilePositionX, newy, zoomLevel);
                    Debug.WriteLine(sqlquery);
                    using (var cmd = new SQLiteCommand() { Connection = conn, CommandText = sqlquery })
                    {
                        var reader = cmd.ExecuteReader();
                        if (reader.Read())
                        {
                            var bytes = reader["tile_data"] as byte[];
                            if (bytes != null)
                            {
                                return new MemoryStream(bytes);
                            }
                        }
                    }
                }
            }
            catch
            {
                return null;
            }

            return null;
        }


    }
}

And then you need to convert your lat lon points when plotting anything on the map to the same projection like:

 

// create new mbtiles provider
this.radMap.Provider = new MbTileProvider("OSMBrightUK0_14_Sept_2015.mbtiles", 1, 14);

// create a converter in the EPSG900913 projection to convert our standard WGS84 lat lon values for display on the map
var conv = new EPSG900913Converter();
var res = conv.ConvertTo(new LocationCollection() { new Location(52, -0.2) });

this.radMap.Center = res[0];
this.radMap.ZoomLevel = 6;

this.visualizationLayer.Items.Add(res[0]);

 

[Petar Mladenov]

Hello Brian,

Thank you for sharing this code with our WPF Developers community.

Regards,
Petar Mladenov
Telerik

[David]

I am trying to use MbTiles as a provider as well, and using the code posted I haven't had much luck. Could you share your TiledProvider, and your TiledMapSource final source?

[David]

I've noticed your file is "OSMBrightUK0_14_Sept_2015.mbtiles", does that mean you have the style encoded into your mbtiles? I got my mbtiles from here - http://osm2vectortiles.org/downloads/. and they don't seem to have a style. I checked my mbtile file with https://viswaug.wordpress.com/2011/06/28/mbtilesviewer/ and it doesn't seem to work, but when hosting it with mapnik by following this tutorial http://osm2vectortiles.org/docs/start/ it seems to work just fine. What am I doing wrong?

[brian]

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace RadMapMbtilesDemo
{
    using Telerik.Windows.Controls.Map;

    public class MbTileProvider : TiledProvider
    {
        /// <summary>
        /// Initializes a new instance of the MyMapProvider class.
        /// </summary>
        public MbTileProvider(string mbtilefilepath, int minZoom = 1, int maxZoom = 6)
             : base()
       {
            var source = new MbTileSource(mbtilefilepath, minZoom, maxZoom);
            this.MapSources.Add(source.UniqueId, source);
        }
        /// <summary>
        /// Returns the SpatialReference for the map provider.
        /// </summary>
        public override ISpatialReference SpatialReference => new EPSG900913Projection();
    }
}

[brian]

I generated by tiles from OSM data for the UK using https://www.mapbox.com/tilemill/ try this tutorial

https://www.mapbox.com/tilemill/docs/guides/osm-bright-ubuntu-quickstart/

I think there is a windows tutorial as well but Id recommend setting up and ubuntu VM as in my experience Ubuntu renders the tiles much much faster (just so you know rendering the UK down to zoom level 14 took over 24 hours and produced a 3GB mbtile file)

What you have downloaded are VECTOR mbtiles not BITMAP mbtiles, in your file each tile is the actual vector data for all the features, in mine it is a bitmap image of the rendered data.

Radmap does not support vector tiles as they are very new and will require a lot more code than bitmap tiles to get working

[David]

Thanks Brian, that was very helpful information!

[David]

I'm trying to avoid using tilemill, since I need to render the entire planet from levels 0-17 offline. That would take a very long time to download. Do you have any suggestions on where to look to figure out how to render vector tiles, or another way to render the world with radmap, offline, without a tile server?

[brian]

Yes if you wanted the entire world down to zoom 17 as bitmap tiles it would take up 2.2TB of disk space (http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=-184.305688,-78.21948,213.652321,79.163911).

The work involved to code somethign that renders vector tiles with a full streetmap style would be enourmous and the only open platforms I'm aware of that can do it at all are listed here:

https://github.com/mapbox/awesome-vector-tiles

All of which essentially replace radmap. Anything that took the vector tiles, applied a style and produced bitmaps for radmap to use would essentially be a tile server.

Sorry not good news I know but in my experience when you need a large set of data offline the only sensible option is to use a tile server