Skip to content

Streamable and read optimized file archive for hosting map tiles at global scale on a cloud object storage

License

Notifications You must be signed in to change notification settings

mactrem/com-tiles

Repository files navigation

Cloud Optimized Map Tiles (COMTiles)

Based on the ideas of Cloud Optimized GeoTIFF and extended for the usage of raster and in particular vector map tilesets. COMTiles are a streamable and read optimized file archive for hosting map tiles at global scale on a cloud object storage. Currently most geospatial data formats (like MBTiles, Shapefiles, KML, ...) were developed only with the POSIX filesystem access in mind. COMTiles in contrast is designed to be hosted on a cloud object storage like AWS S3 or Azure Blob Storage without the need for a database or tile server on the backend side. The map tiles can be accessed directly from a browser via HTTP GET range requests. Via COMTiles an object storage can be used as a spatial database regarding the visualization of map tiles. COMTiles aims to be a MBTiles database for the cloud. The main focus of COMTiles is to significantly reduce costs and simplify the hosting of large raster and vector tilesets at global scale in the cloud, as no dedicated tile backend is required.

For the basic concepts of COMTiles have a look at the lightning talk at the OGC Cloud-Native Geospatial Outreach Event. For a detailed description of the format have a look at the specification.

📝 For a more in-depth exploration of further optimizing COMTiles, please refer to the recent published paper called COMTILES: A CASE STUDY OF A CLOUD OPTIMIZED TILE ARCHIVE FORMAT FOR DEPLOYING PLANET-SCALE TILSETS IN THE CLOUD. The POC discussed in the paper is not yet integrated into this repository but can be found in the com-tiles-evaluation repository.

Tools

  • @com-tiles/mbtiles-converter: To convert a MBTiles database to a COMTiles archive the @com-tiles/mbtiles-converter command line tool can be used.
  • @com-tiles/maplibre-provider: For displaying a COMTiles archive hosted on an object storage directly in the browser based on the MapLibre map framework the @com-tiles/maplibre-providerpackage can be used.
  • @com-tiles/provider: For the integration in other web mapping libraries like OpenLayer or Leaflet the @com-tiles/provider package can be used.

Demo

In the following examples the europe tileset is based on a MBTiles database from MapTiler and converted to a COMTiles archive with the @comt/mbtiles-converter. The index for zoom level 0 to 10 is part of the initial fetch when the page is loaded. For zoom 11 to 14 the index fragments with a size of 37kb are lazy loaded. For exploring a city like munich and surroundings only one additional HTTP request per zoom level is needed.

Europe tileset hosted on a AWS S3 standard storage with a disabled browser cache. COMTiles YouTube video

Europe tileset hosted on a local MinIO storage with a disabled browser cache. COMTiles YouTube video

In the following example the tile requests are batched resulting in only 2 to 3 tile requests for a specific zoom level compared to up to 15 requests with the standard approach (tile by tile) on an HD display. COMTiles YouTube video

In the following example a planet scale tileset is hosted on an AWS S3 and accessed from an AWS CloudFront CDN.
The index for zoom 0 to 7 is downloaded with the first initial fetch.
For zoom 8 to 14 one pre-fetch per zoom level for the specific index fragment is needed, but this only takes about 30 milliseconds.
COMTiles YouTube video

COMTiles archive vs directly hosting the map tiles

Advantages of COMTiles

  • Loading up over 350 million tiles to a cloud storage for a planet scale vector tiles dataset is significantly more expensive because every tile upload has to be paid separately
  • Uploading a single large file is noticeably faster compared to uploading millions of individual files
  • A archive format which combines the metadata and data into one file is more easy to handle
  • Saves storage because a (raster) tileset at global scale can have many repeated tiles (about 70 percent of the earth is water)
  • Tile requests can be batched to significantly reduce the number of requests by up to 90%

Advantages of directly hosting the map tiles

  • For a tileset at global scale from zoom level 8 on at least one prefetch for a index fragment is needed for every zoom level up to 14 -> this is bearly or not at all noticeable for the user regarding the user experience and workflow

Similar formats

Cloud Optimized GeoTiff

A Cloud Optimized GeoTIFF (COG) is a regular GeoTIFF file with an internal organization that let clients ask for just the portions of a file that they need via HTTP GET range requests. A COG is limited to raster data.

FlatGeobuf

A performant binary encoding for geographic data based on flatbuffers that can hold a collection of Simple Features including circular interpolations as defined by SQL-MM Part 3.

PMTiles

PMTiles is a single-file archive format for directories of map tiles. The index of a PMTiles archive is structured in directories compared to the fragment based approach of a COMTiles archive.

Use Cases

  • Displaying map tiles directly in the browser via a web mapping framework like MapLibreGL JS
  • Downloading map extracts for the offline usage in apps
  • Downloading map extracts for the hosting on a dedicated on-premise infrastructure

Repository structure

References

About

Streamable and read optimized file archive for hosting map tiles at global scale on a cloud object storage

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published