Go Tile Server

A tile server in GO

This article is a code walk through on the implementation of a tile server in Go. It describes the Go-Mapnik library developed by Fawick. It is a great library and the code is very well written.

Go is a system language developed for concurrency and is a good fit for web servers and geo-servers. The implementation uses go routines and channels, two features at the core of Go.

Tile Server General Description

When accessing an online map, we can see the browser loading a grid of many images, called tiles. Zooming will load a new set of images, corresponding to the required location. Behind the scene, a server is providing the tiles from the coordinates and zoom asked by the client.

Geographic data is stored in a database, the process to create images from data is called rasterization. Rastering the tiles take some time and performance can be enhanced by caching the tiles once they have been created.

For online map, the client send an http GET request to the server at URL like http://tile.server/{z}/{x}/{y} where z is zoom and x, y are coordinates.

Go routines and channels

A go routine is a thread of execution. A function called inside a go routine will be non-blocking and asynchronous, the main execution thread does not wait for the return of that function but executes the next instruction. In go, to execute a function f in a go routine, it just has to be preceded by the keyword go.

go f()

A channel is a pipe to send data between two asynchronous go routines. It allows to synchronize two go routines, for example by blocking the execution while waiting for a value to be received.

Code Walk Trough

As seen below, a tile server is a web server. It means it communicates to clients through http requests sent to a URL. This behaviour is implemented with the http package of Go standard library. It’s a lightweight web server, the routes are created with the ListenAndServe function, that takes the route string and a handler.

http.ListenAndServe(":8080", t)

The handler t is the tile server. It is a Struct, which is a collection of variables called fields. It has two fields:

• a Tile database: custom type TileDB
• a Layer Multiplexer: custom type LayerMultiplex

type TileServer struct {
  m         *TileDb
  lmp       *LayerMultiplex
}

Tile Database

The tile database is a SQLite database. It’s a self-contained database, it means it doesn’t need any database server or installation, the sqlite file of the database is enough to perform SQL requests. A specification has been made to store tiles in SQLite databases, they are called MBTiles. For example, the database must have the following tables:

• metadata (name text, value text)
• tiles (zoom_level integer, tile_column integer, tile_row integer, tile_data blob)

Full specification here.

TileDB is a Struct with many fields, among them:

• RequestChan: a channel receiving a TileFetchRequest Struct
• InsertChan: a channel receiving a TileFetchResponse

type TileDb struct {
  ...
  requestChan chan TileFetchRequest
  insertChan  chan TileFetchResult
  ...
}

Layer Multiplexer

This element deals with generating the tiles when they cannot be retrived from the Tile Database (cache). It uses Mapnik Library and requires two input: a stylesheet (xml file) and geographic data (can be shapefile, database, …)

Proceedings

The handler has the following code:

func (t *TileServer) ServeTileRequest(w http.ResponseWriter, r *http.Request, tc TileCoord) {
  //un-buffered channel: blocking
  ch := make(chan TileFetchResult)

  tr := TileFetchRequest{tc, ch}
  t.m.RequestQueue() <- tr

  //blocked here waiting for result
  result := <-ch
  needsInsert := false

  if result.BlobPNG == nil {
    // Tile was not provided by DB, so submit the tile request to the renderer
    t.lmp.SubmitRequest(tr)
    result = <-ch
    if result.BlobPNG == nil {
      // The tile could not be rendered, now we need to bail out.
      http.NotFound(w, r)
      return
    }
    needsInsert = true
  }

  w.Header().Set("Content-Type", "image/png")
  _, err := w.Write(result.BlobPNG)
  if err != nil {
    log.Println(err)
  }
  if needsInsert {
    t.m.InsertQueue() <- result // insert newly rendered tile into cache db
  }
}

We now walk through that code:

A request is sent to the server, it contains:

• longitude
• latitude
• zoom

An unbuffered channel “ch” is created, receiving a TileFetchResult Struct. This Struct is composed by coordinates and the tile image object.

type TileFetchResult struct {
  Coord   TileCoord
  BlobPNG []byte
}
ch := make(chan TileFetchResult)

A TileFetchRequest Struct is created, tr, composed by the coordidates of the requested tile and the previous channel ch.

type TileFetchRequest struct {
  Coord   TileCoord
  OutChan chan<- TileFetchResult
}

tr := TileFetchRequest{tc, ch} 

The following line is data sent through a channel:

t.m.RequestQueue() <- tr 

t.m.RequestQueue() gives acces to a field of the TileServer Multiplexer that can receive TileFetchRequests through a channel. The LayerMultiplex is defined as following:

type LayerMultiplex struct {
    layerChan chan<- TileFetchRequest
}

The Layer Multiplexer contains an array of channels for different layers renderer, the simplification here supposes it has only one. TileDB runs into a go routine. tr is sent trough RequestChan, then the tileServer waits to read ch. Because it is an unbuffered channel, it’s blocking.

result := <-ch

The situation is the following:

Case 1:

The required tile is not found in TileDB: when reading the data sent through the channel ch, the BlobPNG is nil.

if result.BlobPNG == nil

The tile was not cache. The TileFetchRequest tr is sent to the Layer Multiplexer through the LayerChan channel. The Tile Server is once again hanging on the reading of the channel ch.

result := <-ch

Using Mapnik, the blobPNG containing the tile is generated and sent through ch. The TileServer can now send the response to the client containing the tile image. Then the TileFetchRequest object is sent to the TileDB through the InsertChan channel to be cached.

Case 2:

The requested tile can be found in TileDB, it has been previsouly generated and cached: TileDB provides the tile image through the blobPNG field. It is then sent back to the client.

Conclusion

The Go-Mapnik library takes advantage of the built-in concurrency of Go to develop an efficient tile server. Other projects can be looked at for more information:

• Geoserver is a Java implementation of the Open Geospatial Consortium standard for tiles servers
• TileStrata is a NodeJS implementation, taking advantage of the asynchronicity of its environment.