Grandine: Vector Tiles, Summary April 2017

Continuation of the documentation of my Prototype Fund related work, for episode one see Grandine March Summary

I continued to work on the effort to create a pipeline framework for working with OpenStreetMap data, starting with important ground work.

Because I got frustrated by third party libraries I started building a simple Go library which is able to serialize/deserialize geometry data. To make things simpler it assumes everything to be 2D. This might sound like a heavy limitation, but it makes the API easier to use. Creating very flexible libraries in Go is non-trivial and usually results in runtime pain. For now I have a working implementation of an WKB format encoder/decoder. More will follow soon.

TWKB

When talking about the new data format, I got hinted at the TWKB format, which aims to be a more space-efficient format for spatial data. Quickly it turned out that it has a few drawbacks:

Specification: There is a specification, which is nice. But the structure of that document is a bit bumpy and partially confusing. E.g. there is a part about encoding integer with zigzag and then encoding it with varint, which doesn’t sound right as varint algorithms need to encode ints with zigzag. Otherwise it would only be able to accept unsigned integers, but this would then be called uvarint.

Saving half bytes: Ok, I get it. It is about small files, but saving four bits in the header? Really?

Non-extendable: It is possible to specify extra features in the “metadata header”. But there is no information about the size of those extras. You have actually to be able to parse those, because there are one or more varints inside.

Test suite: There is a reference implementation in JavaScript. It’s got 13 test cases for the whole format. And it doesn’t even check for negative values. Not a great place to start.

Nontheless I will try to implement the format as well as possible. I’ve already parsed worse formats.

Style

I’ve pushed a work in progress vector map style, Lyrk Light. It aims to be a lightweight version of the original, Mapnik based Lyrk map style. Lyrk Light doesn’t support the OpenMapTiles schema yet, which I want to support in the future.

PR

Once again the mobilecamp, a bar camp, found place in Dresden. I decided once again to do a workshop session about OpenStreetMap, which surprisingly attracted a whole lot of people and got me into interesting discussions about projects.

Summary

I had wished to spend even more time on this project, but looking back there has been some progress and there is still plenty of project time left.