I haven’t written a blog post in a few years. But I think it is time to give it a try again.
Well, I have spent too much time in a map client technique that will need some explanation.
Some history, in non-chronological order
This goes back to 2011 or 2012 when I started to think about a new format for transporting geometry data between a server (PostGIS) and a client (a web client). I wanted a compact but fast format to send data around. The result is TWKB (Tiny WKB) now implemented in PostGIS. When I started this there was no MapBox Vector Tiles (at least I had not heard of it), so compact geometry formats was quite unknown.
The background for this wish for a compact format originated from a decade before. When I worked at Skogsstyrelsen in Sweden (Swedish Forest Agency) we started to use pda with ArcPad. Back then I was able to put shapefiles with all data I needed for may daily work into that pda. Detailed data for Torsby kommun, about 4000 km2. A few years later things like that was just a dream. ArcPad maybe existed, but everything went online, and as maps became something everyone demanded, the user focus changed from field work to “search for a restaurant”. On the new platforms like smartphones and tablets there was no good tools for detailed offline work. Then there came some clients that could download “offline areas”. But mostly just smaller areas.
Knowing that the pda back in 2003/2004 had 64 mb of RAM and a sd-card of 1 GB, I think it s quite strange that we don’t have a lot of techniques for storing whole countries with detailed data today.
But I have some more requests than just rendering. I also want to be able to keep data updated, without changing whole data sets. With the future in mind I also think that access to geometries in an easy way for analyzing and processing is a quite natural request.
The command from the daydream part of me to the doer part of me
So, from this, my conclusion is that we need structured data, compressed and fast to access. That is great but how to balance the trade offs? Some basic rules:
First thing to trade away is “eye candy”. If you have been out in the bush trying to do a job (of course in the rain, and hungry) eye candy is one of the most provoking things in the world as all you see is a well designed circle going around. I am supprized that anyone using a computer professionally asks for smoth transitions before snappy transitions. That is like a carpenter asking for the color of the power drill instead of tourqe. So,performance is more important than eye candy
When performance is descent more performance is classified as eye candy. This means that when we have enough performance, optimizing for small storage is more important.
Data can never be too small. There is always more data to bring that might be “good to have”.
So, to summarize:
Forget about eye candy. It just eats the power of your device, and melts down our planet. Enjoy snappy over smooth!
A funny thing I have found when working with this. There is no contradiction between performance and smallish. Often the smaller size in itself gives a better performance, even when local. Both reading and writing big amounts of data from disc have a cost.
Disclaimer of what is coming
All this is still in a very early stage. Big parts of it is also in domains that I have only learnt about from tutorials. So ideas and techniques is very naive and some parts is just a result of trial and error. So, here is a lot of room for more competent brains to take a look.
This is it. Data is stored in SQLite at the client. The layers and styling is defined in a separate SQLite db. There “project”- db can get data from many “Data” db, to form a project.
The client is written in C.
SDL2 is used to abstract away platform differences and to easily get user input.
The code for the client and link to prepacked map-data can be found at github:
I will get back with info describing the project technically. I will also write about how to pack the map-data.
If this sounds interesting, please join in the effort to make something good out of it.
I have put the GPL v2 license on the code. But that can be discussed if there is other opinions.