Mapnik's Second Act

Mapnik is a tool for rendering maps. Early in the history of Mapbox, Mapnik was the ‘beautiful’ in the ‘beautiful custom maps’ slogan. Unlike many of its competitors, the project emphasizes details like anti-aliasing quality and Photoshop-like compositing filters to yield results usable by professional cartographers.

The direction of geo is toward vector maps, and the renderers on that side are more like Mapbox GL or Mapzen’s Tangram than Mapnik: WebGL/OpenGL-based systems with an emphasis on frame-by-frame rendering. Mapbox’s new map designs don’t use CartoCSS or Mapnik XML - we’re switching to Mapbox GL Style Spec, which opens up whole new possibilities.

But what of Mapnik? What we’re finding out is that the years spent on Mapnik have yielded a project that’s pretty remarkable in many ways in addition to map rendering.

Packaging

mapbox

One of the things that has always made Mapbox different from other node.js shops is how heavily we need to use native modules. node-mapnik, node-sqlite3, node-zipfile, bridge the JavaScript-C++ divide for good reason: interacting with large binary files and connecting to established, complex codebases is tough.

And once we develop these native extensions, they need to be deployed to hundreds of EC2s quickly and without a long compile step. To make this happen Dane Springmeyer and others built mason, a magical build system that yields S3-hosted easy-to-install binary packages.

The end result is that you can install Mapnik with npm install mapnik, and on 90% of computers it’ll download a binary and be done in a few seconds.

I can remember days of struggling to compile Boost headers and link everything correctly. The world is a better place now, and the same magic can be applied to other node modules.

Image Processing

Rendering thousands of tiles a second and paying for the bandwidth is a powerful motivator to optimize image encoding. As a result, Mapnik has raster-image processing superpowers that have been reused in quite a few projects:

• node-blend composites multiple images into one, with offsetting and compression tweaks. It’s incredibly fast, and used in production for map compositing. Under the hood, it’s node-mapnik.
• spritezero renders SVG icons into raster sprite sheets for usage as map icons. It’s what powers the new icons interface in Mapbox Studio, and it’s made of node-mapnik.
• assert-http is a testing framework for REST APIs. It has the ability to test image outputs and do fuzzy comparisons to account for random values in antialiasing - thanks to node-mapnik.
• node-fontnik uses a port of Mapnik’s rendering stack to rasterize SDF fonts for Mapbox GL maps.

Mapnik’s image API covers much of the surface area of Imagemagick

but unlike many of the Imagemagick bindings for node, using node-mapnik doesn’t mean shelling out to some other process, and the image APIs play nice with node.js Buffer data.

Data

Mapmaking is a process of simplification: given the limits of resolution, size, and cognition, maps always include a subset of available information. In its current place in the Mapbox stack, Mapnik is the engine that turns raw data into visualization-appropriate tiles. As part of this task, it has grown impressive data interfaces: in addition to OGR bindings, Mapnik has its own incredibly fast interfaces to CSV, Shapefiles, TopoJSON, GeoJSON, and SQLite datasources.

This means that you can use Mapnik’s incredibly battle-tested code to implement tools like format converters:

var mapnik = require('mapnik');
mapnik.register_default_input_plugins();

var source = new mapnik.Datasource({
  type: 'csv', file: './csv_to_geojson.csv'
});
// get meta-information about this datasource
console.log(source.describe());

// convert this CSV file into GeoJSON
var featureset = source.featureset();
var featureCollection = {
  type: 'FeatureCollection',
  features: []
};
var feature;
while (feature = featureset.next()) {
  featureCollection.features.push(JSON.parse(feature.toJSON()));
}
console.log(JSON.stringify(featureCollection, null, 2));

source