Tom MacWright

Understanding Map Projections

Beware: this is a post about opinions and concepts, rather than about releasing software or learning about things. And, while I am a ‘professional’, I’m a software developer first and a neogeographer second, and have yet to print out a map.

What are Map Projections?

Chastising the Spanish artist for painting unrepresentative cubistic abstractions, a layman withdrew a photograph of his wife from his pocket and held it up to Picasso with the admonition, “Why can’t you paint realistically, like that?” “Is that what your wife really looks like?” Picasso asked. “Yes,” replied the man. “Well, she’s very small, and quite flat.”

Dorion Sagan, in Dazzle Gradually

Map projections are _creative ways to represent a three-dimensional world as a differently-shaped thing, usually something flat – so you can print or pan out your map. They’re compromises between different sorts of accuracy and usefulness.

Even if one were to reject projections entirely and go into space to take a gander at the real thing, they could only see one side of the world at a time – a disadvantage relative to your common world map.

So you always lose something in abstractions. Projections are a trade between distortion, accuracy, and aesthetics. Some projections wildly distort all paths except for one, like Craig retroazimuthal. Others only show part of the globe, like the orthographic or general perspective. And still others will wildly distort the size of things on the earth, like mercator, the familiar face of MapBox and Google Maps.

You can’t get it all: thanks to the Theorema Egregium, we know that projections from a sphere to a 2D surface will always distort either directionality or area.

Projections are Old

mercator old

1569 Mercator World Map

Map projections are about as old as paper maps, though the modern ‘science of cartographic projections’ is a work in progress.

This matters: the ‘Mercator projection’, invented by Gerardus Mercator around 1569, was a hit because it’s useful for sailing: it preserves angles, a property known as conformality. Sea voyaging was a big deal, and having a map with which you could draw a line from one place to another and sail that path without hitting a continent was pretty useful.

Projections are New

It turned out that a variant of the Mercator projection, what we call spherical mercator, not only had this property, but also is really easy to calculate: here’s an implementation in Javascript:

function lonlat_to_meters(ll) {
    var A = 6378137, D2R = Math.PI / 180;
    return [
        A * ll[0] * D2R,
        A * Math.log(Math.tan((Math.PI*0.25) + (0.5 * ll[1] * D2R)))
    ];
}

Sure, this loses critical aspects of traditional cartography: not only does it distort scale, it treats the Earth as a sphere, when it’s more the shape of a squashed, dimpled ball.

But technology is a result of its environment and requirements, not any absolute value: spherical mercator fits the role of a simple, fast, pannable, and zoomable projection like no other common projection.

Much like how the original mercator projection was developed for sea voyages, the projection that we now use is developed for web maps, and carries the marks of that purpose. In a future where dynamic morphing of data is cheap and 3D is the standard, it might seem outdated – but it will still be no more or less right than any other choice.

So maps need to be zoomable: this would be nonsense twenty years ago, but is vital today. The value of Google Maps is not in its whole-earth view, but when it’s zoomed up to street or province level, where it looks pretty darn good. And maps that don’t zoom on the internet are considered broken: user expectations are set hard by our constant scroll-wheeling and pinching to think that anything that could get closer should. This forces modern cartographers to re-weight the importance of global views versus local, and rethink styling.

Paper ends

And they’re typically pannable. There are beautiful projections, like the Albers projection, with beautiful implementations, like the one above from d3, an incredible visualization library. But one can feel the influence of paper: what happens at the edges? Where is the context? Of course, in the age of posters in offices and printouts for GIS class, it doesn’t matter if the world ends at the corners of the US or New Jersey.

And projections like Albers do indeed continue, but not quite like you’d expect: sometimes sweeping up the edges of the page instead of continuing horizontally as you scroll.

The sum of this change is that currently conformality, that aspect of preserving direction that spherical mercator handles deftly, is possibly the highest priority for the majority of web maps.

But these requirements are based on the current state of technology: that square tiles, of size 256x256, are the way that people get maps. And that the map one person sees is the the one everyone else does. These restrictions are based on technological hurdles that are likely to fade: MapsGL is the first ‘production’ vector web map, and Apple is rumored to be working on ‘napkin maps’ that are tailored to one specific route. It’s likely that driving directions will no longer show extraneous roads, or that the projection you see zoomed-out, looking at the entire world, is the same one as you see close-up.

This is the first quick bit I’m going to write on projections. Hopefully the rest will follow with this little projection toy – trying to make this subject area more approachable in a creative way.

Yell, a Yelp Exporter, and TOSes

I’m releasing some code that could get you in some mild trouble if you use it. It’s nothing groundbreaking – just a run-of-the-mill scraper written with nodejs that grabs your data from Yelp and gives it to you either as JSON and HTML formatted with hReview.

It’s open source on GitHub – the yell project.

~/⇾ npm install yell

~/⇾ yell YOUR_USER_ID
fetching articles starting with 0
fetching articles starting with 10
fetching articles starting with 20
fetching articles starting with 30
fetching articles starting with 40

Finished! Find in this directory:
- yelp.html, a hreview-formatted HTML version
- yelp.json: raw JSON data.

Dear Yelp: it only fetches one user’s data – not everyone’s. No need to worry about evil people stealing data, if you really tried to use this tool to that, it’d do a terrible and incomplete job. It’s for users who want their data.

Yelp has an API. It’s right here, but, in the words of a Yelp employee, it’s made for businesses – it doesn’t have a method to get your data.

The Terms of Service

The problem is the difference between what lawyers write about technology and what they write about copyright.

As between you and Yelp, you own Your Content. (5C: Content Ownership)

Yelp is reasonable about copyright; like many other services, they claim perpetual rights to use your content pretty much however they like, and the aggregates of your star rating with everyone elses isn’t “everyone’s” – it’s owned by Yelp. Fair: aggregation is what they do.

But you own the © on your data – what if you want it?

You also agree not to, and will not assist, encourage, or enable others to: use any robot, spider, site search/retrieval application, or other automated device, process or means to access, retrieve, scrape, or index any portion of the Site or any Site Content; (6B, part iii)

All credit to Lawrence Lessig for popularizing the notion of code is law: this is that.

You can write your reviews and post them on Yelp, but there is no way – API or scraping – that you can legally copy them from Yelp, except by visiting each page and copy & pasting. For me, this is a deterrent to contributing to Yelp, even if it’s tepid reviews of coffeeshops. And since I’ve found DC’s best, there’s not much to say there.

So I get it: companies see user-generated data as their competitive advantage. If anyone could get a MySQL dump of Yelp, there’d be lots of competitors who are ‘unfairly’ advantaged by having the work done for them. Yelp has competition, like Google Places, Foursquare and the like, and needs to manage how they reuse and its content.

But that’s not the point: a website inviting contributions but lacking an export API isn’t good enough for conscientious or creative users. In this case, over-eager legal terms really limit the potential of site.

Scrapers and Exporters

The first iteration was in node.io and CoffeeScript, but I rebuilt it with cheerio, a great implementation of jQuery’s essentials along with a relaxed parser. And instead of request, I used the library that I wrote, and that still powers TileMill and some other work projects – node-get.

This really isn’t a significant amount of code: maybe 50 LOC total, and an hour less time to build.

Scrapers are odd like that: I wrote a quick one for Garmin’s website to get running data for my running map and it got a decent amount of usage – and even a meaningful improvement in a fork.

Scrapers rarely work on more than one site, and abstracting the process rarely yields results. This makes them a nice to do every once in a while: it tends to help out a lot of less-technical people to give them the ability to export the data that they own, but is hard to pull out.

The Wary Guide to OpenLayers

OpenLayers is a very popular Javascript API for web mapping. It’s deployed on the White House, used as the basis for a bunch of other projects. In a past life, I maintained one of those projects, the Drupal module and got a lot of experience using OpenLayers. MapBox launched Hosting with OpenLayers and kept it that way for quite a while before switching to Modest Maps. So I’ve had quite a bit of experience using, debugging, and occasionally patching OpenLayers.

Though I recommend you take a long, hard look at Leaflet, Modest Maps, or polymaps for new projects, there are lots of people who are going to use OpenLayers anyway, for whatever reason (please add reasons to the comments). So here are some collected words of wisdom about common pitfalls with the library. Hopefully this saves new users some time and head-scratching.

For new users, OpenLayers is a scary, scary place to be. You’ll find many things breaking in unexpected ways for weird reasons. This post is a quick summary of vital understandings to have.

LonLat is not Longitude Latitude (i)

You would think that LonLat is asking for longitude and latitude values: it reads Lon(gitude)Lat(itude), right?

Nope, OpenLayers.LonLat is a dumb container for any sort of coordinates, whether they’re in degrees, meters, feet, or cubits. It doesn’t take on any responsibility for projecting those coordinates into anything usable by your map. If you do want something usable, you’ll need to do something like

var pt = new OpenLayers.LonLat(-20, 20);
pt.transform(
    // degrees are degrees
    new OpenLayers.Projection('EPSG:4326'),
    // but your map is in meters (probably)
    new OpenLayers.Projection('EPSG:900913'));

There’s another gotcha here, did you notice it? Yep, .transform does changes in-place. It doesn’t return a reprojected pt, it reprojects pt. If you aren’t aware of this, you’ll probably get burned – especially since the API returns the changed point as well.

projection and displayProjection (i)

Much like OpenLayers.LonLat, OpenLayers.Map.displayProjection seems to introduce itself: that’s the projection in which the map will display.

Nope: displayProjection is used in a handful of places. If you add a OpenLayers.Control.MousePosition to the map, or a permalink, it’ll get used there: the permalink will link to something using that projection, and your mouse’s position will show degrees or meters or whatnot. But if you don’t have those controls, it doensn’t matter one bit. The setting could be called ‘textDisplayOfPositions’ or such, and it’d be more accurate.

Your Raster Projections Aren’t Going to Work (i)

Do you have one map that’s in EPSG:4326 and one that’s in EPSG:900913? Or one that’s in some lambert conic local thing and you want to put it on top of a MapBox map?

I hate to break it to you, but this isn’t going to work. You see, projections are baked into images: the images are skewed and sized, and their tile sizes changed, in order to fit your very specific mapping of earth-places to screen-places. And at this point, there is no javascript API that can reproject images from one to the other.

If in this situation, there are a few options:

  • Get your tile server to work in the proper projection for the map
  • If you don’t have control over the server, it’s going to be hard. There are bleeding-edge solutions like MapProxy that can reproject upstream sources, but the task is difficult.
  • Use vector data, if your data and visualization needs are extremely light

Re: the first one. If you’ve got some map in a bizarre projection just because that’s the standard for the unit of company or government you work for, figure out whether you can change that standard. It’s unlikely that Google Maps and co will be supporting boutique projections soon, and having incompatible maps means that many fewer people will be using your data, and you’ll have more limited software options.

Not to say that you should switch entirely to spherical mercator: it’s not the ideal projection for static thematic maps or for local maps of certain places. But not providing tiles in it is unwise: the point of the internet is to combine things, and your odd-projection maps will be orphans otherwise.

When in doubt, put settings on Layers

OpenLayers lets you put lots of settings everywhere. The weirdest place is the Map versus the Layer. This is a simple one: the Map is a fallback for Layers. Put your geographic stuff on Layers. That means extents, projections, resolutions, what-have-you. Don’t repeat yourself and put these settings in two places – that’ll only make things more complicated.

If you’ve got lots and lots of layers in handwritten Javascript and you need to move settings to the map, you’ll have other problems anyway.

maxExtent does not mean what you think it means

No: maxExtent doesn’t keep people from moving the map. You’re looking for restrictedExtent.

maxExtent configures how much of the world is rendered on the map. Actually, it does something more complicated than that, because it’s not thinking about the typical ‘XYZ layer with only a country rendered’ it’s more like ‘renderer thinks that this country is the world and rendered that’. I’ve never been able to get maxExtent to work doing anything but setting it to the maximum Mercator extent.

Your imgPath (and themePath) is broken

Ah, imgPath. The best way to use OpenLayers controls is to not use them, but unfortunately a lot of users are deathly scared of writing their own code, even if that just means hooking up zoom in & out buttons.

But, if you must: imgPath. If you don’t provide it, OpenLayers tries to figure it out. So provide it, because the system will guess wrong. One day you’ll be chillin with a great map and your code is like:

/js/OpenLayers.js
looks for
/js/theme/*

Then someone runs YSlow or something and gets the bright idea of combining your javascript resources. So, the urls become

/OpenLayers.js
looks for
/theme/*

And all of your images break. Moral of the story: specify your imgPath and themePath. And download the images for OpenLayers if you plan on using them.

Don’t hotlink your OpenLayers

Too many sites pull from openlayers.org for OpenLayers.js. Don’t do this, unless you’re just putting a page up for testing.

The OpenLayers.org version of OpenLayers.js is a full build – which means that it’s big. It’s 957K un-gzipped and 220K gzipped. For reference, 220K gzipped is around 7 jQueries.

OpenLayers.org is not always online, and when it goes offline, your maps will break. All of them. And depending on how you include the script tag, your site will take a lot longer to load, because it’s trying desperately to get the code.

Plus, if you use the library on the front page, http://openlayers.org/api/OpenLayers.js, you’ll get the latest API version of OpenLayers. Meaning you build your site against 2.11, and when 2.12 is released, your site breaks.

Hopefully this is enough to convince you: download OpenLayers for yourself, and put it on your own server.

OpenLayers Controls Are Ugly

It’s not too hard to make OpenLayer’s default controls look nicer: they simply haven’t been updated for years. Development Seed’s OpenLayers Theme is one example – try it, and use the source SVG images to make your own theme.

Note that the OpenLayers theme system falls apart at certain points: popups are one part of the library that are near-impossible to style nicely.

Go and Conquer

If you’re really looking to level up with OpenLayers, do a custom build: look in the build/ directory, and you’ll find a build.py script you can use to make slimmer builds of the library that only include parts that you use. A dated example of how to do this is openlayers slim, which we used at Development Seed before switching off of OpenLayers to Modest Maps.

If you’re using OpenLayers with MapBox Hosting maps (as I’d recommend you do), try the connector provided by the Wax library. With the magic of TileJSON, it removes a lot of the potential configuration fail in using OpenLayers.