Extract only data required by a Mapnik file

I have been using Osmfilter to extract data from an OSM data file. In general it works but what I would really like to do is to start with a Mapnik file and extract only the data that Mapnik requires. I guess this could either be a standalone program that takes a data set and a Mapnik file and generates a reduced data set or some way of converting a Mapnik file into an Osmfilter parameter files.

Does such a beast exist?

First, I quote from the Mapnik entry in the OSM Wiki: “Mapnik is an open source toolkit for rendering maps. Among other things, it is used to render the five main Slippy Map layers on the OpenStreetMap website. It supports a variety of geospatial data formats and provides flexible styling options for designing many different kinds of maps. … Note that Mapnik is a software project and should not be confused with any of the common map stylesheets that might be named after Mapnik. One of them, Standard tile layer is used as the main map on the OSM website.”

Therefore, explain in greater detail what you mean by “Mapnik file”.

And, if you mean a Mapnik style .typ file, then you most likely will be disappointed by your results as the Garmin GPS receivers have default styling for many map elements and more data than is mentioned in the .typ file is needed to display a map correctly.

If your goal is minimizing the amount of data that you download, then, downloading a large data file and filtering that is not going to help you as the large download already has occurred to avail the file that you filtered.

Help us to help you, please.

(at the risk of stating the bleeding obvious) the “data that Mapnik requires” could be anything that’s defined in the style file that you’re using.

It might be worth having a look at how osm2pgsql does it. It essentially uses a file to determine what gets included in a rendering database. The default one is https://github.com/openstreetmap/osm2pgsql/blob/master/default.style . Different map styles may define their own, e.g. https://github.com/gravitystorm/openstreetmap-carto/blob/master/openstreetmap-carto.style for the “standard” style.

Depending on what you want Mapnik to show you’ll want to extract less or more data from OSM to display. I’ve not heard of anyone using osmfilter for this (though I guess it’d make sense for really simple maps - say just a main road network and coastline).

Of course, osm2pgsql already does something very like what you want (filters data into a rendering database according to a style file). Would that perhaps do what you want?

I think you’re confusing Mapnik and mkgmap there :slight_smile:

My apologies. I should have made myself clearer. I am running OSM on a local device with limited hard disk space. I have no problems with size of the data download but it occurs to me that I am storing a lot of data in the postgres database that is never going to get read because my Mapnik stylesheet does not render it.

So, it would be nice to be able to strip this all out based on the rules in the Mapnik stylesheet. This could be either a direct filter before importing the data or a selective import to the database. The important thing is not to clutter the database with data that will never be accessed.

If you’re using an “off the shelf” map style then I’d expect that that will come with a style file to be used with osm2pgsql - if something isn’t in there then it won’t be added to the rendering database.

If you’re creating your own map style then you can create your own “style file to be used with osm2pgsql” as well, containing only the data that you actually need. I’m not aware of anything that creates a default.style from a style definition (though it sounds like an easier job than generating a legend based on a style definiton, which people have talked about) - but that doesn’t mean that it doesn’t exist.

Thanks, I was hoping to experiment with a Mapnik stylesheet to get the rendered map that I wanted using a full database and then re-import the data taking only that which I actually needed.

If you’re working with a small data extract (say the size of an English county) re-importing the data will take only a minute or so. I find that actually editing the .style, .mss and .mml bits of the stylesheet (assuming that’s the stylesheet flavour you’re using) takes longer than that.

(again apologies if this is the bleeding obvious but) you can get data extracts in small sizes from e.g. http://download.geofabrik.de/ and if what you need isn’t there in a small enough size you can use https://wiki.openstreetmap.org/wiki/Osmosis et al to cut one down to size.

Actually I am working with rather large areas, at the moment continents but I was hoping that I could reduce the file size enough to be able to import the world. I only have a 32-bit database and cannot change that. I was hoping to exclude everything that won’t be rendered by the default Mapnik style to see if I could then fit it all in a 32-bit Postres file.