Pick only certain objects in osm2pgsql

Help and support
, ,
1

I am in the process of trying to revive Openptmap. I have experience in programming, Linux administration and Docker, but it’s my first time working with Mapnik and the tools surrounding it.

I have found some old instructions how Openptmap used to be set up as well as the configuration files surrounding it. Very roughly, the tiles were created like this:

  1. Download a PBF file of the region to be rendered
  2. Convert the PBF to O5M using osmconvert
  3. Filter out all the irrelevant objects using osmfilter with the supplied filter file
  4. Import the filtered O5M file into a PostGIS database
  5. Generate the tiles using Mapnik with the given XML style file.

While I did have success with this, it does not seem to be the most up-to-date and efficient approach. In particular, the osmconvert step takes really long without providing any actual value, and from what I read online, it seems that a typical setup nowadays would fetch deltas every now and again and rerender only the necessary tiles rather than downloading and rerendering the whole world every few days.

I have found openstreetmap-tile-server, which seems to be the recommended way to host OSM tiles these days. Out of the box, I think it doesn’t work with the existing Openptmap setup for two reasons:

  • openstreetmap-tile-server does not provide a way to use osmfilter on the map data, and I think it also doesn’t support O5M files
  • If I run the osmfilter script externally, convert the result back to PBF and pass that as the “region” to openstreetmap-tile-server, I assume that the automatic updates probably won’t work correctly as the service is not aware that my data is filtered.

I’m wondering whether I could adapt the Openptmap setup to be compatible with openstreetmap-tile-server. I noticed that there is an option NAME_LUA, which will be passed as --tag-transform-script to osm2pgsql. This option is used to do Lua tag transformations. Also, there is an option OSM2PGSQL_EXTRA_ARGS, which allows to specify arbitrary additional arguments for osm2pgsql.

Is the Lua tag transformation something that could replace the osmfilter configuration? If so, are there any resources that could help me convert the osmfilter configuration to a Lua script?

Or are there any other options that I could pass to osm2pgsql that would replicate the osmfilter setup?

Do you have any other ideas how I could improve this setup to improve rendering performance and to support automatic updates?

2

Hello,
Opensnowmap.org still use a daily filtered planet for the ski piste layer.
Yes, the major drawback is there’s no minutely updates, but it’s still efficient if you have heavy data postprocessing on a filtered extract.
I switched from osmconvert / osmfilter to Osmium a while ago though. It maintain an up-to-date planet.pbf file directly without passing trough an o5m format.

See data-opensnowmap.org/tools/00_osmium_pistes_workflow.sh at master · OpenSnowMap-org/data-opensnowmap.org · GitHub

3

What you want to do is exactly what osm2pgsql and its flex output is for: Read a bunch of OSM data into a database, filtering and converting the data while you are importing. You can use the Lua config to decide in detail what data you need and what data you don’t need. And convert everything into database tables in a format that is later easy to access when rendering the map to keep the Mapnik (or other styling configurations) simple. And you can do regular updates if you want to.

Note that the references to --tag-transform-script are really old, don’t use this. Read the osm2pgsql manual, especially the section about the flex output. There is a lot of information in the manual, also about pre-filtering the data with pros and cons. And there are example config files in the repository and examples with code on the web site.