Could need some help: Incredibly slow pre-render of tiles (Mapnik 2.2)

General talk
1

Hi

Earlier I imported europe-latest.osm.pbf to database.

Yesterday I startet to pre-generate tiles from a portion of the europe area as set by this bounding box:
http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=3.98,57.69,32.23,71.46

Now generate_tiles_multiprocess.py has only generated 275 tiles in 18 hours and 15 minutes.
I have set generate_tiles_multiprocess.py to use 4 threads:
NUM_THREADS = 4

as well as specifying this at the end:
minZoom = 1
maxZoom = 11
bbox = (3.98,57.69,32.23,71.46)
render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom, “Norway”)

I can see that the four CPU cores has a load usage of approximately 10 - 30%, most of the times it is about 20%.
More than 10GB of memory is not in use.

I do not understand why it is so slow.

But I do remember that I forgot to remove -l from the options when importing data to the database using osm2pgsql:
I used this:
osm2pgsql -s -U postgres -C 8000 --number-processes 4 -l -m -d osm -p planet_osm -E 3857 -S “N:\osm2pgsql\d
efault.style” T:\planet-osm-2018-07-28\europe-latest.osm.pbf

Can that -l option (–latlng) when importing data using osm2pgsql - cause Mapnik to use extra long time to generate the tiles?
If that option is the cause - can I fix this without having to import whole pbf file from Europe again?
If it’s not the cause, what could be the reason for the extremely slow tile generation?

Computer:

  • The whole DB on an SSD
  • Tiledir on a 10000 RPM disk (SATA 2).
  • 3.2 Ghz CPU with 4 cores.
  • 20 GB RAM (5 x 4 GB)
  • Windows 7
2

I’m still experiencing extremely slow pre-render performance.

In the XML file used by Mapnik, I have this for each layer, among other things:


<Layer name="landcover_line" srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over">
        <StyleName>landcover_line</StyleName>
        <Datasource>
            <Parameter name="dbname">osm</Parameter>
            <Parameter name="estimate_extent">false</Parameter>
            <Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>

Where are the four numbers in the extent parameter coming from, and what are the relation between those numbers and the bbox set in generate_tiles_multiprocess.py? In the generate_tiles_multiprocess.py file I have set the bbox like this:


bbox = (3.98,57.69,32.23,71.46)

and what does the parameter estimate_extent do. Now it is set to false. What happens if it is set to true?

Mapnik is still running since threadstart… and is now on zoom level 8, generating approx 6 tiles per hour. Which is just to slow.

Anyone?