Environment
-
using the officiel docker image: overv/openstreetmap-tile-server:latest
-
Packages
-
renderd 0.6.1-1
-
libapache2-mod-tile 0.6.1-1
-
libmapnik3.1 3.1.0+ds-1ubuntu2
-
mapnik-utils 3.1.0+ds-1ubuntu2
-
osm2pgsql version 1.6.0
-
-
Host: Debian 13, 8-core CPU, 32GB Ram, NVMe (PCIe Gen 4) storage for Postgres data
-
Region: 11 european countries combined by sorting via
osmium sort -s multipass, merging viaosmium mergethen manually deduping (dedup-script verified working correctly, as osm2pgsql would’ve thrown on import otherwise) → ~1.85B nodes, 263M ways, 3.4M relations- imported via
osm2pgsql --create --slim -G -hstorewith flat nodes
- imported via
-
renderd.conf:
[renderd]
socketname=/run/renderd/renderd.sock
num_threads=8
tile_dir=/var/cache/renderd/tiles
[default]
URI=/tile/
TILEDIR=/var/cache/renderd/tiles
XML=/home/renderer/src/openstreetmap-carto/mapnik.xml
HOST=localhost
TILESIZE=256
MAXZOOM=20
This is my first time using osm’s tile-server, please let my know if any vital information is missing.
Hello everyone,
I want to apply incremental OSM replication diffs to a live database via osm2pgsql --append, then using the resulting --expire-output tile list to invalidate only the affected tiles in the renderd cache without either synchronously re-rendering every affected tile nor nuking the entire cache. Per the wiki, this should be achievable with render_expired’s --delete-from=ZOOM or --touch-from=ZOOM.
Tried running --append against a replication diff (one day, unfiltered/global) and --expire-output for --expire-tiles 12-20 - even if the wiki recommends 3 zoom levels max - which resulted in 113,626,496 lines, then filtered down to the subset that intersects the imported region - verified working correctly. A 1k-line sample of the result was used for the following isolated tests:
# Default
docker exec -i <container> render_expired --socket=/run/renderd/renderd.sock < tiles.txt
# Delete
docker exec -i <container> render_expired --socket=/run/renderd/renderd.sock --delete-from=12 < tiles.txt
# Touch mode
docker exec -i <container> render_expired --socket=/run/renderd/renderd.sock --touch-from=12 < tiles.txt
Progress verified via pg_stat_activity on the tile-server’s Postgres:
SELECT pid, now()-query_start AS dur, state, wait_event_type, wait_event,
left(query,150)
FROM pg_stat_activity
WHERE usename='renderer' AND state != 'idle';
However, in all three tests only one tile was processed at a time and each individual tile triggered a Mapnik rendering query taking anywhere from ~20s to 8+ minutes per tile - presumably depending on cache warmth. Verified via pg_stat_activity showing SELECT ST_AsBinary("way") AS geom, "feature", "way_pixels" FROM (...) queries blocked on disk I/O.
The processes also never exited but hung instead (verified by /proc/pid/wchan reading futex_wait_queue) even if the underlying work visibly completes (verified by queries changing PID/text, cache .meta file mtimes going past the container restart time, renderd’s CPU usage stays non-zero). Yet, render_expired itself never terminates so the hang looks like it’s specifically in the client’s own exit-logic.
Is this a configuration problem on my end or is this a known issue with render_expired / mod_tile 0.6.1? - I didn’t find a related issue that described my observed behaviour.
Warm regards