I’ve been attempting to perform a planet import for mapnik rendering for the past few weeks without success. I’ve setup my PostGreSQL database with PostGis correctly, including the various parameter tweaks. I setup PostGreSQL on a separate Windows Server 2008 box with 8GB RAM, in order to separate processes and not have osm2pgsql have missing memory issues. This box has plenty of space for the final database.
I setup a virtual box on our virtualization server using a CentOS distribution, obtained osm2pgsql from source, turned on the 64bit compilation flags and compiled it. This virtual box has 16GB memory.
I’ve attempted several times to import, without success. The last attempt I made was using a boundary box, hoping osm2pgsql would have an easier time just importing North America. I’m using slim mode and a 14GB cache.
Here is the command line and the output:
./osm2pgsql64 -c -s -m -C 14000 --bbox -165,13,-45,90 -U gis -W -H 192.168.1.131 -P 5432 -S ./default.style ./planet-110810.osm.bz2
osm2pgsql SVN version 0.80.0 (64bit id space)
Using projection SRS 900913 (Spherical Mercator)
Applying Bounding box: -165.000000,13.000000 to -45.000000,90.000000
Setting up table: planet_osm_point
NOTICE: table “planet_osm_point_tmp” does not exist, skipping
Setting up table: planet_osm_line
NOTICE: table “planet_osm_line_tmp” does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE: table “planet_osm_polygon_tmp” does not exist, skipping
Setting up table: planet_osm_roads
NOTICE: table “planet_osm_roads_tmp” does not exist, skipping
Mid: pgsql, scale=100, cache=14000MB, maxblocks=1792001*8192
Setting up table: planet_osm_nodes
NOTICE: table “planet_osm_nodes” does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index “planet_osm_nodes_pkey” for table “planet_osm_nodes”
Setting up table: planet_osm_ways
NOTICE: table “planet_osm_ways” does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index “planet_osm_ways_pkey” for table “planet_osm_ways”
Setting up table: planet_osm_rels
NOTICE: table “planet_osm_rels” does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index “planet_osm_rels_pkey” for table “planet_osm_rels”
Reading in file: ./planet-110810.osm.bz2
Unknown node type 8
Processing: Node(1173423k) Way(104443k) Relation(1070274) parse time: 175361s
Node stats: total(1173423751), max(1393174131)
Way stats: total(104443727), max(125432779)
Relation stats: total(1070274), max(1706327)
Going over pending ways
processing way (26160k)way_done failed: ERROR: out of memory
DETAIL: Failed on request of size 419430400.
Arguments were: 75277817,
Error occurred, cleaning up
Now the error occurs with or without 64bits, with or without the boundary box. It seems like the pending ways section of the code is behaving oddly, or I’ve setup something incorrectly. The size of the memory request is odd as well. Anybody have any ideas?
Best regards to all,