Is it possible to mirror OpenStreetMap ?

Please accept my apologizes if the question seems stupid, but is this possible ?

I would like to setup a server, which would have its own data (planet.osm as I guess) and would answer to queries like openstreetmap.org and informationfreeway.org do.

I managed to run the Rails port, it works but still uses tile.openstreetmap.org for the render.
What am I supposed to do ?

You should decide which Map you want to mirror. I guess you mean “Mapnik” ( http://wiki.openstreetmap.org/wiki/Mapnik ) or Osmarender ( http://wiki.openstreetmap.org/wiki/Osmarender ). For Mapnik there is a mod_tile to request tiles over Apache. But Mapnik requires an PostgreSQL-Database. Osmarender uses xml-Files theirselves (extracting them from planet.osm could take long I guess), but needs no further database.

Hope this is what you meant.

Thanks for the answer, it is close to what I’m looking for.

I already have a PostgreSQL db, I followed this tutorial : http://www.weait.com/content/build-your-own-openstreetmap-server.
But once mod_tile is up and running, how should I tell him to use my database ?

Moreover, I wonder if this will be enough : openstreetmap.org seems to rely on lots of servers !

I would like to have a least the slippy map feature, like informationfreeway.org. Any suggestion / information / useful link ?

You actually have to tell mapnik to use your database, not mod_tile.
As far as I know mod_tile says to mapnik to render tile a b c and d, because it was requested by someone. But I don’t use mod_tile, so I cannot say anything about configuration.

You have to edit this file http://trac.openstreetmap.org/browser/applications/rendering/mapnik/inc/datasource-settings.xml.inc.template (actually on your mapnik installation). Then Mapnik will know which database to use.

Here are some examples for a slippymap listed: http://wiki.openstreetmap.org/wiki/OpenLayers

EDIT: Here’s something about mod_tile as well, probably this could help: http://wiki.openstreetmap.org/wiki/Creating_your_own_tiles#Creating_tiles_using_Mapnik_and_mod_tile

Thank you E-Malte, it does help. I didn’t notice that datasource-settings.xml.inc file, but now I know it exists the fog disappears slowly.
Actually mod_tile uses a configuration file which points to osm.xml (from the mapnik distribution), and this one points to datasource-settings.xml.inc.

What do you use instead of mod_tile for your map renderings ?

I user render_tiles.py and render_image.py (I think they’re called that way) simply one time when I update my database. But it’s not a whole planet.osm, just a local extract, so it doesn’t take so long.
You can have a look at http://download.geofabrik.de . There are extract’s from nearly everything.

You can have a look at renderd and Tirex as well (i.e. search for them in the wiki). They are also rendering deamons for tiles or images.

Yes, I knew about Tirex too. But renderd (aka mod_tile, it’s the same) is said to be the reference.
I want to enable the slippy map feature on a dedicated server, so I’m not really interested at static mapping.

I managed to make renderd work, at least it works for osm_tiles2/0/0/0.png so that’s one more step forward.

Now I still have two other issues :

  • how do I generated tiles for something else than 0/0/0 ? I get a “Not found” error when I try, though renderd seems to do someting.
  • how do I “plug” the rails_port to my local tile server ?

Sounds like you are getting close then.

Now I still have two other issues :

My guess would be you are seeing “Not found” as mod_tile is timing out. When mod_tile gets a request, it checks if it has a cached copy of the tile. If it does, it will send it straight away. If it doesn’t, it will send a render request to renderd (or tirex if you are using that instead) and then weight for a response from the renderer for up to a timeout of 3 seconds (configurable in mod_tiles config). Then it returns either an outdated tile if it has one, or returns “Not Found” if it has never rendered that tile. Renderd will continue rendering the tile though in the background and eventually finish. So the next time you visit that tile it will have it cached. For low zoom tiles, the rendering can take on the order of minutes, so much longer than the 3 seconds timeout. For high zoom levels like z18, 3 seconds is often enough.

Unfortunately I think the tile server location is hardcoded in the rails_port. So you will need to modify the code. But it is vert easy to change in map.js in the public/javascript directory of the rails_port

Btw. For clarification, mod_tile and renderd aren’t exactly the same, although heavily linked and thus often used interchangable. Mod_tile is an apache module that handles http requests for tiles and serves tiles from the local filesystem using a special file format (metatiles). If the files are either missing, or outdated, it will send a request off to a rendering backend that will generate metatiles, from which mod_tile can then serve the tiles. Renderd and Tirex are both rendering backends that can talk to mod_tile. Renderd is the original one and the one running on tile.openstreetmap.org. Tirex is a recent reimplementation to make the rendering backend more flexible.

I just wonder… I just imported a local extract, so why do I see the whole world when looking for 0/0/0 ?

I double-checked, the only tile I can see is 0/0/0. Even if I wait for some time after the request.

It’s because on zoomlevel 0 nothing from the database is rendered. It’s just the coastline-data, and this is coming from another file I guess you “installed” already. On higher zoomlevels you will see the coastline everywhere as well, but not the other data from OSM. Just where your extract is.

Thank you again for your support, E-Malte. Are you an OpenStreetMap developer ?

I used the generate_image.py script (from the OSM Mapnik tools) to check that Mapnik uses my datasource.
As you said I saw the “installed” maps, and only the coastline elsewhere.
I loaded a Luxembourg extract into the database and successfully generated a Luxembourg map picture, using ll=(5.5, 50.2, 6.6, 49.4)

But I’m still having troubles with mod_tile. Using the same map extract I just have blue tiles when I try to access, for instance, to 10/50/6 (same results for 10/6/50).

We’re actually all developers, as we all contribute to the map. :wink:
But no, I don’t develop Mapnik or the Rails Port or something else. Just an interested user.

I guess it’s not a mistake, when you only get a blue tile. Look at the output of the main map, it’s only blue as well:
http://a.tile.openstreetmap.org/10/50/6.png
http://a.tile.openstreetmap.org/10/6/50.png

Try to access a tile, that is actually a tile covered by your osm-extract.
Probably zoom there on the main map, and then right-click: Show Image. Then you can read the tile numbers in the adress bar. When there’s something drawn it should be with your local copy as well. So copy the path and replace the domain with your tile directory. Then there should be something rendered.

P.S. On zoomlevel 10 you can actually have lots of tiles being blue only. Zoomlevel 10 coveres less than 50x50 square kilometers. There are more than 1 000 000 tiles in zoomlevel 10. With about 70% water on earth there should be at least in the atlantic or pacific ocean probably 100 000 tiles being totally blue, as there is no ferry or a part of the coastline… Hope I calculated everything correctly.

Ok I get it :wink: I thought that the z/x/y parameters were zoom/latitude/longitude.
You were right : I was looking at the sea. I just corrected the query parameter, now I can see the town !

By the way, I also understood that the slippy map is only a javascript web page built upon OpenLayers.
So I don’t need to setup the Rails_port, a simple Web server is fairly enough !

I am happy now :smiley: