Storing OSM Data

Hi Guys,

Hope your all doing well and enjoying the Beijing Olympics 2008 like i am. Just wanted to ask experts, i would like to store the osm map data in a database but before doing so what format do i need to convert this data into like a shapefile? As there is no option for this. Once converted in the desired format, how can i store this into a database. Cheers

I think you better ask yourself what do I want to do, and then decide what tool to use to do that. At the moment OSM is pretty “unique” among GIS tools, in that we allow anything to be tagged as anything. So to get the whole OSM database is very hard.

If you just want roads or coastlines then it’s easier, but you still need to think about what you want to do…

I saw the Marathon (but just 40km I think?), 18 hours ago, and it was pretty cool. A fun race.

There are many tools to import data in other databases likes pgsql/mysql or files like shapefile. See http://svn.openstreetmap.org/applications/utils/export/ or osmosis. If you are really lazy, the company geofabrik has shape files for european countries available at download.geofabrik.de

Well, what i would like to do is take this map which i took from OSM site, and store the information such as the road names, cor-ordinates, place names etc in a database. Then create an application similar to that of google map that queries the database in such as way that if i type Great Portland Street, the map will then be shown with a pointer indicating where Great Portland stree is. Hope that is a bit clearer.
http://www.megaupload.com/?d=5Y58619K

There’s no tool that will do exactly that, but it really isn’t that hard to do. All the tools are there but not exactly what you want.

So you have two goals, one is to display a map, the other is to search for names in that map. The main server does this by two databases which each week is created from the planet.osm files. One of the DBs are for rendering (mapnik) and the other is for finding names (name finder). So what you describe is more or less the same as a main site setup.

You can display a map in many ways, using a static image downloaded from OSM is actually abit harder than just including the javascript to get the map from out site. To use the static image you need;

  1. home page
  2. the image somewhere on that page
  3. download Openlayers
  4. configure Openlayers for static images (can’t remember how you do this)

Searching for names should “only” be a matter of getting the area you have a picture of as a .osm file, and then load it into name finder. So you need:

  1. php5 webhost
  2. mysql5 database
  3. an .osm file for the area you want. (this can be hard, I’ve seen no ready made files for the US but the java program Osmosis can do it)
  4. import the .osm into name finder
  5. copy the HTML for name finder from the main openstreetmap site.

Or then you can go the traditional interactive web mapping way. For example you could import osm data into PostGIS with osm2pgsql utility, set up UMN Mapserver to deliver the data and use some interactive client on top of that, for example p.mapper or GeoMOOSE. Or set up OGC web feature service (WFS) and use that for queries.

Are there anyguide for setting up mapserver with Openstreetmap? Mapserver isn’t know for being easy to use…

Installing Mapserver is not hard, there are ready made binaries and good documents about how to install. But starting to render maps which look like current Mapnik maps would mean a reasonable amount of work for the first people to do so. Mapnik style file has more than 5000 rows and that would be the same with Mapserver mapfile. But just to make system work and show vectors through WMS with simple styles is rather easy. Perhaps I should have a try and write a beginners tutorial one day.

Yes that would probably be very helpful in the long run.

Though what you are talking about is all “standards” so there should be example applications that will help you look for name= tags in mapserver, or is there?

Cool, although before doing so, i need to know how i can store this map data in a db like in PostGis. On the OSM site there is no option to store it as .osm so i’m a bit stuck with how exactly to convert the PNG Image to .osm file befefore storing it in db.

That’s what JRA tells you to do; use planet.osm and osm2pgsql, but as I say above this can be tricky…

Hi,

Thanks for your comments, well appreciated guys. I have also installed my own wampserver 2. Now, if wanted to read my .png image into a mysql db, would osmos be able to convert this to a .osm? or would i need to go do planet.osm to do this? because at the moment i just want to try it with my static image.

Cheers

You can try to press the export tab, and choose "OpenStreetMap XML Data " as the format to export. and select the same area as you got the png from. This will give you everthing you need, as long as you export the same area as png+osm data.

Hi, i have downloaded the Osmosis file which did take some time to unzip, 25mins :frowning: and i just wanted to know how do i essentially integrate this with mysql db becuase the wiki does provide the commands to read and write the data but i’m unsure how to essentially use with my data and my mysql db on my static image.

Cheers

Install name finder, php5 and mysql:

Name Finder is downloaded from the SVN, you can get info about SVN in how to get the OSM code

Hi thanks,

i have php 5, name finder, mysql (Wampserver2) which has the sql tables but what i am very curious to know is why do i need the following tables:

changeid,
lock,
named,
options
placeindex,
way_node
word
because i am looking at my osm file and i cannot find any of these objects there and cannot find any referrence in the wiki that explains the table objects or why it has thing such as ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;. Is there a referrence available that explains the file?

I am also trying to follow the link http://wiki.openstreetmap.org/index.php/Osmosis of reading my data in the database but when i do this on both the mysql prompt and the standard command prompt they give errors stating that they are unaware of osmosis. So after downloading the osmosis file which was osmosis-latest how do read the data in the mysql db? I have missed something out?

Cheers

Does anybody know how i can use the osmosis to read the .osm file onto my mysql database because i’m confused as to what the standard of doing this would be???

Cheers

The namefinder uses it’s own database schema to suit it’s needs best. So you can’t use Osmosis to load the planet file because Osmosis assumes a different database schema. To import the planet file you will need to use the import.php script from the namefinder/utils plugin. The script will translate and filter the osm data into the database, so the database does not ‘look’ the same as the osm file.

You will have to do some reading on the MySQL site for understanding the ‘create table’ queries.

So let me get this straight, to read my .osm file in my mysql database all i need is to use import php script and create the tables for the for this?? If that is the case what about the name finder sql file business? is that also used for reading in the data because at this stage all i want to do is read the data in BEFORE doing anything else like downloading other utility applications.

I see this project in 3 stages:

  1. Get the data - .osm file from street map site
  2. Store the data - using a database like mysql
  3. View the data - using a search query something like the namefinder to search for the place and view it on a map , specifically using the data i have chosen. (Stage 3 will be more complex than this)

I don’t want to over complicate things and only want to download and use the things i truely need for each stage incrementally.

Am i right in saying that to simply store my .osm file i need to use the php script and creat the tables to store the data?

I have also created table called node with id, lat, long, user, visible and timestamp columns.
Cheers

The .SQL file is used to install the database schema (the tables that information goes into.) into the database.

So

  1. download Name finder
  2. create a database (e.g. with phpmyadmin)
  3. create the tables in the database from the sql file read this for some background
  4. configure namefinder
  5. load info into the database with the tools available