Quering the nodes from defined bounding box in OpenStreetMap

General talk
1

Hello,

I am trying to query all the nodes within certain bounding box.
By querying I mean loading some sort of page, which can then be saved to my hard drive. Direct download of the page would also be fine.
I do not want to use JOSM editor, iD, or any other editor.

I seems that this can be done through http://overpass-api.de.
Here is an example, for the given bounding box coordinates (bottom left, top right corner latitude/longitude):

(44.8159610691, 20.4595548745), (44.8168609308, 20.4608191255)

http://overpass-api.de/api/interpreter?data=[out:popup];node(44.8159610691,20.4595548745,44.8168609308,20.4608191255);out;

The problem is that for some strange reasons the upper link finds only 2 nodes.

While if you download the .osm file with the same bounding box, you will see that there are at least 14 nodes:

http://api.openstreetmap.org/api/0.6/map?bbox=20.4595548745,44.8159610691,20.4608191255,44.8168609308

So for some reason the http://overpass-api.de does not return the other 12 nodes in that bounding box. I do not know where should I report his issue.

If this is a bug with http://overpass-api.de, then does anyone knows how to query the nodes for the given bounding box, other than through http://overpass-api.de?

I would welcome any kind of help.

2

When I replace “popup” with “json” in your first query I get a lot of nodes. Perhaps the “popup” output mode cannot deal with the other nodes in the bounding box ?

3

Thank you for the quick help escada!!

With “json” I get a lot more than 14 nodes. I am not sure what is the problem.

Is there any other way to query those 14 nodes (other than through overpass-api.de)?

4

Your whole approach to OSM is rather different from other new users. Well, completely different. Makes me a wonder sometimes why and what for and why so complicated, if perhaps for some special project, or perhaps the plan is to edit not in a special osm-editor but first exporting, then using some shapefile tool for editing, then importing again… or perhaps not for updating the public map at all but only for creating some special private map… or anything, not the slightest idea, just sometimes riddling :wink:

5

Indeed wycbtma, your assumption is correct: my task is to learn both OpenStreetMap and to use it to create something similar to osm2world.org.
I do not think I will be able to manage that kind of complexity, nor it will be embedded into a website, due to both my non-GIS and non-programming background but something basic at least will suffice.
And it will be an open source project.

The nodes which include “tags” are the nodes that I am looking for. The other ones seem to be some sort of construction nodes maybe (the nodes from which ways are made of)? It does not matter.

The issue is solved by escada’s advice on changing the “popup” instead of “json”.
Thank you once again escada!

6

How you want to retrieve data from OSM will depend on you processing flow afterwards.
If you really need a lot of data, you might be better off downloading a country extract and manipulate that with e.g. osmfilter or in a PostGIS database.

Overpass is fine if you know which type of data (e.g. all shops), but I doubt it’s the best way if you want all nodes with tags.

BTW, I hope you realise that POIs can be mapped as (closed) ways or relations as well. Some people will map shop=* on the closed way of the building. And when that building has a “hole” inside, it can be mapped on the relation as well.

7

Here’s the recommended way to find nodes with at least one tag. It’s even mentioned on the Overpass API by Example Wiki page:


node[~"."~"."](44.8159610691,20.4595548745,44.8168609308,20.4608191255);out;

overpass turbo llink: http://overpass-turbo.eu/s/kdo

8

I think escada is talking about appropriateness. Overpass is intended for complex server side searches. The main API is intended only to be used by people who are going to update the map.

Also both have usage limits. http://wiki.openstreetmap.org/wiki/API_usage_policy for the main API, and http://wiki.openstreetmap.org/wiki/Overpass_API gives limits for individual servers. If this is more than a proof of concept, you should be aiming to using planet.osm, or a regional extract, and running your own server.

More generally, to do a good 3D renderer, I think you need to have been using OSM for a long time, or you need to study the rendering rules used by other renderers.

9

Looking at http://overpass-api.de/output_formats.html#popup the unqualified use of popup only generates output for objects that represent features likely to be of interest to end users. It is not intended to be parsed by anything more than a web page renderer, followed by a human.

10

www.openstreetmap.org/export even provides an option to download all objects in a bbox via Overpass API. So downloading all data in a bbox is a perfectly fine use case.

11

I think the issue would be that if you made a lot of such requests or requested large areas. Having overpass return large amounts of data with very little server side processing is wasting its capabilities at the same time as using a lot of its internet bandwidth. I suspect your particular example is not very heavily used, or is used to populate a working database, updated daily or weekly, that is then used to actually serve data to the clients.

12

As mentioned before, that’s one of the supported use cases. Not sure why you think this is wasting capabilities. We have people downloading hundreds of MB of OSM data.

I don’t know exactly how you define “very heavily used”, but retrieving (some/certain/all) data for a given bbox is indeed very common.

13

That is perfect mmd!!!

Thank you very much!

http://overpass-api.de/api/interpreter?data=[out:json];node[~"."~"."](44.8159610691,20.4595548745,44.8168609308,20.4608191255);out;

It’s good to know that. Thank you.

I didn’t know this. Thank you for the information hadw!
So the upper link is considered to be an openstreetmap API usage:

http://overpass-api.de/api/interpreter?data=[out:json];node[~"."~"."](44.8159610691,20.4595548745,44.8168609308,20.4608191255);out;

?

Does that mean that I am not allowed to use it, and that I need to use planet.osm instead?

14

The overpass API can be used for more purposes than the main API, but it is still a limited resource. If you create a mass market product, you need to provide your own server.

15

I am essentially writing a free and open-source OSM import script. I am not sure if it will be mass market.

When you say:

Does “limited” mean that for example: once 100 users used this script, they will all be blocked by overpass-api.de website?

I am not sure I understand the quote from the Api usage policy article you gave me:

16

I would say that ff your script get data from the OSM API and renders a 3D scenary, is using the API “for read-only purposes or projects”, so it does not comply with the usage policy.

It’s very easy to get a planet.osm once, and then get all the updates you need for your toolchain.
Read https://wiki.openstreetmap.org/wiki/Planet.osm/diffs and https://wiki.openstreetmap.org/wiki/Osmosis/Replication

17

The usage allowed will depend on the server, but, the page I quoted suggested that, at the moment, if the users of your application make more than 10,000 overpass accesses a day, summed across all the users, or more than 5GB a day, that would be excessive even on the .de one. So for, 100 users the limit for the .de one would be 100 accesses a day per user or 50MB a day per user. Those limits may be reduced if the total usage of Overpass increases.

If you expect to get anywhere near these limits, you should talk to the people running the server in advance.

The main API is provided to allow people to add to or modify the map. Whilst it can be used to download maps for viewing, the intention is that you should only view the map in order to provide a context for adding new information to it, not just to have a map to look at.

Providing the main API is more expensive than providing the overpass API, because the main API has to update immediately when a change is made.

18

Thank you muralito! I am a bit confused now.
The limitations of OSM API usage policy article that hadw posted is related with downloading the .osm files through opentsteetmap.org website only?
For example, downloading the .osm file by providing a bounding box:

http://api.openstreetmap.org/api/0.6/map?bbox=11.0,40.0,11.01,40.01

But quering the nodes data through overpass-api.de is something completely different, and does not comply with the upper mentioned OSM API usage policy article? Is this correct, or wrong?

This is very useful information hadw. Thank you.
I do not expect to have such a large overpass-api.de accesses. The script will probably be used by a couple of tens of people, a couple of times a day.

Do you think it is possible to contact the Administrators of OSM, and ask if download of the .osm files through OSM API can still be allowed, if I limit the bounding box maximal size? wiki.openstreetmap.org says that the maximal bounding box size is 0.5 x 0.5 degrees of latitude/longitude. Can I set it to say: 0.1 x 0.1, and in this way limit the total size of the downloaded .osm file?

Thank you for your replies once again, and for being patient with me.

19

To use OSM data in a renderer process it’s seems logical to:

  1. Get OSM data from planet.osm (or only a country extract or continent extract) (it’s better two separate process, one to download the complete extract, and another to download the updates of the data)
  2. Process the data downloaded in step 1. to render whatever you want. It doesn’t matter if you want to render in 3D, a map in pdf for paper, in a HTML page which list hotels, in a Garmin .img or whatever output do you produce.
20

Thank you murlatio.

So planet.osm does not support the direct download of .osm data through a bounding box, the way api.openstreetmap.org does:

http://api.openstreetmap.org/api/0.6/map?bbox=11.0,40.0,11.01,40.01

?