How to import a pbf file into a Overpass (OSM3S) private server db?

Help and support
,
1

I need to issue ~ 75,000 overpass queries to determine admin level areas by WGS coordinates (I assume that Overpass is the best choice for this task). The query works fine, but i thought it would be better to run the whole query batch against a private OSM3S server. Installation worked fine, but the documentation ( Overpass API/Installation - OpenStreetMap Wiki and Installation of an Own Instance ) only describe how to import a whole planet file into the server DB. I don’t need the whole planet data, since all coordinates to query are inside Europe, and my storage space is limited. There are different sources for PBF files just for Europe, but I don’t know how to import them into the server DB. Is there any documentation for this use case? Is this possible at all?

1 Like
2

For this kind of task, you could consider using GeoDesk.

You can turn any PBF into a Geo-Object Library (GOL) (which requires only about 40% more space than the PBF), and run fast queries against it using Python, Java, C++, or the GOL command-line tool.

  1. Download the GOL Tool

  2. Create the GOL (takes about 10 minutes on a modern system):

    gol build europe europe-latest.osm.pbf

  3. Install Python and the GeoDesk module: pip install geodesk

  4. Run your queries – e.g. find all admin areas at a given location:

from geodesk import *

europe = Features("europe")
admin_areas = europe("a[boundary=administrative]")
pt = lonlat(8.12, 50.13)
for area in admin_areas.containing(pt):
    print(f"- {area.name} at level {area.admin_level}")

(edited script)

1 Like
3

Looks great :grinning_face: esp. since I’m using Python (overpy) already. Thanks a lot!

The only drawback: no client/server architecture, so I’ll have to run the queries locally. But that’s not a big problem in my case. I’ll wait some time anyway before labeling this as solution, maybe there are even more ways to solve the issue.

1 Like
4

@GeoDeskTeam Hmm :thinking: i’ve tested your example (geodesk version 2.0.5, Python 3.12.3), but it fails with an error:

Traceback (most recent call last):
File “/.../.../geodesk_test.py”, line 7, in 
for area in admin_areas(pt):
            ^^^^^^^^^^^^^^^
TypeError: geodesk.Coordinate is not a valid argument

geodesk_test.py looks as follows (just copied your example and added an import clause):

import geodesk
from geodesk import *

europe = geodesk.Features("europe")
admin_areas = europe("a[boundary=administrative]")
pt = lonlat(8.12, 50.13)
for area in admin_areas(pt):
    print(f"- {area.name} at level {area.admin_level}")

So, i’m not sure how to test it to handle coordinates. Both building and querying the gol file works perfectly fine (tested a europe pbf with the museums example), but i couldn’t find an example using coordinates. Not sure if I should open a github issue?

BTW, PyCharm shows some warnings since it can’t resolve Features and lonlat by default.

1 Like
5

Sorry, I made a mistake in the script – it should be admin_areas.containing(pt).
I updated the script above (I also changed the import so lonlat is visible).

The output (for lonlat(8.12, 50.13)) should be something like this:

- Taunusstein at level 8
- Seitzenhahn at level 9
- Rheingau-Taunus-Kreis at level 6
- Hessen at level 4
- Regierungsbezirk Darmstadt at level 5
- Deutschland at level 2

(Query results are unordered)

Btw, the IDE may give you some warnings about unresolved attributes (specifically, the tags of features like area.admin_level, because they are resolved dynamically) – I haven’t figured out how to solve that yet. You can also use the longer form area["admin_level"], which won’t trigger the warning).

1 Like