Bulk-Abfrage: Liste mit Lat-Long-Koordinaten mit Maximalgeschwindigkeit anreichern

Ich habe eine Liste mit Koordinatenpunkten (n=200k) von Hausfassaden (lat/long), zu denen ich gerne jeweils das Speedlimit der nächstgelegenen Straße ermitteln möchte.

Gibt es eine Möglichkeit, eine Abfrage in bulk zu starten, die den Koordinaten einen Straßennamen (ggf. Hausnummer) und speed limit zuordnet?

Input (n=200k)

lat;      long
52.0000;  13.0000
52.1000;  13.1000
...

Gewünschter Output

lat;      long;    strasse;     hausnummer; speed_limit
52.0000;  13.0000; Schmalweg;   18;         30
52.1000;  13.1000; Hauptstraße; 20;         50
...

Mit der Nominatim-API könnte ich zwar einen reverse lookup der Adresse machen, aber die API ist nicht für Bulk-Anfragen gemacht/erlaubt.
https://nominatim.org/release-docs/latest/api/Reverse/

1 Like

du könntest Dir einen Deutschlandextrakt (bzw. was du brauchst) in
Postgis importieren und das Nachsehen damit machen. Die Extrakte
bekommst du z.B. von der Geofabrik, wie man die Daten in Postgis
importiert steht im Wiki oder ggf. verfolgst du eine Anleitung, z.B.
hier: https://switch2osm.org/

Evtl. kann man es auch mit Nominatim machen, wenn man rate-limiting
macht und nicht zu schnell gleichzeitig abfragt, aber dann dauert es
natürlich recht lang…

You could turn the OSM data into a Geographic Object Library using the GOL Tool (For Germany, this takes a few minutes on a halfway modern machine and requires less than a tenth of the storage as compared to a PostgreSQL database).

You can then run queries against this GOL (e.g. a 50m bounding box around each coordinate) and find the closest streets, using the GeoDesk library for Python or Java.

If you’re interested, I can post some sample code, I had a similar use case a few years ago where I needed to find the closest street for millions of POIs.

1 Like

Thanks both of you! I have downloaded the data for Berlin from Geofabrik and will start to experiment with the tools you mentioned.

@GeoDeskTeam I’d be very interested in your code samples! :slight_smile: :pray:t2:

1 Like

Gladly! Do you have a language preference (Java vs. Python)?

Either works for me :pray:t2:, I’m currently trying to use the gol CLI, creating a bounding box around every position and querying for the nearby streets.

You can use the command-line tool for this task, but you’ll have to parse the output for each query. It’s easier (and much faster) to use a Python script, which will give you the closest streets as objects that you can use directly.

from geodesk import *;

coords = [
    (13.3898033, 52.5174377),
    (13.4523542, 52.5110493) ]

germany = Features("germany.gol")
streets = germany("w[highway][maxspeed]")

for c in coords:
    c_lon, c_lat = c
    box = Box(lon=c_lon, lat=c_lat).buffer(meters = 30)
    nearby_streets = streets(box)
    print(f"Closest streets to lon,lat ({box.centroid.lon}, {box.centroid.lat}):")
    for street in nearby_streets:
        print(f"  {street.name}: maxspeed = {street.maxspeed}")

This takes a list of coordinates (coords) and returns all streets within a 30 meter bounding-box. Its output should look like this:

Closest streets to lon,lat (13.3898033, 52.5174377):
  Unter den Linden: maxspeed = 30
  Unter den Linden: maxspeed = 30
Closest streets to lon,lat (13.4523542, 52.5110493):
  Warschauer Straße: maxspeed = 50
  Warschauer Straße: maxspeed = 50
1 Like

The above might be good enough for your purpose, but more likely, you’ll want the closest street, not just any street nearby. We haven’t yet implemented GeoDesk’s nearest_to query predicate, but no worries, you can accomplish the same by measuring the distances between the streets in the result set and your test coordinate.

Here’s a revised version of the above code:

from geodesk import *;
from shapely import *;

coords = [
    (13.3898033, 52.5174377),
    (13.4523542, 52.5110493) ]

germany = Features("germany.gol")
streets = germany("w[highway][maxspeed]")

for c in coords:
    c_lon, c_lat = c
    box = Box(lon=c_lon, lat=c_lat).buffer(meters = 30)
    p = Point(box.centroid)
    nearby_streets = streets(box)
    closest_street = None
    closest_distance = 1000000
    for street in nearby_streets:
        d = p.distance(street.shape)
        if d < closest_distance:
            closest_street = street
            closest_distance = d
    if closest_street:
        print(
            f"Closest street to ({c_lon}, {c_lat}) is {closest_street.name} "
            f"at {from_mercator(closest_distance, unit='meters', lat=c_lat): .1f} meters, "
            f"with maxspeed of {closest_street.maxspeed}")
    else:
        print(f"Found no street near ({c_lon}, {c_lat})")

This should print

Closest street to (13.3898033, 52.5174377) is Unter den Linden at  24.6 meters, with maxspeed of 30
Closest street to (13.4523542, 52.5110493) is Warschauer Straße at  11.4 meters, with maxspeed of 50

Perfect, thank you! This works for me, however, only if I remove the maxspeed restriction.

Closest street to (13.3898033, 52.5174377) is None at 14.0 meters, with maxspeed of None

The gol file I have loaded does not seem to contain the maxspeed nor the street name. I followed these instructions and downloaded the osm.pbf file from Geofabrik. Do I need to specify any options to include these missing attributes?

gol load works only for existing tile repositories, but not for .osm.pbf files. For those, you’ll need to use gol build to create a new GOL:

  1. Download the berlin-latest.osm.pbf file
  2. Then, run gol build berlin berlin-latest.osm.pbf

Sorry, that’s actually what I did, I will edit. The attributes are not included though.

That seems strange, since gol build should never exclude valid features or tags. Can you run the following command?

gol query berlin w[highway][maxspeed] -f=count

(This should return 59466 for a GOL built from the current Berlin extract)

Also, just to double-check that gol build completed successfully, it should have displayed output similar to this:

gol build berlin e:\geodesk\mapdata\berlin-latest
Building berlin.gol from e:\geodesk\mapdata\berlin-latest.osm.pbf using default settings...
Analyzed e:\geodesk\mapdata\berlin-latest.osm.pbf in 1s 151ms
Sorted 6,945,026 nodes / 1,111,137 ways / 17,005 relations in 3s 927ms
Validated 38 tiles in 1s 18ms
Compiled 38 tiles in 4s 607ms
Linked 38 tiles in 91ms
Built berlin.gol in 12s 138ms