Overture Maps first dataset release

Hello

Overture Maps has just made the following announcement
https://overturemaps.org/overture-maps-foundation-releases-first-world-wide-open-map-dataset/

Links to access the data are given in https://overturemaps.org/download/
Particularly: https://github.com/OvertureMaps/data/blob/main/README.md#how-to-access-overture-maps-data

Best regards

1 Like

Just to clarify: is the CDLA Permissive v 2.0 compatible with OSM (ODbL)? (In other words, can we use places and admin boundaries files on OSM?)

I saw some compatibility with v1, but didn’t find a more recent discussion with v2.

1 Like

Also mentioned and discussed in this thread.

1 Like

From the blog: Overture Maps Foundation Releases Its First World-Wide Open Map Dataset.

Additionally the GitHub Overture Maps Data Repo.

2 Likes

Didn’t take long for them to have to rely on confusing things by using the 3-letter abbreviation.

From their GitHub page:

Accessing Overture Maps Data
Overture Maps data is available in cloud-native Parquet format. There is no single Overture “entire planet” file to be downloaded. Instead, we have organized the data for the Overture 2023-07-26-alpha.0 release by theme and type at the following locations:

1. Amazon Athena (SQL)

  1. You will need an AWS account.

2. Microsoft Synapse (SQL)

  1. You will need an Azure account.

Ah! NOW (at least parts of) this scheme is making more sense to me: to even access these data, you need to pay Amazon or Microsoft!

4 Likes

Well, anyone can download it without using these accounts, by using azcopy (section 4 of that page, which is still cumbersome). Downloading the data isn’t the bad part, opening and using the data is probably the worst.

Ha, Planet OSM/Geofabrik is light years ahead :sweat_smile:

3 Likes

Thanks for your clarification, @matheusgomesms. I saw the azcopy stuff and it still looked like part of the Azure (Microsoft) fabric, so I thought that was paywall, too, but you have corrected me.

Indeed, “opening and using the data is probably the worst.” I have no idea who would be interested in this, or why. The case to do so remains very fuzzy, at best.

A sort-of “double indeed,” Geofabric’s Planet OSM is “light years ahead,” but I’m not saying “Ha” to this, as when an unknown force is (fast?) approaching us in our collective rear-view mirror, I don’t want to laugh, I want to make sure I am not overtaken (or sideswiped during passing and crash into the ditch).

As I’ve said as a long-time OSM contributor about this whole Overture endeavor: “Eyes wide open” (still). It remains pretty murky / very fuzzy as to what, exactly, we are seeing emerge, but with such alpha.0 releases, we begin to “see.”

1 Like

Indeed, “opening and using the data is probably the worst.” I have no idea who would be interested in this, or why. The case to do so remains very fuzzy, at best.

comparing the numbers it seems in OpenStreetMap there aren’t 60 million pois by a margin, so maybe there is interesting stuff? Unless it’s the kind of old data with businesses closed for years, and so on, it seems this could be worth to have a look.

Cheers Martin

1 Like

Agreed, Martin: it is in OSM’s good interest to “parse” these data (Overture 2023-07-26-alpha.0 release by theme) and comprehensively understand what they are, maybe why they are, not necessarily how they are (unless it is believed there are allegations of “theft” of OSM data, i.e. using them not in alignment with our ODbL) and whether they are “interesting stuff” (or not). I realize the latter is highly subjective, but I totally agree that OSM as a whole WILL find it worth our while “to have a look.”

It was funny to me because I know lots of people who find it hard to actually use OSM data (which I kinda agree). But then, seeing what Overture is doing, shows that we actually provide many ways to use our data (some easier than others, but that’s the beauty of a project like OSM).

I still want to open PoI Overture data to compare to OSM, but well, I’ll have to wait some tutorial for dummies to properly open and explore those Parquet (???) files.

But I agree with you, we have to look at this situation and position ourselves on: what we can learn from this, and how can we improve? For instance, I still think we need to provide better tools to use OSM data (better plugins for QGIS, Python libraries, improved documentation…).

4 Likes

I guess LWG is having a closer look at the licensing situation. While there is some mention of OSM in the Overture context, I didn’t get the impression attribution to OSM is excessively visible. Is there are paragraph somewhere which states how to credit the source when using overture data? If there is, does it contain mention of OSM?

We are seeing the OSM community being reeled in by Big Map Brother. All map refinements will be on Overture, and once Overture data is used as a source for OSM mapping, OSM mappers will be checkers/improvers for Overture. It doesn’t matter whether we see ourselves as independent or not.

“I’m improving Google Maps” might be an even stronger incentive for the general public than “I’m improving OpenStreetMap”.

If I’m right, we will see tooling being provided by Overture to replace the chaotic wealth of tooling available to mappers/data users in OpenStreetMap.

It’s a well thought-out move, perfectly legit, and it will work as long as the parties in Overture are in agreement, and will compete not on the basic map database, but on the services built on it.

2 Likes

A bloke in the states extracted a small set of the POI data, see the discussion here Eugene Alvin Villar: "Given that #OvertureMaps has now released their f…" - OSM Town | Mapstodon for OpenStreetMap

PS: @scai this was previously reported in the thread

1 Like
./azcopy copy "https://overturemapswestus2.dfs.core.windows.net/release/2023-07-26-alpha.0/" ~/Documents/overture/data
INFO: Scanning...
INFO: Autologin not specified.

failed to perform copy command due to error: Login Credentials missing. OAuth token, SAS token, or shared key should be provided for Blob FS

I downloaded a bounding box around Berlin and calculated some coverage stats.

  • Categories – 22.5 % of places don’t have any category
  • Restaurants and cuisines
  • Distribution of the confidence values
  • Brands
  • Supermarkets

Here’s what to expect with regards to category and brand attributes for some popular supermarket brands:

keyword  category_main                       brand_name
ALDI     discount_store                      NaN                1
         food_beverage_service_distribution  NaN                1
         grocery_store                       ALDI Nord        182
                                             NaN               33
         shopping                            NaN                1
         supermarket                         NaN                4
         wholesale_grocer                    NaN                1
         NaN                                 NaN              136
EDEKA    discount_store                      NaN                2
         grocery_store                       EDEKA              5
                                             NaN               61
         retail                              NaN                2
         shopping                            EDEKA              1
                                             NaN               13
         shopping_center                     NaN                2
         supermarket                         EDEKA              8
                                             NaN               92
         NaN                                 NaN              188
REWE     business                            NaN                1
         convenience_store                   REWE To Go         1
         flea_market                         NaN                1
         gas_station                         REWE To Go         6
         grocery_store                       REWE               2
                                             REWE To Go         2
                                             NaN               10
         shopping                            REWE               1
                                             NaN                3
         supermarket                         REWE             212
                                             toom Baumarkt      1
                                             NaN               11
         NaN                                 NaN               41
dtype: int64

Feel free to fork the notebook to run your own analyses.

Use the Azure links provided in the Github, such as https://overturemapswestus2.blob.core.windows.net/release/2023-07-26-alpha.0/theme=places

It still needs credentials.

aws + --region us-west-2 is working for me ( > 200 GB ) - without any credentials.

aws s3 cp --region us-west-2 --no-sign-request --recursive s3://overturemaps-us-west-2/release/2023-07-26-alpha.0/ <DESTINATION>

with : aws-cli 2.13

$ aws --version
aws-cli/2.13.3 Python/3.11.4 Linux/5.4.0-155-generic exe/x86_64.ubuntu.20 prompt/off

( via )

1 Like