How to download all the protected areas of the world at once

PILarramendi · 2017-11-17T11:19:03.000Z

Hi all,

Is there any way to download all the protected areas at once? Right now, I’m downloading the whole OSM database, and it takes too much much time.

Best,
Pablo

GerdP · 2017-11-17T17:38:06.000Z

Did you try the overpass api?

PILarramendi · 2017-11-20T08:10:59.000Z

I’ve tried but with no luck, I’m doing something wrong. do you know what should it be the overpass for boundary = protected_area ?

GerdP · 2017-11-20T12:07:05.000Z

Probably the amount of data is too large for a single request.
Taginfo shows ~ 70.000 objects
https://taginfo.openstreetmap.org/tags/boundary=protected_area
and I assume that many are rather complex ways or multipolygons.

PILarramendi · 2017-11-20T13:59:18.000Z

Ok, I see. How do you think I should download the data?

whturner · 2017-11-20T14:49:17.000Z

I tried the Overpass search in The Netherlands (rather small country ). The amount of data was 5MB.

mmd · 2017-11-20T15:47:46.000Z

GerdP:

Probably the amount of data is too large for a single request.

No. You just need to increase the timeout.

Protected areas for the whole world takes 1.2 GB(!!). Not sure if you really want to download this much data. Maybe start with a small area first.

PILarramendi:

I’m downloading the whole OSM database, and it takes too much much time.

Is this something you want to do on a regular basis? Did you already know that you can also update your local database simply by applying diff files, without a need to download the whole planet again? What exactly did you mean with “it takes too much time”?

PILarramendi · 2017-11-20T16:42:35.000Z

1.2Gb is not much because I want to use the data locally with my GIS software

For now, I just need to download the data for one time.

Basically, I downloaded the whole OSM database, and I tried to extract the protected areas with my GIS software, but it takes too long, I need a faster way to download all the protected areas.

mmd · 2017-11-20T17:31:31.000Z

Osmfilter or osmium tool are both well suited to pre-filter your local planet pbf file. Afterwards you can just use that extract rather than the full planet in your GIS tool.

PILarramendi · 2017-11-24T11:21:56.000Z

I’ve tried Osmfilter, but I get a huge file with the protected ares (178GB), this is what I typed

osmfilter planet.osm --keep-nodes="boundary=protected_area" --keep-tags="all boundary=protected_area name=" --ignore-dependencies -o=PAs.osm

Any idea why is not extracting only the PAs?

alester · 2017-11-24T20:43:00.000Z

PILarramendi:

I’ve tried Osmfilter, but I get a huge file with the protected ares (178GB), this is what I typed
osmfilter planet.osm --keep-nodes="boundary=protected_area" --keep-tags="all boundary=protected_area name=" --ignore-dependencies -o=PAs.osm
Any idea why is not extracting only the PAs?

I’m not even sure why the file would be so huge for the command you ran. There are less than 3000 nodes tagged with boundary=protected_area and you’re only keeping two tags for each, so the file should be very small.

I suspect you don’t really want just nodes tagged with boundary=protected_area, though. Don’t you also want areas, like closed ways and relations? That’s how the vast majority of protected areas are mapped.

Gotegomadi · 2017-11-25T03:40:18.000Z

mmd:

Did you already know that you can also update your local database simply by applying diff files, without a need to download the whole planet again? What exactly did you mean with “it takes too much time”?

How I can do it without unpacking PBF file to database? I prefer to work with compressed planet.pbf, here exist any way to patch this file? I mean not using sql or any long-importing procedures

mmd · 2017-11-25T08:25:23.000Z

Gotegomadi:

How I can do it without unpacking PBF file to database? I prefer to work with compressed planet.pbf, here exist any way to patch this file?

osmupdate would be the right tool to process your local planet.pbf file and create a new updated planet pbf.

OCPyrit · 2018-03-03T00:53:19.000Z

Hello, I have a similar problem: OSMFilter seems to produce way too large files! I start with a dataset for everything of germany, that has 63GB. Then I run osmfilter to filter for all historic objects. I end up with a filezize of 57GB! This sems way too big, I expected a smaller output when only looking for a small subset such as historic tags.

Furthermore, running ogr2ogr on the produced file to convert the 57GB osm file to shp leaves me with 290mb of data…

Does osmfilter output file have some unnecessary overhead or why is it so big and how can I fix that?

I use this commands if it helps:

osmfilter germany-latest.osm --keep-tags="all historic=" -o=germany-historic.osm

this gives me 63gb → 57gb
Way too big, no??

OCPyrit · 2018-03-03T17:08:33.000Z

I just found the error, --keep-tags=historic just drops all tags other than historic from all objects, but keeps all the objects.
–keep=“historic=” Just keeps objects with tag historic.

SK53 · 2018-03-03T19:29:30.000Z

Yes, definitely use --keep in the first instance!

Also always save osmfilter output in .o5m format. Once happy with the result turn it back to .pbf using osmconvert to save more space.

Using overpass with wget might be a way to download all the protected areas, but this misses out the area assembly provided by overpass-turbo.