New Jersey Buildings Import

Communities United States
1

I’ve assembled a proposal for an import of New Jersey buildings from the NJDEP, I’d appreciate feedback!

Documentation

This is the wiki page for my import:

This is the source dataset’s website:
https://njogis-newjersey.opendata.arcgis.com/
The data download is available here:

This is a file I have prepared which shows the data after it was translated to OSM schema:

License

I have checked that this data is compatible with the ODbL.
This data is distributed under a custom terms of use that does not conflict with the ODbL.

Abstract

The data comes from the Impervious Surfaces (2015) dataset provided by the New Jersey Dept. of Environmental Protection’s Bureau of GIS, which, when buildings are isolated, totals 8.21GB of information as saved in .gpkg format. When the Buildings were isolated from that database (selecting only features with CLASS=Building), these were then compared to the existing set of buildings within OSM with QGIS’s Select by Location function, and any new features that overlapped with OSM’s data were removed from the set (which does make for a rather conservative import, but better safe than sorry). This was then exported to .shp and imported with JOSM’s OpenData plugin. In accordance with the dataset’s use terms, no tags were affected from the original dataset, and the only additional tag used was building=yes. No address conflation was performed.

Cheers!,
-BMACS001

2

I had a look at the .pbf file:

osmium tags-filter "Building Footprints.osm.pbf" -R w/building=yes -f opl

And see:

....
w-3792466 v0 dV c0 t i0 u TOBJECTID=242954.0,CLASS=Building,ACRES=0.031106445961883,MUN=RAHWAY%20%CITY,COUNTY=UNION,MUN_LABEL=Rahway%20%City,MUN_TYPE=City,NAME=Rahway,GNIS_NAME=City%20%of%20%Rahway,GNIS=885363,SSN=2013,MUN_CODE=2013,Shape_Leng=165.39606837723844,Shape_Area=1354.9913661179166,building=yes Nn-29825636,n-29825637,n-29825638,n-29825640,n-29825641,n-29825643,n-29825645,n-29825647,n-29825648,n-29825649,n-29825636
w-3792464 v0 dV c0 t i0 u TOBJECTID=242953.0,CLASS=Building,ACRES=0.00365490545517,MUN=RAHWAY%20%CITY,COUNTY=UNION,MUN_LABEL=Rahway%20%City,MUN_TYPE=City,NAME=Rahway,GNIS_NAME=City%20%of%20%Rahway,GNIS=885363,SSN=2013,MUN_CODE=2013,Shape_Leng=54.44688689443615,Shape_Area=159.2070447971044,building=yes Nn-29825631,n-29825632,n-29825633,n-29825635,n-29825631
w-3792462 v0 dV c0 t i0 u TOBJECTID=242952.0,CLASS=Building,ACRES=0.033667981209735,MUN=RAHWAY%20%CITY,COUNTY=UNION,MUN_LABEL=Rahway%20%City,MUN_TYPE=City,NAME=Rahway,GNIS_NAME=City%20%of%20%Rahway,GNIS=885363,SSN=2013,MUN_CODE=2013,Shape_Leng=169.55104077943054,Shape_Area=1466.5713951928853,building=yes Nn-29825616,n-29825617,n-29825618,n-29825620,n-29825621,n-29825622,n-29825624,n-29825625,n-29825626,n-29825627,n-29825629,n-29825630,n-29825616
...

So the .pbf has still non-OSM tags I assume these will be cleaned up.

Also that OBJECTID will that be translated in to a some kind of ref key? For BAG we use ref:bag.

In addition to the .pbf, I think you should also provide a sample .osm file for a small area that can be reviewed in josm.

3

The proposal page states that the import would preserve all the attributes from the source layer due to the terms of use. I wonder if that’s a reference to this condition:

Digital data received from the NJDEP may not be reproduced or redistributed without all the metadata provided.

I would’ve assumed this is about the layer’s metadata, not the individual feature attributes. As far as I can tell, this is the same NJDEP Data Distribution Agreement that appears elsewhere on the NJDEP website. The agreement also says they aren’t making any guarantees about the data’s copyright status, but I would sort of expect it to say something like that, since the site also provides imagery. Has there been any outreach to NJDEP’s GIS Bureau regarding this dataset?

OBJECTID is a primary key that ArcGIS assigns to every record in the layer. It has no meaning beyond the ArcGIS layer. In the past, some imports have preserved the object ID, but we’ve learned over the years that it isn’t even that useful for updating an import later on.

2 Likes
4

I hadn’t reached out to NJDEP yet, as from my vantage point it looked like I could comply with the ODbL if I kept the tags, but with these insights I’ve now emailed the NJDEP’s GIS Bureau:

Hello!

I have a couple of questions regarding the NJDEP Data Distribution Agreement, specifically in regards to the Impervious Surfaces (2015) dataset. Firstly, to the best of your knowledge, would this dataset comply with the Open Data Commons Open Database License (ODbL)? Second, the agreement requires that "Digital data received from the NJDEP may not be reproduced or redistributed without all the metadata provided”- does this apply also to tags applied to features within the dataset, or is this just for the overarching layer metadata?

Thanks!

-Ben

Here’s a link to a sample section of the import: https://drive.google.com/file/d/1aNTwREP9y24CkhdJFf3AjSC1tFWkBdaS/view?usp=sharing

5

For what it’s worth, I wouldn’t hold my breath for a prompt, positive response on the first question. The ODbL is pretty unknown among government agencies in the U.S. My usual advice to just ask if there are any restrictions unfortunately might not work here either, since they’ve already communicated a bunch of restrictions. Your second question is more specific, so it should help to clarify the situation. I’d think overall we just want to know if our usual manner of attribution (tags on the changeset and/or elements, mention on the Contributors page) would satisfy their requirements, and whether they can waive some of the more technical requirements (like keeping a certain CRS), since we’re deriving a building dataset for informational purposes instead of tracking impervious surfaces like they are. You can also e-mail the Licensing Working Group for more expert advice.

6

You can also ask their GIS department if they would be willing to release the data for you/OSM under CC0 license which is compatible with ODbL.

7

Received a reply from the Bureau of GIS:

Good afternoon Ben,

Thank you for contacting the BGIS Help Desk.

To the best of my knowledge, the 2015 Impervious Surface layer does comply with the ODbL. If you wish to redistribute our data, you must follow the agreement provided with the metadata. This does include tags.

Best,
NJDEP
BGIS Help Desk

8

Dang, I didn’t think I’d see the day.

I really hope they don’t mean you have to preserve each and every tag verbatim though. You can’t guarantee that someone won’t come along and delete all those extra tags because they look like cruft. Half the tags are automated ArcGIS filler and the other half are actually about the municipality rather than the building.

9

Just sent another email to them:

Hello,

Thanks for the reply! Just to clarify, does this mean that the tags must be kept in perpetuity? I plan to use this data on behalf of OpenStreetMap, and seeing as it’s a map that the public at large can edit (much like Wikipedia), using the NJDEP dataset there means that the data can (and most likely will) be altered in the future. However, OSM does preserve the history of objects in its database for the public to easily see, so it will be plain to see that, when imported, the first version of the dataset did include the tags.

Thanks,

Ben

1 Like