Hi everyone, I have something I’d like to do, but I need feedback first, please. I don’t want to break the rules regarding imports, and also I’m looking to see if this is an ok idea in general.
In short, I’ve tentatively generated about 13,000 potential new ways representing dams from the National Inventory of Dams using the coordinates from the NID and OSM’s existing bodies of water. I’ll provide details and examples (and the whole data set as a .osm file) below, but basically I’ve hit a limit of how good I’m able to generate these (not saying someone else couldn’t do better, but I can’t) and now to reasonably add them to OSM I need to manually check each one. I’m trying to figure out the most efficient way to start doing that, which is why I’m here.
This feels like it’s an import, but I haven’t done one before. I appreciate all advice (even if it’s to stop). Basically, I want to speed up drawing dams on the edges of existing lakes and ponds by doing it programmatically, but still looking at each one before saving any changes. Looking at the examples at the bottom of this post, you can see a lot of them look pretty good, but there are also many where the dam is stuck to the wrong side of the body of water due the to NID’s coordinates being over to one side. Or stuck to the wrong body of water entirely. In most cases, looking at satellite imagery makes it pretty easy filter the good from the bad. But there’s 13,000 of them… So, how to do this more quickly?
If I were to turn these into individual MapRoulette cooperative tasks, I’d include the proposed tags as a table so they could be selected and copied if needed, and then something like:
Does a dam exist at this location? If based on satellite imagery it looks right, please open the proposed changes in JOSM and save it, making any edits you see fit. If the highlighted new dam looks wrong, please either draw it in a better position or simply mark this task as “not an issue” or “can’t complete”. These dams were generated using coords from the National Inventory of Dams and existing OSM bodies of water, identifying the bad ones is still very useful, no matter what you decide. Thank you!
And then even if it’s just me, I’ll work through them all! This seems like it could be similar to challenges to turn GNIS-imported reservoirs into ways. But is this an acceptable way to use MapRoulette?
How I made this:
The NID has recorded over 90,000 dams, and to date about 47,000 dams, weirs, and dykes in OSM have been identified and tagged with their ID numbers (using the tag ref:US:NID).
What I’ve done, is taken their remaining 43k+ dams, defined as a single point of latitude and longitude and:
- Filtered out ones that don’t have a “length” paramter specified in the NID
- Filtered out instances of multiple dams having the same coordinates, so there’s no easy way to tell which one is the right one without looking at it
Then:
- Query Overpass Turbo at the NID dam’s coordinates
- Looked for the nearest node belonging to a water-body relation or way
- OR the nearest mid-point of a way segment between two nodes, if one is closer
If a natural=water was found:
- From that node (or two nodes), come up with a new way by expanding in both directions along the edge of the body of water until it’s close to the length from the NID
- When deciding which direction along the water’s edge to expand, favor more shalow angles over going around sharp corners if possible
- If the new way is over half the length of the dam specified in the NID and both ends have reached corners sharper than about 30 degrees, then stop early
- Otherwise, get as close to the stated length as possible and then stop
- If the new way has any nodes that currently belong to any existing dam, weir, dyke, or embankment, then toss it (unless it’s a single-node waterway=dam/weir). Otherwise, add it to the pile.
Also earlier this month separate from all this I took the about 8000 entries in the retired GNIS historical files for dams that I couldn’t find existing in OSM (nodes deleted or never imported I guess) and tried to conflate them with the NID. I matched like half of them and saved the info to Wikidata. So, in the below OSM files, for the ways I’m proposing to create I was able to include gnis:feature_id and wikidata tags for 1,830 of them.
Example:
Here’s an example of 5 dams generated by this process displayed in JOSM (highlighted in yellow, with red triangle nodes so I can see them easier). The white circles and text in the background are the information I got from the NID.
And the same example with all of the existing OSM objects downloaded. 100% of what I’ve generated uses only already existing nodes.
OSM files:
Five dams in Wisconsin (4KB):
All dams (332 ways) in Ohio (263KB):
Everything (12,959 new ways and 98,238 existing nodes) (13.0 megabytes)
All 3 files zipped (2.1 megabytes):
So, is this ok for MapRoulette? Should I document on the wiki first? Thank you everyone for your time and for any/all feedback you have! I appreciate it greatly.
