Adding bridge information in Kentucky and other states

[tekim (Mike)]

@csomerville

Bridges will have deep foundation, typically will span more than 30 meters, and have a distinguishable superstructure (deck, girders) and substructure (abutments, pier, pile cap). Conversely culverts are buried structures and rely on the combination of internal capacity and the surrounding soil to carry vertical load.

This seems reasonable. Also, I agree with the edit, I don’t think there is any minimum length for a bridge.

By the above definition, many of the “bridges” in the proposed import are culverts, e.g.:

One end of the culvert is to the NE in the image, and the other to the SW.

Looks like there is actually at least two culverts in the above image.

@Minh_Nguyen

for those of us lucky enough to have access to oblique imagery, street-level imagery, or a field survey for a given spot

Even without such imagery, you can generally apply the above definition, for example, if there is significant vegetation showing between the edge of the road surface and the structure in question, it is probably a culvert as vegetation generally doesn’t grow on road surfaces (exception for roads that are seldom used, in which case vegetation may grow in the pavement cracks). This indicates that the structure has been buried (covered with soil).

[Minh_Nguyen (Minh Nguyễn)]

@tekim

Even without such imagery, you can generally apply the above definition, for example, if there is significant vegetation showing between the edge of the road surface and the structure in question, it is probably a culvert as vegetation generally doesn’t grow on road surfaces (exception for roads that are seldom used, in which case vegetation may grow in the pavement cracks). This indicates that the structure has been buried (covered with soil).

This is the case for which my stance has evolved over the years. The examples I gave earlier would technically meet your definition of a culvert, but I think they lie in a gray area where culvert tagging would be pedantic. Atop many of the culverts I’ve retagged as bridges, there is a narrow grassy berm on either side of the roadway. Maybe technically it’s a box culvert that just has a very shallow layer of earth atop it, but there’s a lot more to say about what’s on top than what’s below, such as weight restrictions, bridge numbers, and even bridge names that include the word “Bridge”.

[tekim (Mike)]

@Minh_Nguyen

technically meet your definition of a culvert

It isn’t my definition, I just said I thought it was reasonable. The NBI definition is reasonable to me as well, with the exception of the part about length - I think there can be very short bridges, and there can be culverts where the impacted roadway is rather long.

@Minh_Nguyen

Maybe technically it’s a box culvert that just has a very shallow layer of earth atop it, but there’s a lot more to say about what’s on top than what’s below, such as weight restrictions, bridge numbers, and even bridge names that include the word “Bridge”.

Even if we were to say that “box culverts” are “bridges”, there are a lot of “bridges” in the proposed import that are clearly culverts (i.e. they don’t fall into this grey area).

[Maneesh-Mahlawat]

Thank you for this insightful conversation. We have made several changes and automated the whole process. It was a lot of work and we have documented all the changes with complete step by step guide here.

Here are the highlights of the changes based on your suggestions:
Filtered out culverts: We are not editing culverts which are non-posted (do not have any weight restrictions). However, we do plan to process and mark posted culverts as bridges in OSM based on data obtained from NBI (using the columns ‘STRUCTURE_TYPE_043B’ column to identify culverts and the ‘OPEN_CLOSED_POSTED_041’ column to determine culverts that have weight restrictions). This follows guidance from @ElliottPlack that it is not recommended to put weight restrictions on a culvert in OSM.
Buffer adjustment for existing bridges: We noticed that some bridges already marked in OSM appear in our processed data. Earlier we were using a 30m buffer around OSM ways marked as “bridge=yes” to filter out NBI bridges. To minimize such overlaps, we have increased the buffer width to 80m. This follows the guidance from @tekim
Nearby bridges: As pointed out, some bridges from NBI lie very close to each other, which is physically not possible. Hence, we have eliminated bridges which lie within 10m range of each other. This follows the guidance from @tekim @Minh_Nguyen
NBI bridges far-away from actual locations: Using NHD hydrography data, we are able to determine if the bridge point from NBI is located near a waterstream. If any bridge does not have a stream nearby, we do not process it further for editing because we can’t confirm its location accuracy. This follows the guidance from @tekim @Minh_Nguyen

We have the updated code along with links to the output files within the Github repository: GitHub - mapup/edit-osm-add-missing-bridge-for-truck-restriction. I will request you to have a look once again at this document and suggest changes as you see fit. Or you can look at the output files and suggest edits.
Here is how this process works for Kentucky:

#	Description	# bridges	Data-links
1	Total bridges in Kentucky in the NBI database	14,493	Kentucky-NBI-bridge-data.csv
2	Not editing: overlapping bridges - bridges with duplicate coordinates	115	Done within code
3	Not editing: Non-posted culverts	2,316	Done within code
4	Not editing: Bridges already exist in OSM	5,389	NBI-Filtered-Yes-Manmade-Bridges.gpkg
5	Not editing: Bridges near/on freeways	0	(already filtered as existing bridges)
6	Not editing: Bridges on opposite directions (parallel bridges) at the same location	113	Filtered-NBI-Bridges.gpkg
7	Not editing: Nearby bridges	2	Filtered-NBI-Bridges.gpkg
8	Not editing: Bridges not having stream intersections nearby	59	bridge-osm-association-with-split-coords.csv
9	Editing: Rest of the bridges	6,499

Thank you!

[Maneesh-Mahlawat]

@ElliottPlack

If these issues could be avoided, then I also think we could tag the NBI bridge_refs on the section of roadway above the culvert.

@ElliottPlack Love this idea “If these issues could be avoided, then I also think we could tag the NBI bridge_refs on the section of roadway above the culvert.” How do you intend to do this? Also, we can add this tags to all bridges in the United States. This will help us apply truck restriction tags to all the bridges using data from states.

[watmildon]

Looking though all this and trying to catch up. Is there a specific artifact of the process that could use a few more eyes trying to find oddities?

I see a ton of great collaboration and want to make sure I’m jumping in at the right place.

[tekim (Mike)]

Hi @Maneesh-Mahlawat thanks for the updates, some more comments:

In the next two examples, the situation has already been mapped as a culvert. Assuming for the moment that they are incorrectly mapped and they are now to be mapped as bridges, the culverts should probably be removed (do we really want a given situation to be mapped as both a bridge and a culvert?).

In the case below the wrong OSM way has been associated with the bridge. The bridge is actually on the minor road to the east (at least according to Bing aerial imagery). The NBI does have a field called “feature carried”, and you could compare that to the name of the OSM way which you are associating with the bridge. You also might consider making a Map roulette challenge and reviewing everything individually.

In the case below the proposed import is 30 meters from the actual bridge as depicted in Bing aerial imagery:

To me, this is definitely a culvert (or some other sort of tunnel) as it is buried a significant depth below the roadway (2 meters according to 3DEP):

[tekim (Mike)]

@Maneesh-Mahlawat one other comment, I think your JS will not produce the correct results if a given OSM way has two or more bridges on it that are to be imported. It relies on the OSM ID, but once you split the way for the first bridge, the way that now should be associated with the second bridge may have a new ID.

[tekim (Mike)]

Hi again @Maneesh-Mahlawat,

First, I want to state that I think that this project is very valuable to OSM, so thanks for taking it on. It is amazing how many bridges are missing from OSM in Kentucky. However, it is important that we get this right.

Here are some more interesting cases from the data. I am looking at the bridge-osm-association-with-lengths.csv file as it existed yesterday afternoon (hopefully this is the correct file that we should be looking at).

In the case below, the bridge length differs from the actual length as shown in the Bing aerial imagery by a factor of three. To illustrate the length of the bridge as recorded by the data, I have made a way of the appropriate length.

This next case concerns the same bridge in reality, but shows that there are two bridge points here in the data (the second bridge point in the data does have a more reasonable length):

NBI has recorded part of the dam depicted below as a “bridge” apparently because it spans the spillway/sluiceway tunnel that is buried 45 meters below (according to 3DEP). NBI says “Features Intersected” = “Taylorsvl Lake Spillway.” Again, for reference, I have created a way with the length the NBI specifies for this “bridge.” "Do we really want to map this situation as a “bridge” in OSM rather than simply a tunnel?

Assuming for a moment that the culvert shown below should be mapped as a bridge, the roadway it supports certainly isn’t 141 meters long. Like in the cases above, I have created a way that is as long as the length specified in the NBI.

Assuming that this box culvert should be mapped as a bridge (I actually wouldn’t object in this case as it isn’t buried very deep, if at all, and could actually be a more traditional “bridge”), the length should not be 101 meters.

The screenshot below from Bing StreetSide shows that the above bridge is much shorter than 101 meters:

Here is a bridge that is ~800 meters away from the location depicted in the NBI, but appears to more closely match the description given. Again, I have created a way of with the length specified by the NBI for visual comparison. Although, this bridge appears to be longer than what the NBI specifies. It is already mapped in OSM.

[Maneesh-Mahlawat]

@watmildon If you are looking at the data, believe you could start with the final files. Perhaps view the data and see all the issues with bridges being added. We can figure out what specific code is creating this issue and fix it. Our goal is to get whole of NBI data for all the states in OSM starting with Kentucky. Since it is automated import, we can check and fix all issues first rather than fixing after making changes to OSM. Thanks for stepping up to help.

[tekim (Mike)]

The more I look at this data the more I am convinced that this should not be an automated import, but rather a Maproulette challenge where each case is reviewed individually. It is quite easy to find issues in the data. At this point I am not sure whether these are from the NBI or from the processing of it. Here are yet more examples.

In the case illustrated below, the NBI length is much greater than what appears in the Bing aerial imagery. It is possible that the NBI is referring to the bridge to the west in the image. This bridge, and the associated road, do not appear in OSM.
STRUCTURE_NUMBER_008=013C00075N

In the case illustrated below, the NBI length is much greater than what is shown in the Bing imagery: Like above, I have created a way with the length specified in the NBI for visual comparison.
STRUCTURE_NUMBER_008=105B00077L

The screenshot below is from the NBI record for the above bridge. As you can see it is supposed to intersect I-75 and carry Kentucky state highway 620 (KY-620), neither of which is the case for the location depicted in the first image for this bridge.

However, about 8km away there is a bridge that carries KY-620 and intersects I-75, and is of the approximate correct length. This bridge is already in OSM:

[tekim (Mike)]

More evidence that this import should not proceed (careful manual addition recommended):

38.2446771, -83.0023807
STRUCTURE_NUMBER_008=022B00018N
There does not appear to be any bridge at this location, and certainly not one of the indicated length (89.6m). I have drawn a way of this length for visual comparison.

NBI says that the feature intersected is “Grayson Dam Spillway”:

Referring to the first image for this case, there doesn’t appear to be a dam at this location.
However, 1.9 km away there clearly is a dam:

In addition, I don’t think a dam’s underground outlet (spillway) should be mapped as a bridge in OSM. However, this case does illustrate that one should look at both the specified length, and the features intersection and carried to determine whether a bridge should be mapped at the location specified by the import.

37.2273969, -87.2656148
STRUCTURE_NUMBER_008=089B00039N
In this case, Bing aerial imagery does show a bridge at the location specified by the import, but the NBI length (80.8 m) is approximately four times the actual length. I have drawn a way of this length for visual comparison:

NBI says that the “features intersected” is “use to be RR” (presumably RR = Railroad):

Given the sharp bends in this linear depression shown in 3DEP, this does not appear to be located over an old railroad cut:

However, about 880 meters to the East, 3DEP shows what appears to be a highway overpass over a railroad bed:

Here is the same area as shown by Bing aerial imagery:

37.336572, -87.803628
STRUCTURE_NUMBER_008=054B00116N
There is some road-waterway crossing here, but there doesn’t appear to be a bridge that is 81.4m long, the length specified by the proposed import. As in the above cases, I have drawn a way of the length specified by the proposed import for visual comparison.

NBI states that “Features Intersected” = “Tradewater Overflow” (Tradewater is the name of a river):

The very narrow waterway shown in the Bing aerial imagery is unlikely to be a “Tradewater Overflow”, and it appears that this waterway flows into the Tradewater River.

37.656662, -85.593673
STRUCTURE_NUMBER_008=090B00050N
Like the previous case, there does appear to be a road-waterway intersection at the location specified by the proposed import. Still, there doesn’t appear to be a bridge here, let alone one that is 71 m long - the length specified by the proposed import.

According to the NBI, “features intersected” = “Slough” (marsh):

Referring to the first image for this case above, there does not appear to be a slough that the roadway crosses. However, approximately 430 m to the South, there does appear to be a bridge if the length specified by the proposed import and which may cross a slough:

37.7025397, -87.0660547
STRUCTURE_NUMBER_008=030B00036N
STRUCTURE_NUMBER_008=030B00037N
The proposed import has two bridges in this area at the exact, or nearly exact, same location

About 450 m to the NW there is another bridge on the same highway that has the approximate length of one of the two bridges specified by the import at the above location. This is probably what the NBI record for this bridge is referring to:

[tekim (Mike)]

As opposed to continuing to look at individual cases, I started to look for bigger patterns in the proposed import.

The first observation is that there are many cases where the data preparation process has moved the NBI location a significant distance. For example, in 102 cases the import preparation process moved the NBI location by more than 500 meters, and in 29 cases it moved it by more than 2,000 meters!

38.5854685, -83.6997145 (import location)
38.601944, -83.7375 (NBI location)
STRUCTURE_NUMBER_008=081B00024N

In the above case, the import preparation process moved the NBI location by over 3,700 meters!

In the above screenshot, one can clearly see a bridge at the NBI location.

36.9312123, -88.5182525 (import)
36.91083, -88.52194 (NBI)
STRUCTURE_NUMBER_008=042B00249N

In the above case the import preparation process moved the NBI location 2,293 meters.

As shown above in the Bing aerial imagery, at the NBI location there is clearly a bridge.

36.9102061, -88.5223863
STRUCTURE_NUMBER_008=042B00247N
STRUCTURE_NUMBER_008=042B00248N

Even moving the NBI locations a short distance can cause problems. In the case pictured above, the import preparation process moved one bridge 17 meters and the other 25 meters to occupy the same location, when in reality, there are two separate bridges here as shown by the 3DEP hillshade.

The Esri World Imagery (Clarity) Beta shown above also shows two bridges here.

The second observation is that there are 71 locations in the proposed import where two or more bridges share the exact same location - probably as a result of the import preparation process moving bridge locations as illustrated earlier:
Latitude Longitude Structure Numbers
36.719673424004, -89.038317151181: 053B00039N, 053B00041N
37.489934268573, -87.495942596539: 117B00018N, 117B00017N
37.117242693894, -87.003885952907: 089B00016N, 089B00017N
36.897110145777, -86.580231876884: 114C00027N, 114C00034N
36.708704430084, -88.690148836385: 042B00093N, 042C00212N
36.884785947056, -88.550484108321: 042B00224N, 042B00223N
38.217520041029, -85.188622958580: 106C00060N, 106C00070N
37.545446678329, -82.199250609809: 098B00180N, 098B00195N
38.064938458925, -83.848522039311: 087B00037N, 087B00058N
38.397971860108, -84.649695276197: 105B00140N, 105B00134N
37.637375021148, -86.594565038168: 014C00045N, 014C00053N
37.522970144052, -84.637339562357: 069C00108N, 069C00062N
37.582181390558, -86.756692708028: 092B00021N, 092B00020N
37.624318452150, -88.130280121859: 113B00088N, 113B00084N
36.742423000000, -88.281900619737: 018B00113N, 018B00114N
37.133489547966, -83.440230090560: 066C00071N, 066C00029N
37.223088950481, -87.353745057594: 054B00050N, 054B00051N
38.585468460958, -83.699714491691: 081B00054N, 081B00024N
37.290011759755, -82.471367926402: 098C00156N, 098C00175N
37.351410272539, -87.465548380698: 054B00223N, 054B00027N
37.822437058773, -87.794825105446: 051B00040N, 051B00041N
36.597051291298, -88.871663944170: 053B00108N, 053B00109N
37.274663833216, -87.143255214437: 089C00038N, 089C00039N
37.592242697759, -84.499461961902: 040C00053N, 040C00054N
37.437899808535, -86.929116074300: 092B00146N, 092B00144N, 092B00145N
37.722192058596, -86.993474976373: 030B00128N, 030B00127N, 030B00129N
37.463407068981, -86.660764490683: 092B00037N, 092B00036N
37.820018167222, -82.888733548501: 058B00079N, 058C00016N
37.293318752678, -88.447274213415: 070B00056N, 070B00057N, 070B00055N
38.341319406124, -84.660999218820: 105C00065N, 105C00064N
37.048336229059, -84.773427356314: 100C00075N, 100C00079N
36.781739050665, -88.529110966495: 042B00038N, 042B00039N
36.725665443193, -88.232144284468: 018B00067N, 018B00066N
37.575924102746, -85.740234943490: 062B00034R, 062B00038L
38.282484767604, -84.338040330519: 009C00066N, 009C00065N
36.864932005621, -85.847457004015: 005C00050N, 005B00017N
36.910206124189, -88.522386336237: 042B00248N, 042B00247N
37.645265593433, -87.010772760179: 030B00034N, 030B00032N, 030B00033N
37.211421423005, -83.047723668701: 097B00090N, 097C00070N
37.822802009976, -85.223064650848: 115C00030N, 115C00050N
37.811685952382, -85.047663686514: 115C00049N, 115C00079N
38.205655984159, -83.772049008656: 006B00090N, 006B00079N
37.757484654868, -85.130204603737: 115B00064N, 115B00053N
38.529149623464, -83.348219150978: 068C00063N, 068C00095N
36.733160115499, -88.150970971632: 018B00118N, 018B00119N
36.971832294713, -87.641032879677: 024C00200N, 024C00199N
37.273997264650, -82.867729512408: 060C00028N, 060B00031N
37.156088067383, -87.097296688527: 089B00116N, 089B00104N
37.677297290858, -85.891319142599: 047B00144R, 047B00144L
38.528135226454, -83.908715094360: 081C00060N, 081B00016N
37.463222462160, -86.916929262422: 092B00163N, 092B00178N, 092B00182N
37.289633579008, -87.672898704820: 054B00127N, 054B00128N, 054B00126N
37.867407828103, -87.810518980426: 051B00044N, 051B00045N
38.490963058294, -83.039803968521: 045C00076N, 045C00144N
36.862418145618, -88.572620696859: 042B00209N, 042B00210N, 042B00208N
38.101972330196, -84.181383294555: 009B00020N, 009B00021N
37.476920521727, -82.429421318952: 098B00157N, 098B00235N
37.062396893151, -88.797050607638: 073B00035N, 073B00036N
38.170541512282, -85.689587710971: 056C00076N, 056C00077N
37.724342411705, -87.610961771595: 051B00103N, 051B00102N
37.062543900115, -88.799498855587: 073B00037N, 073B00038N
37.302679944885, -87.323587550133: 089B00050N, 089B00049N
37.457587754759, -85.095110007007: 078C00112N, 078C00125N
38.174638765698, -83.515493875958: 103C00114N, 103C00101N
37.152627859635, -88.945213780696: 004B00033N, 004B00034N
36.865680122951, -83.192478874376: 048B00158N, 048C00131N
37.681282909487, -85.876897326563: 047B00143L, 047B00143R
37.702539744809, -87.066054679100: 030B00036N, 030B00037N
37.459626628999, -87.850550570797: 117B00038N, 117B00041N
37.265749812919, -82.654625933708: 098C00121N, 098B00189N
37.856829047498, -85.446693461061: 090C00047N, 090C00054N

The bottom line is that this import should not be done as currently proposed. Instead, the NBI should be loaded into Maproulette and each case should be handled individually - making sure that the features carried, features intersected, and the bridge length match the location.

[Minh_Nguyen (Minh Nguyễn)]

@tekim

There does not appear to be any bridge at this location, and certainly not one of the indicated length (89.6m).

I wonder if we really should be attempting to map new bridges with accurate lengths as part of an import. When you resolve a missing bridge warning in iD by adding a bridge, it automatically guesses the bridge’s length based on average road widths. This behavior has always annoyed me – I’d prefer it to come with a fixme=* at least. On the other hand, maybe this import shouldn’t be trying so hard to make a better guess than iD, so that we can detect these bridges along with iD’s.

@tekim

Even moving the NBI locations a short distance can cause problems. In the case pictured above, the import preparation process moved one bridge 17 meters and the other 25 meters to occupy the same location, when in reality, there are two separate bridges here as shown by the 3DEP hillshade.

Maybe there should be different cutoff thresholds for lateral movement versus movement along the roadway. Even if we have to resort to a MapRoulette-based workflow for these cases, at least the initial geometries presented to contributors would be closer to the right answer to start out with.

[tekim (Mike)]

@Minh_Nguyen

I wonder if we really should be attempting to map new bridges with accurate lengths as part of an import.

We want bridges as ways, not nodes, so there has to be a length.

In any event, the bridge_length is a good indication if the import preparation process got the right bridge - which it seems it often did not.

@Minh_Nguyen

On the other hand, maybe this import shouldn’t be trying so hard to make a better guess than iD, so that we can detect these bridges along with iD’s.

If one looks at the actual NBI data (not the import data), the given lengths are pretty close to what one sees in the imagery.

@Minh_Nguyen

Maybe there should be different cutoff thresholds for lateral movement versus movement along the roadway. Even if we have to resort to a MapRoulette-based workflow for these cases, at least the initial geometries presented to contributors would be closer to the right answer to start out with.

From what I have seen the locations given in the actual NBI are close enough for a MR challenge. It is usually pretty obvious what bridge to which they are referring. However, the NBI point is often not in the center of the bridge so manual intervention is required.