Resolve linting issues in Netherlands

Hallo van het Mapbox Team!

Het spijt ons eerst, maar we spreken geen Nederlandse taal. Dit is een diepgaande vertaling van dit bericht.

Als onderdeel van het lopende werk om de kwaliteit van OpenStreetMap data te verbeteren, hebben we onlangs osmlint-osmium (https://github.com/osmlab/osmlint-osmium) en osmlint (https://github.com/osmlab/osmlint) gebruikt voor het opsporen van problemen met betrekking tot de gegevens van het wegennet in Nederland. In totaal zijn er 2432 issues gedetecteerd over 10 soorten linters.

Als volgende stap is ons team (https://wiki.openstreetmap.org/wiki/Mapbox#Current_Data_Team_members) van plan een deelverzameling van de detecties te herzien om een beter inzicht te krijgen in het soort problemen, en ook om eventuele geldige gegevensproblemen direct in het OSM op te lossen. U kunt meer lezen over ons plan in OSM Wiki hier (https://wiki.openstreetmap.org/wiki/Organised_Editing/Activities/resolve_linting_issues_in_Netherlands).
En hier ons ticket in Github (https://github.com/mapbox/mapping/issues/366)

Ons doel is om voor het einde van volgende week te beginnen met de herziening. We stellen uw feedback, uw vragen over dit project en uw lokale inzichten die ons helpen om de gegevens beter te begrijpen zeer op prijs.

Met vriendelijke groet,

Lid van het Mapbox-team, Vlada


Hello from the Mapbox Team!

First we’re sorry but we don’t speak Dutch. This is a deepl translation of this message.

As part of on-going work to improve the quality of OpenStreetMap data, we recently used osmlint-osmium (https://github.com/osmlab/osmlint-osmium) and osmlint (https://github.com/osmlab/osmlint) for detecting issues related to road network data in Netherlands. In total, there were 2432 issues detected across 10 types of linters.

As a next step, our team (https://wiki.openstreetmap.org/wiki/Mapbox#Current_Data_Team_members) is planning to review a subset of the detections to better understand the type of issues, and also fix any valid data issues directly in the OSM. You can read more about our plan in OSM Wiki here (https://wiki.openstreetmap.org/wiki/Organised_Editing/Activities/resolve_linting_issues_in_Netherlands) and Github ticket (https://github.com/mapbox/mapping/issues/366)

Our aim is to start reviewing before the end of next week. We would really appreciate your feedback, any questions you have about this project, as well as local insights that you think will help us better understand the data.

Cheers,

Member of Mapbox team, Vlada

Welcome at our Dutch forum

First remark…

You are using Satellite Imagery: Bing, Maxar, Esri These images are outdated in the Netherlands .

BING has also a substantial deviation in some areas !

In OSM the Netherlands we use PDOK Actueel_ortho25_WMS

Please contact the local former mapper in case the former changeset is recent.

Greetz,
Eggie

Welcome.
Remark 2 :wink:

Please be careful around building sites.
Even Dutch aerial photography (PDOK) is not as fast as local mappers. Check change-dates in OSM against PDOK imagery, or ask local mapper.

Greetings,
Jan

Even voor mijn (e.a.??) begrip: wat zijn ‘linting issues’?

Fouten gevonden door een ‘linter’; softwaregereedschap waarmee automatisch problemen gevonden kunnen worden. Denk aan de issues die Osmose vindt.

I take it you speak English? Please feel free to post in English, and don’t bother with the machine translated Dutch. You’ll find that most — if not all — of the Dutch mappers that frequent this forum are well-versed in English.

Looked at some of the Cross Highways, but the first 3 I checked were OK, viaducts with layer=1, road crossings with one of them @ layer -1.

What are you planning to do with these errors?

edit:

Same with Impossible Angle, some are really this sharp, some are an issue.

Again, what are planning to do with these?

I corrected a few ‘impossible angle’ issues in Limburg.

I’m now doing Cross Highways from the south (Limburg) working upward.
Can you update the files? Maybe daily?

Thank you for all your messages!
We will use the satellite image which you advised where it possible.
Our editors will check every case separately and if there’s no need to edit we will left it like it is, or if we won’t have enough ground truth sources we can put a note to the map with our question.
We are not going to update linters for now. But maybe you can do it by yourself, there’s some kind of instruction https://github.com/osmlab/osmlint.

Cheers,

Member of Mapbox team, Vlada

Why not?

You’ve asked for feedback and received it, and local mappers have shown a willingness to engage with your linter-reports by fixing errors. We are showing you where the linters are broken in response to your request for feedback. By fixing them and providing clean linter reports you can prevent your own mappers from making mistakes that might result in lots of reverted edits, angry local mappers, and possibly even banned accounts as well.

If fixing one linter bug means you don’t have to manually judge hundreds of false positives worldwide, then that is a sensible investment of your time. Using broken linter reports as input for your own mappers is not a responsible course of action.

We’ll see what can be done with this and whether we have enough resources for it. I’ll give the answer during the next 2 days. Also you can offer your improvements in this repo https://github.com/osmlab/osmlint by creating a ticket, or we do it after our review.
Thanks for supporting our project!

Cheers,

Member of Mapbox team, Vlada

You know, I was always using keepright but I think this is more convenient to use. As far as I’m concerned files like these should be made with a daily update for every country (or region if it is a large country). Also because keepright is not updated very frequently.

Would be nice if someone would give the resources for that. But still thumbs-up for the tool.

Vlada, I’ve got the osmlint working at my box, but the output is a bit different. It seems that every single item found should be a separate geojson file, whereas your files have multiple items in one geojson file.
Is there a trick to get it that way? I’ve just used the command like in the processors examples with a redirect of the output:

$ osmlint missinglayerbridges --bbox="[6.34,50.75,7.22,53.6]" --zoom=12 netherlands.mbtiles > missinglayerbridges.geojson

Maarten, here is a handy tool that can help to convert *.json into *.geojson or combine several *.geojson into one.
https://github.com/osmlab/osmlint-helper

osmlinth -g unconnectedhighways.tofix.json > unconnectedhighways.geojson

Thanks, that does the trick.

Other question: osm-lint works on you mbtiles files. I downloaded the one for the Netherlands from http://osmlab.github.io/osm-qa-tiles/country.html and it is 7 days old. Is it updated on a weekly schedule?
There is also osmo-lint. Does that have the same functionality with all the validators like osm-lint, but working on the pbf files?

osm-qa-tiles service is out of the influence of Mapbox and now is running by HOT. You may find instructions on how to prepare your own mbtiles for QA here - https://github.com/osmlab/osm-qa-tiles/tree/osmium. Also to check raw *.osm.pbf take a look at https://github.com/osmlab/osmlint-osmium

Hi!
As I promised:

Unfortunately we haven’t such possibility to update linters.
We appreciate the work of the community and do our best to listen to comments. Our efforts, as well as the efforts of the OSM community, are aimed at expanding and improving the quality of data in the OSM database. Unfortunately, mappers sometimes make mistakes in tagging and topological connections of objects. We used open source tools (OSMLint, OSMLint-osmosis) to make sure that most of the OSM data are free of errors. The low volume of errors identified allows a high assessment of the quality of the data and community efforts in the OSM in the Netherlands. Based on this, we do not plan to carry out regular (daily, weekly or monthly) data quality checks and the publication of identified deficiencies. However, we will follow internal procedures to ensure that our clients receive services they can trust. If someone from the community wants to carry out such checks on a regular basis, he can study the instructions in the linter repositories, as well as the instructions for converting data for verification - https://github.com/osmlab/osm-qa-tiles/tree/osmium. All data and tools for their verification are open source and available to anyone for use. Thanks again to the community for their contributions to the OSM development.

Cheers,

Member of Mapbox team, Vlada