It seems the “link” function of the posts, shows the number of the post counting all the posts that were here before they got separated into other topics.
1 Like
Wanting the OSM API to implement controls is an understandable suggestion. But it’s hard to make happen because there is one single software instance running which affects everyone.
Everyone who uses OSM data imports a pbf file. Can we do something at this step? Can we (as a community) generate tools which parse an OSM PBF file, and “clean it”? Remove $VANDALISM (whatever that is).
If we had a tool(s) to filter OSM data, then everyone can configure it differently. Someone can accept everything live, vandalism and all, others can be increidbly cautious. People can share configurations quickly with each other, without requiring the deployment of a new version of the OSM API.
We, the OSM community, are not providing the best to OSM users. We do not provide a simple answer to “how can I get OSM without vandalism”.
3 Likes
(at the risk of continuing to flog the same dead horse that I’ve beeing flogging for some time now):
… except the people “just using OSM’s standard map tiles in their maps”.
However, people who do “just import a pbf file” but don’t consume diffs can get an idea of the status of it by looking at the extract date. For example here there’s a section that explains where the data came from (a process I suspect you’re familiar with 
) which currently says “This file was last modified 9 hours ago and contains all OSM data up to 2024-06-22T20:21:15Z”. A glance at the all blocks page shows (as I write this**) nothing out of the ordinary at that time.
People who do consume diffs will get the “unvandalism” after the vandalism; as I’ve said before they can periodically update at a time of “no known vandalism”. The challenge that e.g. CyclOSM still has now is that they weren’t able to process the “unvandalism” fast enough and they still haven’t manually cleared their cache of it. Perhaps someone is on holiday .
** sorry for tempting fate
1 Like
Yes, but if we had a way to make clean pbf file, OSMF (or someone else) could host that data as a tile server, as well as the current approach of “use the latest”. We could even change the tile.osm.org URL to point to this new clean data.
There is no list of what these “now it’s ok to update” service. You know have to know. If there was something (eg a replication URL which would just pause if Someone™️ presses a big red button, that would also solve the “how do I get clean OSM data” issue.
I’m a little sceptical about “big red buttons”, for reasons including those covered 7 minutes into this**. I’m somewhat familiar with the stories behind most of the first two pages of the OSM “all blocks” list, and I suspect that what might count as “showstopping vandalism” for one consumer group might go completely unnoticed by another (what, someone changed contact:phone to phone?).
** yes, it’s a comedy, but the government adviser depicted was real. Viewers in the UK might be amused at a current reference at about 11:00 too…
I really need to get round to watching all of those at some point.
Open data becomes de-facto closed data when every downstream user has to invent and manage routines to make it usable.
The core data should be vandalism-free, full-stop. It is surely more efficient to deal with the problem at source, once, than to have every consumer having to manage problems.
I disagree with you on that point. It remains open-data and “everyone” can implement his* version of “vandalism-free” and share it with the “world”. This “everyone” not necessary needs to be the OSMF, it can also be you. 
*) since it might differ from usecase to usecase.
2 Likes
Yes, data post-processing and augmentation is not a central task in such an open data project.
But there is an urgent need for the OSMF to provide a central “redaction feed” as quickly as possible!
This involves central information such as reported cases of suspected vandalism and, above all, changesets reverted by the DWG!
This misses the point for a number of reasons.
For starters, at the point where the “OSM standard tile layer looked wrong” (the title of this thread) the core data had already been fixed; it was the users of tiles from OSMF’s CDN (which contained out of date data) that hadn’t. Subsequent to that a number of third-party consumers have also had problems with “vandalism still appearing in cache” many days after it has been fixed.
Put bluntly, all third-party consumers of OSM data that aren’t just “showing OSMF tiles” have a choice how they do that. They can try and apply updates on the fly so that their maps show up-to-the-minute OSM data, or they can apply updates less often, and only do so at a time when there isn’t an ongoing global vandalism issue.
When they do apply updates, they can choose to “reload from scratch” or apply minutely or similar updates (including both vandalism and reverts of it) to get from A to B. Which is best for them depends on the toolchain they have implemented and how that handles e.g. “tiles that should be rerendered” (or the equivalent for vector tiles). It is clear, based on the ongoing issues that are still being reported, that a number of third parties don’t really have a handle on removing e.g. cached metatiles showing invalid data. As a data consumer you have the option right now to only display vandalism-free data. It just just won’t be up-to-the-minute up to date.
As ever, the all blocks list is a useful indicator of “serious current problems”. Right now, I don’t know of any outstanding “vandalism awaiting removal” associated with those. There was a revert of some vandalism in Germany yesterday and in Greece today. In addition to that there’s an ongoing effort by many people (not just the DWG) to tidy up after golf mappers, Pokemon Go fake beach adders, untrained HOT newbies and many others. There always will be some problems requiring more work; these tend to get fixed in a relatively short period of time, and of course then there will then be new contributions from other HOT newbies etc.
The other reason why “The core data should be vandalism-free, full-stop” misses the point is that to do that would mean that every edit to OSM was vetted by an expert (both on OSM, and familiar with the local culture in the area of the edit). It would absolutely be possible to create a project like that, but it wouldn’t be OpenStreetMap, and it likely wouldn’t be a success (just like nupedia wasn’t and wikipedia was).
16 Likes
Agree with all of that.
Sorry, in retrospect the use of ‘full-stop’ was not really what I was trying to convey. I meant that we should be doing everything possible in technical terms, to stop obvious vandalism getting in the first place, so that the core dataset is as vandalism-free as possible, rather than requiring too much downstream post-processing. Certainly bits may get through, but recent changes have shown the usefulness of targeted measures to deal with the most egregious kinds of cases. Well done Tom and others, and to the DWG on whatever is happening behind the scenes.
PS Having a date column added to the All blocks list would be a useful improvement. I’ll add it as a Github issue if others feel it would be too.
6 Likes
/me waves vaguely at everything else that is happening (rate limits, IP restrictions, automated potential vandalism detection etc…)
4 Likes
First: What is vandalism? Depending on how you use the data, a deletion of a single route-relation can be vandalism or nothing you bother about at all. For some users a mass-deletion of “former” railways is vandalism, others would thank god for his mercy.
Second: Only the tip of the iceberg on vandalism is reverted by DWG. The majority is reverted by normal OSM-users. How would it be possible for the OSMF to publish anything of this?
2 Likes
That’s an easy one. Vandalism is an edit made by someone with intention to destroy the data.
The only problem is that osm community loves this word and uses it every time someone makes a mistake. If somebody deletes railways believing this is a right thing to do, this is NOT a vandalism. This may be an incorrect edit, but not a vandalism.
8 Likes
Some time back, I developed a detailed map showing OSM blocks but never really shared it widely, using it mostly for internal purposes. I realized it might be beneficial to others here looking for similar data.
Feel free to check it out and let me know if you find it useful.
Here’s the link: OSM User Blocks @ ResultMaps.neis-one.org
1 Like
That definitely shouldn’t be available to non-loggedin users (aka such that have accepted the ToS), but it seems it is.
I’d suggest that this post (and replies to it) would probably be better in a new topic because I can think of lots of comments about it, but they don’t really belong in this thread.
1 Like
I do disagree with that.
Making a wide-ranging edit without bothering to understand community norms is vandalism. If I paint the 12th century church down the road purple without asking anyone, that’s vandalism whether or not I think it genuinely looks better purple.
As an OSM example, in the UK we occasionally encounter muppets who decide they know better than the settled will of the community on highway tagging. They therefore take it on themselves to extensively retag a bunch of roads, and generally get a hundred or so in before anyone notices. Wikipedia calls vandalism “damage to public or private property”; blatantly overwriting the will of a community with your own opinions fits that pretty neatly.
7 Likes
As I have written before, this site was initially created for me personally because (as so often) I found it interesting. If there are concerns about its publication and/or accessibility, I can “hide” the site again and there will be a discussion about possible conditions for access to the site. This can also be done in a separate thread.
It’s not the same situation. You KNOW that painting the church is illegal and not accepted by the community. Similiar situation actually happened, there was a time when a lady painted over the Jesus’ picture. If I recall corrctly she was punished for this and this was considered as the act of vandalism. Even though her intention were initially “good”, she wanted the renovation, but she was fully aware that what she was doing is forbidden.
If the person that changed railways was informed that his actions are wrong and is still doing this - yes, this is vandalism. If he wasn’t aware, not really. That’s why we approach the user first, and then make further actions, not just immediately ban people for one bad edit. Unless it’s obvious–there is no way someone would draw a sheep and tag it as motorway thinking this is correct.
You would also need to consider the impact on the user that made a mistake. What would you feel if you enter the forum and there would be a conversation about one edit you have made and it would be claimed vandalism?
PS. You have deliberately removed one word from Wikipedia’s definition of vandalism
2 Likes