I’m noticing that while almost all the tile layers on openstreetmap.org currently look free of large scale vandalism to me, the CyclOSM layer is still massively vandalised on every zoom level. Are those tiles not being rebuilt? Or rebuilt slower? Or is that a different dataset that is still vandalised? “Query features” doesn’t show any of the vandalism. Ctrl+F5 or clearing my cache doesn’t help.
Edit: thanks for the answers. I didn’t know those tiles came from a different organisation.
Their tileserver got totally overloaded by the vandalism and choked. They do use the data diff streams of OSM to automatically update. This happened to other downstream diff users that do automatic updates from diff stream as well depending on how the downstream data processor handles this and how powerful the systems are. Remember: CyclOSM is run by a local chapter (OSM FR) that is also build on volunteers.
If you mean that when someone wants to implement a rendering delayed by a week they could configure their own server by their own taste - then I fully agree. OSM is a freely available database and people are free to configure their own rendering engine as they please.
Otherwise: no, OSM does not offer a free generic public rendering service. There are many providers where people can buy that, with whatever delay amount available.
Or they can use Facebook’s Daylight Map Distro (filtered planet). (I don’t know how actual MaRS is, but you may have that as well.)
@SomeoneElse by the way do we have any profanity based alerts on edits? Seems pretty useful in the current situation to automagically alert DWG, and it’s highly unlikely that someone genuinely need to add name=<CENSORED> street anywhere. 
(In general alerting is the soft alternative of rejection, if you let me to state the obvious here.)
The cyclosm project is (a) independent of OSMF and (b) aware of the problem. They’ll need to sort it out at their end.
No ****ing comment
Should a tile provider which is unable to deal with sufficiently fast wiping of vandalism still be featured on the OSM home page? Should such a requirement be added to the criteria for inclusion of layers?
Open whatever map is independent of the OSM and aware of the project. “They’ll need to sort it out at their end”.
OSM has a long history of genuine and small projects showcasing our data. These projects prides themselves to keep their data minutely up to date. Yet they don’t necessarily have the manpower to handle data filtering upstream.
Filtering minutely diffs for vandalism, is it even a thing, I mean a thing happening, not theoretically?
Maybe those days are over when anybody could invent new stuff on OSM data with their flaws here and there, but trusting they’re good overall. But I think the OSMF should try to take a stand and explore alternatives to patching data after the fact, and propose them to the community to choose a direction.
I’m no friend with the current tile layer criteria as I think that some of those criteria (or the pride of minutely updated data as @yvecai describes) even fosters the occurrence of vandalism in layers provided by third parties like OSM FR (e.g. the CyclOSM layer or the HOT layer).
I do not think that minutely updating a tile layer is needed for anything but mapper feedback. So this is good for the standard layer by OSMF as that is used as mapper feedback, but not the other layers that are a showcase of what OSM data can be.
It is also no good idea that the OSMF mapper-feedback layer is spilling the milk by trying to also serve a purpose it is not useful for (at least in times of vandalism waves or even with smaller problems like a drained lake Erie etc.)
I rather think that a monitored and controlled update of data is a better solution for not-for-mapper-feedback tile layers that are used for viewing a map to not let the vandalism creep in.
After the vandalism bad press disaster in 2018 some Mapbox maps needed many months to show updated data from OSM as QA got and is forever since then way more important than actuality. This is too long (and different today as their QA pipeline got faster) but finding the right balance is the key in running a useful tile layer for a map to be viewed.
It’s totally different for a map that has to provide mapper feedback (and gratification) like the OSMF provided tile layer does.
Maps should update for sure but it is always better (for a not-for-mapper-feedback map) to show either an outdated tile without vandalism or no tile at all than to show a tile full of vandalism and foster the bad actor in his doings.
I managed OK with this. The actions that I’ve had to take have been:
You could argue that I’ve got “inside information” in that I’m actively fixing the problems as they come in, and perhaps we should report “known vandalism activity times” more clearly, but other than that CyclOSM could do the same as I did (particularly, delete invalid cached metatiles).
Hi, I’m using the Nominatim-API, yesterday Germany was referenced as “Nazi-Staat” for a short while - related?
That was a small copycat offender, see here: Achtung: weiterer Vandalismus im Gange... :(
For geocoders etc. that uses OSM data everything I said above in regard to tile layers applies as well. The Nominatim instance by the OSMF used for www.osm.org should update minutely for mapper feedback/usefulness, others should not but do QA. But for geocoders vandalism is not that much a problem in comparison to tile renderers as updates are fast and you do not have the overloaded toolchain that a tile-renderer may have (or depending on the system, overloading is less likely for a geocoder, it still is a bunch of work (manual, by perfect tools and also in compute power) to e.g. update while “overjumping” the vandalism state).
We’re investigating exactly this for Americana. Thankfully the render server’s failing disk controller saved us, ironically, from rendering the vandalism. I anticipate that it would be quite straightforward for “planet as a whole” render pipelines to keep one or two prior planets to fall back to.
Yes, much easier for vector tiles than raster tiles (but still quite more effort if you want effective, fast vector tiles as one of the biggest problems in the raster toolchain at OSMF is not just the rendering in the tilelayer toolchain but the cdn as well, cdn cache invalidation, etc. pp. The same applies to a lesser degree to e.g. CyclOSM or HOT that also have a small cache in front or any other tilelayer that works with a cdn or sth. similar).
We used to have essentially that. The Mapnik layer updated on Wednesdays, while the cruder, more comprehensive Osmarender layer updated within minutes of saving a change, faster for those who knew how to work the Tiles@Home site (the one that later became a FOSM renderer). I wouldn’t mind turning the clock back to those days, except it was actually excruciating to wait a whole week to see my changes validated by the nice-looking style, only to discover that I got the winding order wrong and had to mop up a flooded tile. But hey, at least if we want busways to render, there would be very little to stop us.
do you remember the moment when mapnik rendering switched from weekly to “instantly”?
That might be the case, though I have my doubts. Editing the border of pick a larger country will result in a bounding box almost as large as a continent. Same goes for changes to other larger natural objects, like lakes between US and Canada. Just to name a few.
Even without the above, I would assume if the limit is x km2 in size or y in changed objects or max. movement of z km, just the attackers will adjust accordingly and instead of one bad changeset, there will be plenty of them. That’s not a solution to the problem, only a stopper for mappers.
Such kind of attacks can only be stopped by detecting bad changesets “on-the-fly”. Like filtering them and changesets with a certain threshold they get manually reviewed before they getting applied to the database. Or you accept everything to the database and somehow flag the diff-files. Like DWG or some group of mappers or everyone could flag a diff-file as “bad”, to warn others not to use that diff file. Or even the diff-creation could be stopped if vandalism was detected and after the cleanup would be restarted.
Not necessarily that needs to be operated by OSMF, though I would think OSMF should drive/support it. The weakness of OSM in this regards is, that we are (as far as I know) only able to detect vandalism only visually / manually. Means like, we are only able to detect it, if it’s happening in larger scale and if it’s visible e.g. in OSMcarto. But not all the OSM-data is visualized, but used e.g. only in Nominatim. So if someone changes a couple of alt_name, it would take quite some time until we detect it.
Everyone, please repeat after me:
Yes, it was the whole map.