I don’t want to jinx things and say “nice save,” everyone (too early), but nice save.
Eternal vigilance is what it takes to keep doing this. Please, take a bow, everybody. Adults learn at age 20, we’re fine. Smarter, wiser, fine.
1 Like
this threat actor severely affects at least two pillars of infosec, integrity and availability. the short term impact, at least, is a lot of people wasting their time trying to mitigate or nullify their malicious effects, instead of using their time to add new data or other type of valuable contributions. at mid term or long term, it will or could affect how people or at least new contributors collaborate with OSM, which is one of the intrinsic values of the project.
it affects the project as a whole, mining one of the key features that is was that any user could start adding data from the beginning and without limitations, because to protect the data from the threat, the project is forced to impose limitations to legitimate users. Some kind of user classification could be done, so each user is assigned to a level, and he can level up proving that he deserves it for the contributions he has made. but I think of me, getting started in a project like this, and it would be completely different from the OSM when i started +10 years ago.
4 Likes
Not only should this be considered a restriction for new users, it should be considered a restriction full stop.
The API ought to implement a hard reject for each of the following scenarios. Even legitimate mappers would not be doing them. It is surely unacceptable that our increasingly mature map allows continental-level changes to anything whatsoever.
-
Moving of any Node or Way beyond a relatively short distance. I cannot think of any scenario when an object should legitimately be movable more than a few km. Real world objects (perhaps except shipping routes) simply do not change like that; the API should therefore reflect that reality.
-
Drawing of any object (other than boundaries and shipping routes) over a very long distance. Nothing else legitimately spans these distances, surely.
-
Changesets whose bbox is larger than a continent (and certainly worldwide). Individual changes should never span continents, and the wiki already makes clear bboxes should be small, as otherwise changes are poorly-scrutinisable. Correctly-implemented mass-edits similarly are not supposed to be creating bboxes of large size for the same reason; scripting should be parcelling sensibly by area.
-
Any changes to boundaries (which are one of the few long-distance objects) by users with a low number of changesets. Even if legitimate, this is a skilled operation, with high risk of breakage, that no inexperienced member should be permitted to do.
Would any of these actually be difficult to implement in coding terms? Are there really any scenarios when they would interrupt legitimate mapping?
I would propose the numerical limits for the above could be set reasonably high (e.g. prevent continent-level stuff) to start with, and then gradually reduced as experience determines cases like genuinely long roads through rural / desert areas.
12 Likes
Changes to the map aren’t showing up for me except at the nearest zoom level, even with a force refresh. Is something going on?
If you’ve got an exception then we’ve got to have a pre-approved list of tags. Pre-approved tags in general goes against “any tags you like”. I think we also have long subsea cables, time zones etc. etc.
This prohibits any edit to the France relation, the main USA relation, the aforementioned timezones, anything that spans the anti-meridian and probably other tings.
I think the current rendering process marks tiles dirty when an edit affects them and puts them in a rendering queue. My gut instinct at the moment is that it would be good if a changeset bounding box over a certain size didn’t trigger tile re-rendering. That way the criss-crossing would only appear when another, more local, edit occurs and fewer overall problematic tiles are rendered overall. Of course this might just lead to spurious “local” edits too to boost visibility.
3 Likes
There will be unfortunate tradeoffs to almost any strict limit we impose on not-yet-trusted users. Done well, user limits can be a minor nuisance, akin to this forum’s restriction on posting more than three links in a post. (New users often chafe against that restriction, but it’s for a good cause.) The key is to design limits that well-meaning newbies can intuit and sympathize with, and offer recourse to those who feel unfairly hamstrung by these limits due to their unique circumstances.
In various recent threads about vandalism, I’ve been careful to advocate only for technical measures that operators can configure on the fly in response to current threats. Although general tools are more difficult to picture and probably more difficult to implement, I think it does more good than to brainstorm specific heuristics and their false positives and negatives on this forum. These vandals are likely sophisticated enough to know how to find this thread, not to mention any copycats who come along in the future.
6 Likes
I’m afraid the word is out, how to damage OSM. Maybe even toolkits. People will be doing this, even without “messages”, just for “fun”. They get the same gratification as mappers get: immediate results on OSM-based maps.
So far, I have not seen suggestions that can effectively stop this kind of vandalism. Probably the current crude MO of these “Long roads” attacks can be made harder or even impossible, but can those measures prevent other types of damage? As attackers get more knowledgeable, they will find other ways AND spread the word how to circumvent the measures.
Yes, I’m preaching doom, and I hope I’m proven wrong. Anyway, an essential part of fighting this will be to stop the instant gratification and its comet’s tail of lingering debris.
Any API restriction that is based on the current tagging of an object is trivial to circumvent.
A general API bounding box size check is likely to happen, there are a couple of minor difficulties and some disagreement exactly on how this should be done, but I expect there will be something in place rsn (at which point in time the vandal will simply change how his attacks works, but at least such a restriction is useful outside of the current issue).
7 Likes
And there won’t be. It’s impossible to prevent 100% of bad edits in osm, as it’s also impossible to prevent all vandalism or crime in real life.
To make a comparison - if you leave a house, you close the front door using a key. Is it making your house an unconquerable fortress? Of course not, there are still a couple of ways to enter, but this makes it difficult enough to many people to enter your house. Probably everybody will agree that closing the front door is a good thing.
The same goes with vandalism protection. Osm protection right now is on the level of the “open door”. Anyone can vandalise even without any actual IT skills. Making restrictions will raise the level of protection to “door closed with a key” - far from perfect, but good enough to block some vandalism acts.
8 Likes
“Undoing vandalism” is the reason that I most often want to do it
4 Likes
Yes, 399 posts into this thread, something is definitely going on 
You may find that other map layers are able to respond more quickly to your edits - have you tried looking at some of those?
1 Like
What if the main map on OpenStreetMap.org was only updated once a week, and with a delay before tile generation that is long enough to give us a chance to spot vandalism - while a separate map for instant mapper feedback is visible only to logged in users?
Basically Should OSMF offer free raster tiles for end users (not for OSM map QA)?, plus those raster tiles being the default layer on OpenStreetMap.org
Here’s a legitimate examlpe were an area was moved from one city to another:
It can happen that an OSM Object moves for a long distance, and potentially across continents.
3 Likes
Not difficult to implement at all, but I think it’s fair to say that there is a difference of opinion about how urgent such fixes should be:
The main map is provided for the benefit of mappers anyway, making it “more stable” is treating it like a product for general use, which it isn’t. If commercial or “public facing” tile servers want to implement a policy that an edit must survive intact for a week before being rendered then that would probably be sensible as a crude vandalism filter. The logic for that probably gets a bit involved with the way our elements all reference each other, but it seems fairly reasonable on the face of it.
4 Likes
I hear that assertion a lot on this forum, but is it really true? Looking at osm.org I see no indication that it’s merely a tool for mappers. Instead the welcome message on the website says “OpenStreetMap is a map of the world, created by people like you and free to use under an open licence.” So the standard layer is not just a tool for mappers, it absolutely is public facing, in fact the OSM Carto maintainers describe their style as “a major part of the public face of OpenStreetMap, for many people the map on osm.org rendered with this style is OpenStreetMap”
If the public face of OSM gets vandalised that’s a much bigger problem for us than some internal mapper feedback tool getting vandalised.
I am not suggesting any sort of complicated QA logic. Just something like a separate set of tiles that is generated weekly from the planet dump. The dumps are released on Fridays, but only contain changes up until Monday, so by that time it should be clear if they contain vandalism. If they do, just skip that weekly update.
6 Likes
This is what a lot of downstream data processors and/or tile providers do in one way or the other to not have the vandalism in their systems or tiles. The misuse of the QA layer (OSMF minutely updated tiles as a mapper feedback tool) as a map tile provision elsewhere where this would not be needed is the mayor problem of giving a vandal actor the exposure he wants. But this has been discussed elsewhere here to no avail. It’s probably best to keep that question in mind but not to open it up here again. I think a lot of people in the OSM community just need some stress-level rest before the community of OSM goes into any strategic decisions (also in regard to the upcoming vector tiles).
The OWG (and others from the community that help them through pull requests or code improvements) is doing a tremendous job of lessening the impact of vandalism waves every time they occur through analysis and finding solutions post-incidient but this is a cat-and-mouse game. We can all be glad that e.g. the methods used in last year are not available anymore for those vandalism acts because of the improvements in APIv06 and in the rendering/updating toolchain used by the OWG.
1 Like
and this is why waze is a crappy project with crappy people. Basically the whole map is locked, and only the cabal can edit it, and on one hand they keep cryping that they need more editors and on the other they tell newbies how stupid they are and they shall beg for edit anything.
I have tried to help them (despite Google above) several times but gave up due to the arrogant elitist closed circle approach.
I am definitely against the general idea.
3 Likes
I did explore that suggestion in a different topic here (although it’s not something that I’m personally in favour of - OSM already makes map data available to everyone; why should it also provide free map tiles to big commercial organisations too stingy to run their own servers?).
In reality OSM is a large project and the people that are part of it hold many different views, including “the main osm.org website shoudn’t have a map on it (or at least not be mostly map)” and “osm.org should try and replace Google as an end-user maps website, but without the advertising revenue”. My personal view (see e.g. here) is closer to the first of those than the second.
I think we should continue to remind the people misusing OSM’s standard map tiles in their commercial products what it says in the tile usage policy:
OpenStreetMap data is free for everyone to use. Our tile servers are not.
9 Likes
Or like I would say: if providing mapper feedback is a “milk bottle” and serving tiles to third party commercial sites/apps/software is a “sledgehammer”, the OSMF tile layer is a perfect milk bottle but it is not a good idea to offer that milk bottle as a sledgehammer substitute. (Let’s say that “milk” is “osm reputation”).