Cyber attacks in the OSM space

General talk
8

This is a hyper-simplistic and wrong description of what cyber security entails. What we need is a holistic analysis of the project’s assets and threats, and a measured and thoughtful approach to data protection that protects our ability to crowd-source geodata.

The only knee-jerk reactions we have are rapid attempts to band-aid controls in when attacks are happening followed by a lull of complacency when we pat each other on the back until the next attack.

Poo-pooing an entire field of study only serves to demonstrate the project’s cavalier attitude towards data protection and lack of recognition that the world and the threat environment has changed.

9 Likes
9

should vandalism of Wikipedia articles also be called a cyberattack?

2 Likes
10

I think we should move from a discussion about semantics to the pressing concern that ZeLonewolf actually wants to stress out and that is: the OSM database has no protections whatsoever at the moment for probably sophisticated vandalism acts to come in the future that may be way worse than anything OSM has seen in the past.

So it is this:

that we should think about, not if we want to call it an attack or not.

And, as an addendum:

Even worse than vandalism acts would be a situation like - as an example - a ransomware attack against the planet osm db infecting mirrors and backups as well that would immediately halt any mapping activity - and this is a cybersecurity threat.

13 Likes
11

The OSM data is full of errors and inaccuracies, and for the mapping side we cannot prevent that without impeding the most important data source: the mapping crowd. Some measures can be taken, are in place and effective against mapping errors. Most important: given the variety of mappers, and the BYO-tags policy, there is amazing (though far from complete) unity in basic tagging. Still, the data is inherently flawed. Intentional attacks, introducing false data, is a very visible form of flawed data.

The consumer side (users of apps and applications, non-mappers, persons and organisations) require a consistent, non-flawed, stable product with verified updates. Reliable, available. Content flaws in maps are often accepted as long as it is not too obvious and does not touch the core functionality of the application.

From a helicopter perspective (where I am in the helicopter wearing a business manager’s cap), OSM data is good enough as a background map; for routing and navigation it is acceptable, but it’s way too unreliable for core functionality in other end user apps.

For OSM to open up the larger world of end user map data applications, it must offer a more stable, more reliable, more available data with quality assurance. That means, it cannot happen that one malicious common user this easily corrupts the delivered end product and hits all the end user applications. If OSM does not offer that, this larger world is not within reach. There is a strategic choice here, I am not sure if this choice has been made.

Any solution has to keep the mapper side as open as possible.

That is why I would prefer to solve this conundrum at the data delivery side. And I don’t think it suffices to point out that anyone can set up a filtering system and a cleaned-tile server, because the decision makers will then, as they do now, generally prefer a paid service with no-attack guarantees.

3 Likes
12

This is precisely why Overture has appeared. It addresses this exact need, for quality assurance of data provided to end users. What I find interesting as an economist is that the current OSMF budget for supporting the project is roughly a million dollars, while the Overture budget for building a cleaned-up version of OSM data augmented by additional data from other sources is multiple millions of dollars.

13 Likes
13

There are important differences between good-faith errors and intentional vandalism. Sometimes accidents are widely felt too, but we look at them in a different light than vandalism. We can mitigate good-faith errors in a variety of ways without affecting the distribution mechanism: user education, improved editor usability, better documentation, tagging scheme reform. We have room for improvement in all these ways, but we’ve done a good job of managing accidents in general.

The concern expressed in this thread is that we’ve been slow to respond to the rise in malicious edits with systematic measures. We have some guardrails against casual vandals, like iD and Rapid “locking” features that have Wikidata tags, suggesting some degree of notability. But to the extent that we have any countermeasures against more persistent vandals, they’ll inherently grow outdated over time.

Today, our first line of defense against persistent, high-profile vandalism is a relatively small group of elite mappers who know how to use a third-party service[1] to detect spot fires and various arcane revert tools to extinguish them. It’s easy to see how the rate of vandalism could potentially outstrip our capacity to fight it as OSM becomes more prominent and vandalism techniques also become more widely known. We can’t say for sure if or when this will happen, but planning for this scenario wouldn’t be wasted effort, because it would also cut down on the effort to fight ordinary vandalism, which we currently take for granted.

In discussions about the future of countervandalism, we’re quick to dismiss approval systems, no matter how nuanced. This is only natural, because many of us are familiar with the approval systems of other platforms such as Google Maps and we don’t want to be like them. Gating volunteer contributions behind an an approval system can fail too, sometimes disastrously. But anyways this is a sledgehammer when perhaps there are scalpels we haven’t considered yet. My suggestion would be to investigate the countermeasures employed by similarly situated projects that also need to maintain a high degree of openness despite being prominent targets for vandalism.

To pick on a project I’m familiar with, many of the technical measures against vandalism in the MediaWiki ecosystem are responses to common behavioral patterns that the Wikipedia community has identified over the years. These measures also ended up helping the project deal with other problems besides vandalism. Abuse filters block many attempts at vandalism but also block SEO spam, a scourge we’re also familiar with. CheckUser helps administrators block sockpuppets whether for block evasion or for astroturfing in a content dispute. Revision scoring not only catches subtle vandalism but also lets Wikimedia claim that they’ve been using AI all this time. :smile:

Our analogues to these tools would look different due to our data model, but they wouldn’t be impossible, and I think it would be possible to avoid a backlash over egalitarianism or transparency.

  1. Or does OSMCha count as second-party because of its OSMUS sponsorship? You get my point. ↩︎

14 Likes
14

I disagree only with this point. We are very fast to respond to malicious edits. It’s just that the tools we have available to respond to a malicious edit are increasingly insufficient to prevent harm to our data and reputation.

8 Likes
15

If you’re really paranoid you could look at the backdoor that a rogue maintainer put into xz, or at the deepfake impersonators that have been popping up recently, who could be used to disrupt the board or a working group.

16

In my opinion, the description of

is pretty much accurate and close to what’s in the Yellow Pages thereunder.

If there are studies that go beyond, it is certainly not crowding the market place, perhaps some “basic research”?

17

So true. My point is one step further: we have to accept harm to our the data, because we do want to keep free and open public editing. Intentional abuse cannot be fully prevented without closing the door, and taht would be the end of OpenStreetMap as we know it.
The question is: (How) can we prevent that seriously compromised data reaches end user applications, causing unacceptable displays (current case) and functionality (less obvious but just as easy).

End user application providers face more or less the same problem: currently, they have to assume that serious errors can be in their data source; if they refresh very often they have all the improvements fast but are vulnerable to major errors; if they delay the refreshes they miss actuality (important for routers).

Note that at this point there is no difference between intentional and unintentional harm to the data. The criterium is: how bad/harmful/unacceptable is it for the end user applications using the data.

I think providing a cleaner, more protected source would be the best overall solution. Besides providing the opportunity to prevent distribution of harmful data, there are other benefits, which are discussed in amother topic, I believe.

18

The Foundation has started to seek out financial support for a
three-tiered solution:

  1. A tool to isolate and throw out all of the edits of one or more users
    in one step
  2. A downstream database that keeps only those objects and object
    properties (tags vs coordinates) that are stable over time X. With X
    being a day, a week, or a month.
  3. More tooling to track ongoing edits

I expect those things to take two to three years to come to fruition.
The good thing about the federal nature of OpenStreetMap is that
everybody is encouraged to produce third party tools quicker.

For example, I’m working on that tool to throw out users in one go,
robust against arbitrary user stacking, partial reverts, and any
changeset shaping that is based on Overpass spitting out an OSM file
that can be opened and applied by JOSM. That is not a great UX but
definitely the best step to figure out which finer semantics makes sense
when the time for a feature in openstreetmap-website comes.

Again, that is expected to take one to three months before something
becomes usable. We have already read a lot about the general mood, and
it might make sense as a next step to actually implement things. The
more third party tools are present to be scrutinized the better we
understand the actually useful semantics.

18 Likes
19

I would note two points

  • we already have a mechanism to redact edits at a very large scale (larger than anything that has happened in more than a decade), instead of creating something new, what about an effort to clean those tools up and make them run in the current environment.

  • we have plenty of existing tooling to detect bad edits, it just has the same problem any newly developed tooling will have, it wont be run by the OSMF.

2 Likes
20

What exactly does “stable” mean? Unchanged?

21

While “throw out” is rather vague; if you mean “revert”, we’ve already got one of those. The challenge tends to be that people using individual changeset-based revert tools that inadvertently cause more problems than “doing it properly” in the first place. If we think that a community-based revert has been more successful than it actually has (as was the case with the vandalism in the eastern Mediterranean last year) then subsequent reverts are more difficult.

If you mean “redact”, I think we have a good solution there too - I found it easy to script “redact all changes by users X, Y and Z” the last few times I’ve had to do it.

For the benefit of those that are lucky enough not to have been involved in OSM in 2012, this is the “redaction bot”. It was used for removing all changes made by certain users from the database and all dependent changes. Technical changes would be needed to make that Oauth2 complient (this is something that’s already been flagged up within the DWG). However, iIf you catch vandalism early, there tend not to be many dependent changes, and “reverting to the status quo ante” prior to a redaction is already supported by the existing tools.

However:

  • I’d probably have expected to see an initial approach to find out what we as an organisation already had before doing that?
  • It sounds like you are proposing to do option 4 as described here. Is that something that the board have decided upon and do they intend to put in place all the other requirements for that beyond the merely technical (support etc.)?

How that will (or even could) work really needs discussion in its own thread…

Edit: s/west/east above.

9 Likes
22

This comment suggests that the OSMF (or maybe the EWG?) is coming up with vandalism-prevention solutions without actually talking to the DWG, who sits on the front line of combating this stuff.

2 Likes
23

It isn’t just that, anything that would/could be inserted in the flow of data post editor and pre-diff/dump distribution is only going to fly if the buy in from the OWG and operations exists. Without that, it it is even less likely to be deployed than OSM-NG, aka “not at all”.

1 Like
25

Andy, thank you for the quick feedback. If there is a surprise then this is always a reason to proactively sort out and clear any potential misunderstandings.

When it comes to being able to respond to large scale bad edits,
you confirm that you have enough tooling to clear out the database from all edits of bad mappers in short time?

The redaction bot is (beside of OAuth2) fully operational and sufficient to yield the necessary reverts to get a cleared database replicated through the minute diffs? The redactions take place strictly after the reverts, and as such it is no problem that those redactions do never get replicated?

The thread around the Mediterranian event in 2023 shall not be interpreted in a such way that the DWG has felt overwhelmed? That is good to hear, because of an overworked DWG has made a couple of people felt nervous.

I will then encourage Brian (or anyone else) to rewrite the wiki page in a way that suggests is it still a useful and used peace of software. Or do it myself if it is not going to happen. (Brian has shown competence in writing documentation and is apparently following the thread.)

Do you realistically see the raster tile stack to persist still long enough that more than CDN level tuning is justified? Paul has made good progress in showcasing that a minute diff vector stack is possible. The uniform answer about vector tiles has been that they are also production ready in any other regard. Hence I would presume that the OSMF will no longer distribute raster tiles in any form in 2027 (and maybe earlier, with no promise).

In particular many people have expressed concern that the OSM Carto style has not exactly an abundant number of active maintainers and adviced rather against repair attempts.

The part of the message that I have apparently not made clear enough: the board sees it within its mission to get development funded where no volunteer activity takes place but development is needed. This and other threads give the quite clear impression that vandalism is indeed a problem where community members expect the Foundation to help protect.

This does not mean that the board has a micromanaging plan how to implement that. It is more like that at a point the board will approach the working groups and ask them what to do with the now available resources if they come into sight.

The other possibility would be to ask the working groups to stockpile ideas just in case some money comes around. With the back story of the stockpiled vector tiles, the stockpiled GDPR improvements, the stockpiled overhaul of the data model, and other stockpiled ideas this sounded like exactly the kind of idea stockpiling that most people consider unappealing and demotivating.

So yes, I’m absolutely interested in hearing further ideas. But I fully accept that other people rather want code than talk, and therefore will restrain from throwing ideas in with no prospect of implementation.

A downstream database that keeps only those objects and object
properties (tags vs coordinates) that are stable over time X. With X
being a day, a week, or a month.

How that will (or even could) work really needs discussion in its own
thread…

That deserves definitely a thread in its own. In particular if one of the most competent people about OpenStreetMap I know (Andy, you) has doubts that it is possible.

5 Likes
26

Speaking as an OpenStreetMap Carto maintainer, we have never advised anyone on vandalism. We provide a stylesheet that turns OSM data into a rendered map. Issues like incorrect names, data, and vandalism is nothing we can do anything about.

There are issues with maintenance, but these are more complex and tie in to the state of the CartoCSS and Mapnik ecosystem. As these are off-topic for a discussion about vandalism, I will not go in to them here.

The redaction bot works fine right now. It will stop working when OAuth 1 goes away, but that’s not a huge issue - it needs a whole bunch of updates really. The problem is that the software is so specialized that there has generally been zero to one people who run it. For 99% of redactions other software does just as well and is easier to use. For the other 1% it’s easier to struggle with software you know than get the redaction bot running.

Redactions have never been replicated. This is a largely theoretical issue for content that we can’t distribute, but not an issue for vandalism.

This is also a side issue. The redaction bot is designed for redactions, which are not necessary for vandalism. A revert is sufficient. What the redaction bot was designed to do, disentangling edits from each other, is not necessary for a revert. Large-scale vandalism does not go undetected for years with lots of useful edits on top of it.

5 Likes
27

Hypothetically, suppose someone managed to inject a high volume of edits in a short amount of time (let’s assume sleeper accounts or perhaps that a large number of regular accounts that were past the new-user edit volume constraints were compromised). I’m talking to the point where the inflation of the full-history planet file and/or changeset diffs became problematically large. Is that a real potential problem and if so do we have tools to combat it?

28

It is not a problem for diff size or the full history planet file. There are potential problems around complexity for downstream consumers of diffs - some consumers find some types of data hard to handle efficiently, but size by itself is not normally a problem.

Given how the full history file is 127GB compressed, to make a big impact to its size you’d need to have a load thousands of times our current API load. It grows by about 0.7% a month right now. A vandal would have to fit in 14 months worth of changes before they are detected to make the full history 10% larger.

1 Like