The new rate limiting prevents participants to Missing Maps mapathons from saving buildings

Foundation
, , ,
1

Dear OSM Data quality working group and whom it may concern,
On behalf of Missing Maps (MM) and Médecins Sans Frontières, member organization I serve as a MM community engagement coordinator for, I would like to report that the recently imposed rating limit (OSM Community Forum thread How about limit new accounts?, Github issue 1 and 2) on new mappers has prevented the participants to several of our November mapathons from saving edits in good faith. We train our mappers, support them individually as needed and guide them to ensure as good data quality as possible throughout the mapathons, so we believe this anti-vandalism measure should not limit them from adding buildings in areas of communities vulnerable to crises. This has created a problem for the organizers of at least 5 MSF mapathons across South Korea, Czechia, Spain, as we cannot provide a solution, when it stops the mappers from saving their data. For example, experienced mappers in my community note that all new mappers at a secondary school mapathon reached the limit. At a mapathon where we teach JOSM right away, they were mapping also round buildings in West Darfur, Sudan, torn by conflict, and this limit was reached very quickly because they count as many more changes. Therefore we would like to collectively appeal for a solution that does not have the adverse effect of stopping useful data from being added to the map. I also fear this inconvenience may stop enthusiastic mappers from coming back to map, as not being able to save well-digitalized buildings can be frustrating. This is what we propose as possible solutions:

  • We agree with others that the limit should be higher. Easy fix could be tripling or at least doubling the limit.
  • Would it be possible to set a limit specifically on deletions?
  • Would it be possible to exempt saving changes in JOSM? Of MSF mapathons those which teach JOSM to new mappers are done by very experienced mappers, who assure the quality of the data. They validate most of them during the mapathons or right after. We never had an issue of mass deletions (for sure in the past 4 years I have been around).

Thank you for considering this, as you take further steps to combat vandalism. We are with you to uphold the open spirit of OpenStreetMap and its value to all.

9 Likes
2

that is definitely not feasible, vandals can also use JOSM

sadly, vandalism is not limited to deletions

some vandalism waves mass editing thousands of names to include slurs/insults/informations that were not name of given object etc

or deleting names from 400 000+ objects.

It could help to provide few minutes of screen recording of an example of affected session. At least minute-two of editing resulting in rate limit blocking edit (I use Peek on Linux for screen recording, there could be also other tools)

Not entirely sure what kind of edits gets blocked and whether it should be blocked.

Looking at edits from new accounts almost triggering rate limits would also help to judge situation.

Note comment

Where new users map their first changesets very quickly, and especially when they do so in JOSM, there is a clear correlation with poor-quality editing. A significant subset of DWG tickets created about new users have simply been about poor editing quality, and our advice is often “slow down, and make smaller but better quality edits”. This has been more noticeable since the recent waves of vandalism where large number of objects were edited (or deleted) in a short period of time - we (the DWG) have been using various means to detect this, and in a few cases have found ourselves dealing with poor quality new editors who just needed to be encouraged to slow down a bit.

from Add the ability to rate limit edits by tomhughes · Pull Request #4319 · openstreetmap/openstreetmap-website · GitHub

2 Likes
3

My comment at github is relevant as a reply to most of what you wrote, even though I wrote it 13 hours before your post here :slight_smile:

To address some of the specifics:

Would it be possible to set a limit specifically on deletions?

No, that wouldn’t address problems such as “moving the Mediterranean Sea 30m to the west” or “changing all of tag X with a profanity”.

Would it be possible to exempt saving changes in JOSM? Of MSF mapathons those which teach JOSM to new mappers are done by very experienced mappers, who assure the quality of the data.

My experience, both from within the DWG and more generally, is that new, large edits made with JOSM are often poor quality. As I said last night, the DWG’s advice in such cases is often “slow down, and make smaller but better quality edits”.

They validate most of them during the mapathons or right after.

My experience is that that does not always happen.

We never had an issue of mass deletions (for sure in the past 4 years I have been around).

I also can’t think of a mapathon contributor “deleting everything in an area” either, but I’m sure we’ve had reports of “mapathon person X maps some buildings and roads with poor quality; a couple of months later mapathon person Y deletes X’s contributions and adds their own, of a similar quality”.

What might help your case is if you can give examples of the edit sessions that were rate-limited so that people can see what sort of objects it was that inflated the object count and caused the rate limiting. In the one example posted on github mapping non-circular (but not rectangular) buildings as circular seemed to be one factor.

It would also help if you can link to the organised edit pages for each activity that was affected (I’d expect an entry for each here), so that we can see what participants were being asked to do, and also explain why that is useful for MSF etc.'s work more generally.

11 Likes
4

At least one example changeset I’ve seen consisted of lots of 20-node 2 metre diameter buildings which naturally exacerbates the issue. But even with less nodes you are not going to gain the one order of magnitude that you would likely need to avoid the limit.

The reality is that HOTs / MissingMaps classical business model is just at odds with slowly growing mappers. I actually thought they had more or less stopped using it, but seems I’m mistaken.

5 Likes
5

I will add my 2 cents… I think that anyone who is Active Contributor Membership – OpenStreetMap Foundation should have ability to “vouch” for new users this can either be done via registration invite code, so “Active Contributor” will have autogenerated links “referrals” that s/he can send out to participants of hackathon, but risk being banned if new users that s/he vouched for misbehave(vandalize). Another way would be after normal registration Active Contributor could vouch for any account, similar as reporting UI, just instead of “Report this User” would be “Vouch for this user”.

I know this complicates things, but we will probably have to keep moving in this direction to be able to combat vandalism and balance with HOT and other similar contributors…

17 Likes
6

From what I have seen, the new users do not map features quickly in JOSM their first day, because the intro to JOSM training takes almost the whole time at our Missing Maps CZ mapathons :slight_smile: Yet, they reported to have reached the limit on Wednesday, 6 to 9 pm CET, with round buildings in West Darfur HOT Tasking Manager. I will try to ask for user names next time this happens.

2 Likes
7

We are slowly growing mappers in our community (even though lots of the MSF mapathon participants do not come back, admittedly, and if not all of the changes of the one-off participants are not saved, I suppose so be it… but it is a shame if it stops from coming back those who could grow to be good mappers). What do you mean by Missing Maps business model? There is no business model, I would say after 4 years of being actively involved in Missing Maps :woman_shrugging:t2:

4 Likes
8

Thanks. Can you provide the other information requested too - details of organised edit pages for each activity that was affected, and why that is useful for MSF etc.'s work more generally?

9

Hey,
I think this is not an easy situation for OSM at the moment and preventing vandalism should be our main focus. My impression is that some of the comments here go to wide.

Whether or not you agree with way how Missing Maps and projects from other humanitarian organisations organize volunteers to map buildings and roads in OSM, I think that we all agree on the fact that at the moment we want to prevent vandalism.

Getting the rate limits right is a difficult task. And probably we should all assume that the first limit we set, can never be perfect. That’s why it might be feasible to adjust these limits.

@SomeoneElse and others who definitely know more about the vandalism issue. Do you already have any numbers on the effectiveness of the implemented measure to stop the vandalism?

I think that now we have a better understanding of how the limits affects the people mapping at mapathons. It would be nice if you can share some numbers and insights on the effects on vandalism as well.

Once we have both, I’m pretty sure that we will find a way to adjust the limits without decreasing the value they bring to prevent vandalism.

8 Likes
10

Hi, on the organised edit page, we are just under Missing Maps and Missing Maps CZ SK, there is not a single entry for each mapathon. Nonetheless I will attempt to list the details of the sessions where it happened by group and date and can provide HOT TM projects as well (the reporting of the trainers and event organizers of the communities is not detailed enough to know how many users it affected, or which OSM accounts):

  • MSF Korea mapathon Nov 16 - several users

  • MSF validation mapathon Nov 16 5-7pm UTC - 2 new users (they were the only new users)

  • University of Sevilla & MSF Spain Nov 22 5-8pm UTC - many new users

  • SPStav Plzen mapathon (in cooperation with Missing Maps CZ and MSF CZ) Nov 24 2 to ? UTC - all new users

  • Missing Maps Brno mapathon Nov 29 5-7pm UTC - several new users

There are still eight mapathons planned over the next 2 weeks that will normally map MSF projects, with the aim to update or create useful OSM data for our medical humanitarian operations. We worry that all of them will face the rating limit problem. What am I to tell the mapathon organizers to do to avoid for their participants to run into the problem?? I already explain the reason for and existence of this measure and recommend to emphasize quality over quantity, and saving every few buildings.

7 Likes
11

The mapathons mentioned above mapped buildings in Homa Bay, Kenya, with the exception of the mapathon in Brno that mapped in West Darfur, Sudan. MSF has been working in Kenya since 1987, providing medical services including treatment for HIV, Tuberculosis and non-communicable diseases. In Homa Bay, MSF is working on ensuring continuity of care for people with chronic diseases. MSF operates in Homa Bay’s state-run hospital where around 250 patients from all over the region are admitted every month. MSF works as well to enhance community initiatives by proposing differentiated care models based on a person-centered approach. This mapping campaign was originally created for the OSM Geo Week 2023, and the updated data/better base maps will serve the MSF team to improve the understanding of patient origin and facilitate and rationalize home visits to chronic disease patients.

5 Likes
12

I think that needs to be fixed before anyone does anything else. Currently the “Missing Maps” link is just to here, which seems to be just a promotional page about the organisation. There is nothing in there that says what each activity is for. That should be “job 1” for any organised editing activity.

8 Likes
13

No, Missing Maps’ failure to file the proper wiki paperwork is very much not the highest priority problem that needs to be fixed here. I would even argue that it’s off-topic, as filing said paperwork would not make the rate limiting disappear.

The goal of the rate limits, as I understand it, is to slow down bad-faith vandalism without noticeably impacting good-faith edits. The goal is not to prevent organized editing activities, regardless of some posters’ dislike for them or their real or perceived shortcomings in terms of documentation and quality.

If the limits prevent good-faith edits, and not just on rare occasions but regularly, I consider this evidence that these limits cannot stay as they are.

11 Likes
14

that depends on quality of this edits, good faith edits of questionable quality made at huge volume can be problematic

It was not a goal, but if this rate limits also impact organised editing activities that would benefit from a bit slower and more careful mapping then it is not a problem.

If rate limits impacts organised editing activities that produce high quality data and would not benefit from slower and more careful mapping then it is a problem.

Well, starting to follow OEG rules should be priority for Missing Maps, other involved people/organizations will have different priorities. And it would be a good idea to do it first before asking people to research situation and check what is going on. (and do it first before OEG activities)

9 Likes
15

I don’t think that anyone is questioning that these are good-faith edits (I’m certainly not). We’re just trying to understand how we can best distinguish between problematic edits (that the rate limiting is designed to prevent) and the sort of editing that any new user might make (in various circumstances).

As these are mappers introduced to OSM via organised editing sessions, I was just trying to understand what edits they were actually making before they hit problems (see also the github issue for more on this, including suggestions about how to change what is measured).

7 Likes
16

If the OSM server (possibly through an API) receives information about organized editing activities, including:

  • The BBOX (bounding box) area,
  • The duration of the campaign in hours,
  • The OpenStreetMap keys affected by the campaign,
  • The link to the campaign
  • The suggested hashtag
  • The requested ratelimit increase
  • The details of at least two organizers involved (user_id)

Then, I believe it might be possible to develop a smarter vandalism filter algorithm for the specified area. This could include a higher rate limit, and be active only within the specified BBOX and for the duration of the campaign, under the responsibility of the organizers.

8 Likes
17

ImreSamu has a good suggestion, which could also trigger better practices on term of paperwork.

Now, the rate limit introduced as a necessary reaction to recent vandalism was undoubtedly needed. However this can of worm is now open and unfortunately for volunteers time, this will need constant attention as this thread shows.
This came with responsabilities: some stuff was possible before, they are not anymore, and that’s not OK, wether we think such edits are okeyish or not. It would be bad if this sadly needed measure generates bad sentiment within the community, and care must be taken not to express one’s own feeling about Mapathons in general here.

9 Likes
18

Looking at this in more detail now that I’m at home, how can I identify the edits that were made as a result of this (or any other) editing session? I’ve got access to a changeset database and so can do:

changesets=> select distinct id,user_id,tags -> 'comment' from osm_changeset where created_at > '2023-11-16 00:00:00' and created_at < '2023-11-16 23:59:59' and tags -> 'comment' like '%something%';

or similarly tags -> 'someothertag' like '%something%'. The challenge is knowing what something or someothertag should be when searching.

19

** edit: @yvecai’s post has been edited so now it looks I am misquoting. Apologies for the confusion.

OpenStreetMap seems to be rather averse to policies. Even the Organized Editing Guidelines–often referred to as “rules”–specifically state that they are not:

[The OEG] are not a policy, but following them is the best way to make your organised edit successful and receive constructive community feedback.

I agree that actual policies would be better than the current situation.

2 Likes
20

Hey @SomeoneElse ,
I think using #msf would give you a good string to start with.

https://stats.now.ohsome.org/dashboard#hashtags=msf&start=2022-11-15T20:47:00.000Z&end=2022-11-17T20:47:00.000Z&interval=PT10M

1 Like