The new rate limiting prevents participants to Missing Maps mapathons from saving buildings

[SomeoneElse (Andy Townsend)]

I don’t think that anyone is questioning that these are good-faith edits (I’m certainly not). We’re just trying to understand how we can best distinguish between problematic edits (that the rate limiting is designed to prevent) and the sort of editing that any new user might make (in various circumstances).

As these are mappers introduced to OSM via organised editing sessions, I was just trying to understand what edits they were actually making before they hit problems (see also the github issue for more on this, including suggestions about how to change what is measured).

[ImreSamu]

@Tordanik

The goal of the rate limits, as I understand it, is to slow down bad-faith vandalism without noticeably impacting good-faith edits. The goal is not to prevent organized editing activities, regardless of some posters’ dislike for them or their real or perceived shortcomings in terms of documentation and quality.

If the OSM server (possibly through an API) receives information about organized editing activities, including:

• The BBOX (bounding box) area,
• The duration of the campaign in hours,
• The OpenStreetMap keys affected by the campaign,
• The link to the campaign
• The suggested hashtag
• The requested ratelimit increase
• The details of at least two organizers involved (user_id)

Then, I believe it might be possible to develop a smarter vandalism filter algorithm for the specified area. This could include a higher rate limit, and be active only within the specified BBOX and for the duration of the campaign, under the responsibility of the organizers.

[yvecai]

ImreSamu has a good suggestion, which could also trigger better practices on term of paperwork.

Now, the rate limit introduced as a necessary reaction to recent vandalism was undoubtedly needed. However this can of worm is now open and unfortunately for volunteers time, this will need constant attention as this thread shows.
This came with responsabilities: some stuff was possible before, they are not anymore, and that’s not OK, wether we think such edits are okeyish or not. It would be bad if this sadly needed measure generates bad sentiment within the community, and care must be taken not to express one’s own feeling about Mapathons in general here.

[SomeoneElse (Andy Townsend)]

@janabau

MSF Korea mapathon Nov 16 - several users

Looking at this in more detail now that I’m at home, how can I identify the edits that were made as a result of this (or any other) editing session? I’ve got access to a changeset database and so can do:

changesets=> select distinct id,user_id,tags -> 'comment' from osm_changeset where created_at > '2023-11-16 00:00:00' and created_at < '2023-11-16 23:59:59' and tags -> 'comment' like '%something%';

or similarly tags -> 'someothertag' like '%something%'. The challenge is knowing what something or someothertag should be when searching.

[ezekielf (Zeke Farwell)]

** edit: @yvecai’s post has been edited so now it looks I am misquoting. Apologies for the confusion.

@yvecai

A proper policy is needed

OpenStreetMap seems to be rather averse to policies. Even the Organized Editing Guidelines–often referred to as “rules”–specifically state that they are not:

[The OEG] are not a policy, but following them is the best way to make your organised edit successful and receive constructive community feedback.

I agree that actual policies would be better than the current situation.

[Hagellach37 (Benjamin Herfort)]

Hey @SomeoneElse ,
I think using #msf would give you a good string to start with.

https://stats.now.ohsome.org/dashboard#hashtags=msf&start=2022-11-15T20:47:00.000Z&end=2022-11-17T20:47:00.000Z&interval=PT10M

[SomeoneElse (Andy Townsend)]

Alas no, that doesn’t really pin it down:

changesets=> select count(*) from osm_changeset where created_at > '2023-11-16 00:00:00' and created_at < '2023-11-16 23:59:59' and tags -> 'comment' like '%msf%'; 
 count 
-------
  2915
(1 row)

[PierZen]

@janabau

Dear OSM Data quality working group and whom it may concern,
On behalf of Missing Maps (MM) and Médecins Sans Frontières, member organization I serve as a MM community engagement coordinator for, I would like to report that the recently imposed rating limit (OSM Community Forum thread How about limit new accounts? , Github issue 1 and 2 ) on new mappers has prevented the participants to several of our November mapathons from saving edits in good faith. We train our mappers, support them individually as needed and guide them to ensure as good data quality as possible throughout the mapathons, so we believe this anti-vandalism measure should not limit them from adding buildings in areas of communities vulnerable to crises.

@Hagellach37

Hey,
I think this is not an easy situation for OSM at the moment and preventing vandalism should be our main focus. My impression is that some of the comments here go to wide.

Whether or not you agree with way how Missing Maps and projects from other humanitarian organisations organize volunteers to map buildings and roads in OSM, I think that we all agree on the fact that at the moment we want to prevent vandalism.

@janabau

The mapathons mentioned above mapped buildings in Homa Bay, Kenya, with the exception of the mapathon in Brno that mapped in West Darfur, Sudan. MSF has been working in Kenya since 1987, providing medical services including treatment for HIV, Tuberculosis and non-communicable diseases. In Homa Bay, MSF is working on ensuring continuity of care for people with chronic diseases. MSF operates in Homa Bay’s state-run hospital where around 250 patients from all over the region are admitted every month. MSF works as well to enhance community initiatives by proposing differentiated care models based on a person-centered approach.

@Tordanik

The goal of the rate limits, as I understand it, is to slow down bad-faith vandalism without noticeably impacting good-faith edits. The goal is not to prevent organized editing activities, regardless of some posters’ dislike for them or their real or perceived shortcomings in terms of documentation and quality.

If the limits prevent good-faith edits, and not just on rare occasions but regularly, I consider this evidence that these limits cannot stay as they are.

I have long experience collaborating for humanitarian mapping and did coordinate the major OSM Disaster Response before the arrival of Missing Maps. I also wrote often about data quality problems related to Tasking Manager related coordinated mapping projects.

You all want to strictly discuss about rate limits. Looking at the history of HOT/Missing Maps projects mapping escalation via tasking manager and mapathons vs quality problems plus the dificulty for the community to have a good discussion with these humanitarian partners, I personally think that it is important to again discuss about better monitoring of such projects and the quality of edits even more with the arrival of AI related data imports.

For those not familiar with previous discussions, below are a few links :

• OSM-talk July 2018 - Why the HOT obsession with low quality buildings in Africa ?
• HOT Dec. 2018 0 The point on the OSM Response to the DR Congo Nord Kivu Ebola outbreak
• OpenDatalab 2018-09-10 - OSM Database Buildings Quality Analysis : The Tasking Manager
• OSM-talk 2018-10-01 -Tool update from HOT: MapCampaigner

Below is an Overpass query over Homa Bay where 1,319 buildings loaded, 1,286 with source=microsoft/BuildingFootprints.
Overpass Query- Buildings from nov.15 - Homa Bay Kenya

See for example the changeset 144106801 where 940 buildings are imported. Do you want to remove the rate limit to import more rapidly such type of data ? No reference in the Changeset about the imagery used to compare the import and fix quality problems, and no reference to the tasking manager or any coordinated project.

Changeset: 144106801 | OpenStreetMap

<changeset id="144106801" created_at="2023-11-16T18:05:34Z" closed_at="2023-11-16T18:05:40Z" open="false" user="obraynt456" uid="12114869" min_lat="-0.5306056" min_lon="34.4503570" max_lat="-0.5230729" max_lon="34.4618476" comments_count="0" changes_count="5114">
<tag k="comment" v="buildings in homa bay town"/>
<tag k="created_by" v="JOSM/1.5 (18822 en)"/>
<tag k="mapwithai" v="939"/>
<tag k="mapwithai:options" v="version=816;maxadd=999;url_ids=facebook_mapwithai"/>
</changeset>

Comparing the building data added to OSM with Esri or Bing, makes me think that no effort was made to correct / align buildings with standared imageries. If we speedup such AI detected buildings imports, we will surely not assure quality.

It is not enough for our humanitarian partners to say that these projects are important for their project and often admit that they worry more about the number of buildings in an area then about mapping quality.

[Tordanik (Tobias Knerr)]

@PierZen

You all want to strictly discuss about rate limits.

Or at least a quick solution to the immediate problem. That might involve tweaking the limits, but it might also involve, say, marking the accounts of mapathon participants as exempt from the limit in an unbureaucratic fashion. (I think we already have such a solution for import accounts.)

I’m fully in favor of better organized editing practices as well, but that should be a separate conversation. Our humanitarian partners deserve a timely solution to their pressing issue as a first step.

[fititnt (Rocha)]

I just looked at the changeset of one of the mentioned maphatons. And sorting by changeset with most deletions, this is the top one.

Note that some objects are deleted then recreated, instead of just reuse (which would allow stay easier under the limits)

[Jorieke_V (Jorieke Vyncke)]

Hi, we have also an osm wiki page here: Médecins Sans Frontières - OpenStreetMap Wiki
When back from SotM Africa I will see how to improve this.

[Wynndale]

Are these mapathons meeting in person or online? If in person itʼd be a good idea to have an experienced OSM contributor looking over shoulders and give them the authority to ease limits if necessary.

[PierZen]

@Tordanik

Or at least a quick solution to the immediate problem. That might involve tweaking the limits, but it might also involve, say, marking the accounts of mapathon participants as exempt from the limit in an unbureaucratic fashion. (I think we already have such a solution for import accounts.)

I’m fully in favor of better organized editing practices as well, but that should be a separate conversation. Our humanitarian partners deserve a timely solution to their pressing issue as a first step.

We have started these discussions from the start of the Missing Maps project back in 2014-2015. We accomplished a lot for the Nepal 2015 Earthquake response but a lot of frustration was expressed by experience mappers about mapping quality and validation.

If you look at the link above for the Nord Kivu Ebola outbreak in 2018, mapathons organized by partners impacted with bad quality of tracing. In the context of a major emergency, we did have to erase all buildings for Butembo and restart the mapping

Looking more closely at Changeset 144106801, it seems that this was a contribution from a local contributor using AI generated data .

Missing Maps TM-Project-15737 from Nov.14 was in the eastern part of Homa bay and this Overpass Query covers the area from Nov.14. (22,772 buildings from nov.14).

I have to say that Data quality has improved from a few years ago. But still some mappers that trace non-orthogonal buildings or misalligned them comparatively to base imagery. Validation function from JOSM returns for these buildings :

• section of a way repeated twice - 5 buildings
• area not closed - 5 ways
• duplicate - 1 way
• similar name - 40 (name=edificio)
• no zone style for a multipolygon (1 building traced with 2 ways grouped in a relation)

Part of this extraction, there are 66 buildings that overlap. There are also many non-orthogonal buildings. But note that using the JOSM validation function, it did not report these non-orthogonal buildings The Changeset session144564707mapped three days ago shows such quality mapping problems. Have the JOSM rules been modified and not catching overlaps and non-orthogonal buildings anymore ? Or do I need to specific parameters in JOSM ? JOSM Preferences - Data validator panel - all options are already checked.

OSMCha for Changeset 144564707

[pnorman (Paul Norman)]

@Jorieke_V

Hi, we have also an osm wiki page here: Médecins Sans Frontières - OpenStreetMap Wiki

This doesn’t describe this specific organised edit. The OEG page needed would contain a bunch of information needed to investigate what was being done and if tweaks to the limits are necessary.

[SColchester (Sam Colchester)]

Tested by creating a new OSM account and mapping buildings in Somalia until I hit the limit. I mapped 52 buildings in iD before the limit was hit. This is one of the changesets that went through before the block came in. You can see here all of the buildings that comprise this limit exceeding activity (in iD against black for clarity):

The same in JOSM against Bing imagery (buildings in teal):

And the block:

I chose an area with circular buildings to demonstrate how the limit can be reached quickly when features require more nodes. But please note that this is not exceptional in humanitarian/development, lots of areas that MSF focuses on include plenty of circular buildings.

[Jorieke_V (Jorieke Vyncke)]

Hey, I have added the info about this mapping project to the page. I hope this is sufficient? As said above, I will work to improve our documentation in general the upcoming days.

[Wynndale]

It occurs to me that circular buildings wouldn’t be the problem they are if we were using Jochen Topf’s API redesign. Is there a way we could simulate some of that in the limits?

[SK53]

Note that there is an open issue for iD about reducing the number of nodes used for small circles. Currently these huts which are about 9 square metres get 19 nodes in iD and 8 in JOSM.

Would implementing something closer to the JOSM algorithm raise the point where rate limiting comes in for this kind of edit?

[SomeoneElse (Andy Townsend)]

@SK53

Would implementing something closer to the JOSM algorithm raise the point where rate limiting comes in for this kind of edit?

The test changeset above has 44 ways in it and 836 nodes, so yes (the linked example way is circular and has 20 nodes).

[Tordanik (Tobias Knerr)]

@SK53

Note that there is an open issue for iD about reducing the number of nodes used for small circles. Currently these huts which are about 9 square metres get 19 nodes in iD and 8 in JOSM.

8 nodes are too coarse an approximation for a circle, as 45° angles are not small enough for a heuristic to safely assume that they are supposed to represent a curve.

I don’t think the response to overly strict API limits should be to decrease mapping quality. In any case, it would only allow about twice as many buildings to be mapped before you’d run into the limit anyway.