Proposal: Add reactions/votes to changesets

General talk
, ,
1

Background

While the discussion on rate limits continues, a post by @kaartjesman got my attention (emphasis mine):

I’ve actually been thinking about ways to improve the OSM API for quite some time now, and though my thoughts hadn’t been able to describe it as clearly as the above quote, that has been one aspect of it.

What?

So, therefore, I propose that we* add support for a rather simple form of one of the most common aspects of social platforms: Likes (and dislikes)

In summary, any user would be able to add a “reaction” (not happy with that term, does anybody have any better suggestions? “Votes” would be an alternative, but I’m not really happy about that either) to a changeset. Initially, the available reactions would be just “good” and “bad”, though they could in the future be extended such as with a larger spectrum (“awesome”, “awful”) and “neutral” (for “I’ve reviewed this but don’t have anything to say”), though I think unless someone really feels like they should be included from the start it’s better to focus on just “good” and “bad” for now.

Apart from wanting them to be usable semantically (for rate limiting etc., see below), I’m also not too huge a fan of “emojis everywhere!!! :smiley: :stuck_out_tongue: :wink: :heart:”, so I’m explicitly not proposing reactions to be arbitrary emojis (like some platforms allow) or even a subset of emojis (such as here on the forum).

EDIT: Instead of “reaction” I’m now considering “review”. Other terms that have been mentioned are “rating”, “vote” and “thank”
EDIT: Based on comments I’d want to start out with three possible reactions: “good”, “can improve” or “bad”. We need to (at least loosely) define the meaning of these, for example “good” could be anything that overall improves OSM while following established best practice.
EDIT: Based on comments, a review other than “good” should also require a discussion comment
EDIT: Casting a reaction/review should not be possible for new accounts (to prevent sock-puppeting), some requirement (maybe similar to rate limits) should be present. Negative reactions (“bad”) might have stricter requirements that positive reactions, similar to how it is on StackExchange.

* to be clear I don’t intend this as a “we as in someone in our community that isn’t me” but rather the opposite, if the community likes this idea (or some variant thereof) I fully intend to do the development work for it (just have to dust off my Rails knowledge…)

Why?

This would bring several benefits:

  • It can become an input to the calculation of rate limits
    • For example, changesets with a majority of “good” reactions from users who in total have at least N “good” reactions (EDIT: replaced by not allowing reactions for new accounts) would not count against the limit, while a majority of “bad” reactions would count twice against the limit
    • This could be a solution to “the mapathon problem”, as then the experienced mappers who handle the mapathon can, through a review on the spot, effectively increase the rate limit for new mappers
    • This could also be coupled too other “privileges”, such as preventing changes to high-level administrative boundaries unless the user has some number of “good” changes, thus preventing accidental edits by inexperienced users (as well as vandals) while retaining the spirit of the “no one has more privileges than anyone else” idea that OSM is built on (just adding a “…once they have made the same number of well-received edits”)
    • Note: Unless to the extent necessary to discuss this proposal try to refrain from discussing the rate limits in this thread, these points here are just examples
  • Bragging rights and can serve as input to sites calculating badges (can’t find one that’s working right now though I’m pretty sure I’ve seen one at some point), possibly even here on the forum
  • It provides a way to mark a changeset as “reviewed” if it is tagged review_requested=yes
  • It brings the “verify” state of OSMCha into the central database so that other sites can show it as well
    • I’d also like to collaborate with the creator of OSMCha to get their existing data on this into the OSM database if at all possible

How?

  • A new table in the database (changeset_id, user_id (of the user reacting), reaction, timestamp)
  • A new endpoint POST /api/0.6/changeset/#id/react, taking just the reaction (or null to remove a previous reaction by the user)
  • GET /api/0.6/changeset/#id?include_reactions=summary returns a summary of reactions ({ "good": 5, "bad": 2 })
  • GET /api/0.6/changeset/#id?include_reactions=yes returns a full listing of reactions ([{ "date": "...", "uid": "...", "user": "...", "reaction": "good" }, ...])
  • The changeset view on osm.org shows a summary (some icon and a number for each type of reaction) at the top, hovering over an icon shows the list of users that have reacted, clicking an icon reacts (similar to how it works here on the forum):
    image
  • For now I don’t propose adding this information to planet/changeset dumps

Importantly, all of these changes can be made in the current API since they are purely additions in functionality, so they don’t have to wait for some hypothetical future API rework.

By providing an API it becomes possible to integrate this functionality in tools such as OSMCha. It would also be possible to let the existing QA tools react with “bad” when they find some grave issue, though that would need to be separately discussed since that could likely introduce another bag of problems if not done right.

References

Some prior art (sorry if I’ve missed something!):

And now…

What do you think? Is this overall a good idea? Are you onboard but would like to change some details? Can you think of a better term than “reaction” or “vote”, or do you like one of them?

8 Likes
2

A crude functionality does more harm than good, based on experience and usage of OSMCha:

  1. There is no ability to mark it as mixed. In effect, OSMCha’s function to mark individual objects bad is not well-used. It needs to be scalable to accurately label as partly good or bad on multiple objects, including differentiation of tags vs geometry changes.
  2. Can’t separate intention from result. “Bad” can be discouraging to genuine effort and well-intention attempts. “Good” can be encouraging some subpar practices, and there are different levels of controversies in different issues.
  3. Can’t flag suspicious ones that requires more investigation or discussion.
  4. Potential for abuse by disgruntled users in disputes (only a single user is able to review; persists if the 1st user leave a negative vote, cf voting in Reddit etc)
  5. Absence of requirement for a review comment
6 Likes
3

Maybe we could add the option to flag the flags?

4

Good points that need discussion.

  1. If I understand you correctly, there should either be a way to also “react” to individual changes? Question then would be to what level; just individual objects, or even individual tags? This would probably be hard to get right UX wise as well though. Could it be enough to have changeset-level reactions, but apart from “good” and “bad” also having a “mixed”? Alternatively, we could say that the reaction should capture the “worst” part; so if one part of the changeset is bad then the reaction should be “bad”, or the overall aspect (adding a lot of new houses with good quality but removing and recreating a few existing ones would overall still be “good”, since it taken together improves OSM)
  2. I guess unless we want to allow reactions on different aspects (like in some online shops, see screenshot), which I think would make it to complicated, we would have to agree on the meanings of the reactions, as well as somehow make that clear in the UI. Maybe this could be helped by introducing other reactions, for example instead having “good” (good intentions, follows standards and best practices), “needs work” (good intentions but badly executed), “bad” (damaging to the data or the community).
    image
    image1053×221 2.86 KB
  3. Maybe another state could be “flag” (“…to the DWG”), which could automatically also report the user? (I actually had something similar when I first wrote the proposal, but removed it before posting)
  4. I think this one is hard to avoid (flagging/reporting reactions as suggested by @Peter_Elderson is of course possible technically, but would add extra work to the DWG or whoever would handle that), though I’m sceptical if we’d have the same patterns as e.g. Reddit. Sure, when already having a bad review (maybe that’s a better term?) the next reviewer would likely look a bit closer but hopefully make up their own mind. And in the worst case scenario, if a group of disgruntled users pile onto some other user then the mapper in question could post their changeset here (or in some other community channel) and request more reviews, which should hopefully balance out the results. A single changeset having many conflicting reactions could also be a useful data point, as it indicates a controversial topic that might need further discussion and clarification.
  5. Could add a requirement that the reacting user has left a discussion comment before reacting negatively (risks skewing the result, but requiring comments for all reactions would likely just introduce a lot of useless comments such as “looks good”, “ok”, etc.)
1 Like
5

Is “good” intended to show appreciation for a changeset that is “better than average” in some way? Or does it simply mean that it doesn’t appear to have any technical problems? If it is the latter maybe “OK” or “acceptable” would be a better word than “good”.

It seems that mappers in regions with a lot of active mappers are likely to get a lot more reactions than contributors in undermapped regions, where there may be nobody systematically reviewing changesets. That could be discouraging for new mappers in the very places where we most need them. Especially if, as suggested, this is used to determine some kind of status (rights to edit certain objects, or “bragging rights”). I’m not sure if this really fits with the idea of “no one has more privileges than anyone else”, as it depends not only on making good edits, but also on attracting attention to those edits.

4 Likes
6

First, Jan, thank you for kickstarting this!!

This question indeed goes to the core of the matter. I guess nobody wants this to be a twitter/facebook kind of like.

Instead the intention is to record some sort of approval from the community that this mapper is doing work that is appreciated (or not).
With the underlying intention that a new mapper is supposed to work with the community in order to keep the tagging database from becoming chaos. (or just stick to simple changes like streetcomplete stuff or other things that are safe, that’s obviously Ok)

Examples:

  • a maps me user adds tags on POIs that are clearly not intented for a globally shared map. It would be nice to point out to them that their work is not a private tag for their eyes only.
  • Someone started using Id to edit bus-lines, messing up the groupings.
  • A (new) user adds a bunch of new houses as buildings formerly unmapped in their neighbourhood.
  • A user creates 20 changesets in a day, clearly using something like streetComplete

I think a good outcome is that new users have a limited number of changes that they are allowed to make every day before they are required to ask for approval, or be stuck at the level of creation they started with.
The first two bullet-point examples would then be caught early on and we can correct the behavior of the user before a lot more mess is created in the DB.
The last 2 examples are easy approvals, “carry on” style! And ideally that means these users will continue to map, but it also makes approaching community members easier in the future as they have shown up and stated they like the work.

Voting is a priviledge

The one change I’d make to the proposal is that voting is a priviledge. Meaning that you need to already have received enough upvotes in order to be able to upvote others. This avoids the sock-puppet upvoting problem that would undermine the actual useful signalling.
Downvoting likely is best to be unlocked even later.

I don’t know the best way to do a “downvote”. Although I prefer the term you used later “changes required”.
Maybe UX there is not the most important part at this point, and I’m sure others can chime in with suggestions.

Again, thanks Jan for pushing this one!

1 Like
7

I’m thinking of it that “good” means “improves OSM” (which theoretically could also include a changeset that doesn’t fully follow best practices if it overall is an improvement to was before, for example adding a lot of buildings that weren’t mapped before but also removing and re-creating some existing buildings (not keeping the history)). But this is definitely something I think we should discuss, as it’s definitely a definition that many people will have different opinions on.

This is a good point, especially when starting a discussion on concrete benefits (or punishments) of getting reactions. Though I think this should also be balanced against the improvements a system such as this would bring (for example, if the benefits are either purely cosmetic (such as badges) or achievable with a few good changes then the effect for mappers in underrepresented regions could be negligible compared to upsides such as better rate limits).

8

Indeed, I think we should try to have reactions be based on the best practices etc. rather than personal opinions (another reason why I don’t like the term “reaction”, I think I’ll start saying “review” instead). Requiring a comment for negative review as mentioned previously could help with this, as it could reduce the number of knee-jerk I-oppose-everything reviews.

I guess this could work, my initial thought was to take this into account when calculating things such as rate limits (only counting reviews/votes from people who have enough reviews/votes themselves), but having the functionality not available at all before reaching a certain level would both simplify calculations and make them the same for all uses of the reviews.

Limiting downvotes could also work, and would be in line with how it works on many other platforms (StackExchange for example).

9

I think in the long run we want to have something like OSMCha oder Avchavi integerated into OSM.

Reviewing changesets without these tools is merely impossible.

OSMCha has recording of “thumbs-up” on a changeset, it only records one/first voter.

Flo

10

One of the things that OSMCha does that I’m not a fan of is recording “BAD” in capital letters in comments.

I’ve dealt with a number of people who’ve been upset by that, and haven’t engaged with the actual comment as a result.

4 Likes
11

Yeah, that definitely sounds like something to avoid (and also an indication that it would be good to get this functionality into OSM proper, so that tools like OSMCha don’t have to resort to such workarounds).

12

On the OSM Wiki, you can “thank” someone for a change. It spreads a little more encouragement around OSM, and I’d love to see that for OSM editing changes. We could add a few other types of reactions to send other messages/

I don’t think this will properly address “the mapathon issue”. We don’t want mapathon instructions to include “Now everyone stop editing, and click like on everyone else’s changesets”.

8 Likes
13

No, that would of course not solve anything. But as I’ve understood it (hopefully correctly!) the more experienced mapper(s) who also serves as the instructor at a mapathon also check the edits done during it, as part of which they could also review/mark a changeset as good or bad. As has been noted earlier in the thread, new mappers should not be able to review a changeset, so the scenario you describe should not be possible.

Some safeguards against the one validating doing a proper job might be needed, for example, their ability to review changesets could be revoked by the DWG if it turns out that a lot of changesets they have reviewed as “good” weren’t in fact good. This aspect isn’t currently part of my concrete proposal, as it is initially intended to just lay the groundwork to build upon later, so such safeguards would then need to be discussed next.

14

I do not have a problem with good or thanks (or needs further review) but do not welcome bad or thumbs-down for a changeset. If something is not good I am more than happy to know but I want to know what is it that is not good.

6 Likes
15

What makes MediaWiki’s “Thanks” feature special is that it’s discreet. Thanks are logged as a precaution against harassment, but otherwise, unlike emoji reactions in many chat and forum platforms, it’s only easily accessible to the user being thanked. It doesn’t serve as a rating or a gauge of public opinion, but it does help to make the editing experience more humane for ordinary people.

Thanks and its predecessor, WikiLove, were inspired by wiki editors’ longstanding practice of awarding barnstars to each other on user talk pages. Barnstars are more public but still a form of one-to-one communication: you give someone a barnstar, and they choose whether to display it publicly. Barnstars exist in real life too; I’ve come across quite a few but apparently never thought to map them.

I think a shortcut for thanking a mapper is a fine idea, but we should probably consider it separately from reactions or scoring, since the social dynamics are so different. The last time I proposed this idea, it quickly got derailed by ideas about micropayments, but hopefully we can muster enough enthusiasm for plain old-fashioned kindness to get traction on either public or semi-private expressions of appreciation.

5 Likes
16

I have been doing that basically. Only used “good” for thanking, and “bad” for solely bads or vandalism, not using the thumb for mixed good and bad changes. Considering the technical limitation now, as only one user can rate each changeset, I don’t want to give a “good” in case I missed any mistakes when I glance through them.
Fundamentally, I find it unhelpful to rate a series of related changesets “good” or “bad”. Changesets are broken up for upload for different reasons, and doesn’t reflect the number of changes for the actual scale of contribution or issues. Having smaller changesets inflates the number of “good” or “bad”, and forms an inherent flaw to allow gaming the system.
It’s tiring to rate every affected changeset in a batch now. Still not reflective of the result if selecting and multiple changesets is possible. (While the reviewing of each changeset independently needs to be ensured to avoid judging without regards, it’s often clear which changesets are offending)

17

Would this be solved by requiring a “bad” review to be accompanied by a discussion comment (so that you get to know why someone thought it was bad)?

I guess one step further could be to allow a review on a later changeset that fixes the issues to be marked as “solves the issues from changeset XYZ”, but that would complicate things significantly., both technically and usability-wise.

I understand your reasoning, but I’m still a bit skeptical to having separate features for this. One would have to be really careful UI wise to make it clear what each means (one is private/doesn’t affect anything, the other public/can affect the mappers ability to map). Additionally, even though we might say initially that “thanking” isn’t considered for rate limits etc., sooner or later someone will bring that up for reconsideration (especially if the review/reaction feature gets less use).

Any implementation using the reviews/reactions would have to take this into account, and the exact details could vary. As an example, the current rate limit code looks at the number of changes (added/updated/removed) elements in the users changesets, rather than just the number of changesets.

Finding this balance will be somewhat tricky (just as the implementation of rate limits so far have been tricky) and require some thought/discussion, but I think it’s best if we leave that discussion for the next step after having a system for reviews/reactions in place.

18

I am afraid not, if somebody only has the “energy” to press a dislike button, there will be often no “energy” to write a meaningful reply on what is wrong.

The current practice is that commented changeset can be counted, that is not prefect but in my view adding a bad or thumbs-down button only makes does not make things better.

What would be good to add is some way to easily see if on a changeset that has been commented, the original author did reply. This could also be used for “scoring” purposes.

19

That’s already available from Pascal Neis? Asking to add these tools, or at least links in the official website will still take some discussion. https://resultmaps.neis-one.org/osm-discussions
Attaching hashtags or labels to changeset replies won’t be bad, if not as “BAD” as OSMCha after rating. It would be helpful to quickly tell whether they are requesting information, starting a discussion, doubting, or outright concluding it’s a negative.

20

Please, hashtags in changeset comments are bad enough, let’s not encourage hashtags in changeset discussions.

1 Like