Changing RFC time for proposals including deprecation

lonvia · 2022-11-23T19:28:10.116Z

Minh_Nguyen:

This project wasn’t starting from scratch. There was never a serious risk of rivers vanishing from renderers after retagging, because natural=water enjoys better support (this being an argument in favor of the retagging). The retagging didn’t affect other use cases such as routing or geocoding. The analysis use case was also largely unaffected, as global queries had had to account for both tags for some time.

Sorry to disappoint but this change did affect geocoding. It is a stellar example of something that might have looked like a bit of a simple cleanup but in reality was a structural change to the tagging that can suddenly pose quite a huge problem on the software side.

Here is the problem: we’ve had this double tagging of rivers with waterway=river/stream + waterway=riverbank. Often both were tagged with a name. This needs some deduplication for geocoding purposes. Nominatim did this by simply ignoring waterway=riverbank. After all, the river line is the more interesting result and ignoring a tag is easy enough. Then @ZeLonewolf came along and changed the tagging from a simple waterway=riverbank to natural=water+water=river. What was previously a primary feature now became an attribute to another tag. The way the tag processing works in Nominatim there is no way to filter by such an attribute, so deduplication of rivers has been broken since this particular mass edit because fixing it requires a change in the design of the software.

Please don’t discuss now the peculiars of how to implement secondary tag filtering. It’s not really the point. The point is that any changes to existing tagging may have an affect on data consumers that you don’t have foreseen. So if you want to have deprecation and ‘tagging cleanup’ in a major way, you have to devise a plan that gets feedback from data consumers already during the RFC phase. I consider any proposal doomed that cannot show that considerable thought has gone into the secondary effects that the change might have.

Minh_Nguyen · 2022-11-23T19:30:30.020Z

lonvia:

Here is the problem: we’ve had this double tagging of rivers with waterway=river/stream + waterway=riverbank. Often both were tagged with a name. This needs some deduplication for geocoding purposes. Nominatim did this by simply ignoring waterway=riverbank. After all, the river line is the more interesting result and ignoring a tag is easy enough. Then @ZeLonewolf came along and changed the tagging from a simple waterway=riverbank to natural=water+water=river. What was previously a primary feature now became an attribute to another tag. The way the tag processing works in Nominatim there is no way to filter by such an attribute, so deduplication of rivers has been broken since this particular mass edit because fixing it requires a change in the design of the software.

I stand corrected on this point, and am glad that we agree that deprecation is not normally as simple as it seems on the surface.

ZeLonewolf · 2022-11-23T19:45:45.195Z

lonvia:

Here is the problem: we’ve had this double tagging of rivers with waterway=river/stream + waterway=riverbank.

It is not double-tagging, they are entirely separate features. One is the path of the river, and the other is the water-covered area of the river.

lonvia:

Often both were tagged with a name. This needs some deduplication for geocoding purposes. Nominatim did this by simply ignoring waterway=riverbank.

Tagging a name on a river area is always incorrect tagging. If Nominatim is indexing names on waterway=riverbank or natural=water + water=river, then that behavior is simply incorrect, and has been incorrect for over a decade.

lonvia:

Then @ZeLonewolf came along and changed the tagging from a simple waterway=riverbank to natural=water+water=river.

False, both tagging systems were widely in use at the time I became involved, sometimes just one, sometimes just the other, and sometimes both. If Nominatim was failing to correctly handle the former but not the latter style tagging, then it was already failing to properly handle a significant percentage of the river area objects tagged worldwide when I became involved.

If this effort further exposed an issue with Nominatim that was already widely present, I would consider that a positive outcome.

ezekielf · 2022-11-23T20:14:10.165Z

We would do well to establish a clear and predictable process for changing a tagging schema, deprecation, and finally full replacement. Even the smallest change is going to annoy someone.

xkcd.com

Workflow

There are probably children out there holding down spacebar to stay warm in the winter! YOUR UPDATE MURDERS CHILDREN.

Matija_Nalis · 2022-11-23T21:53:44.365Z

Zeke Farwell:

We would do well to establish a clear and predictable process for changing a tagging schema, deprecation, and finally full replacement. Even the smallest change is going to annoy someone.

I do agree, but who is going to do the work?
Using passive voice (“we would do well to xxx”) is almost guaranteeing nothing will happen.

How about you actually suggest a rough outline draft of the text/chapter how you think it should go (click on that arrow and “Reply as linked topic” so others know to follow), and I promise I’ll jump right in and offer changes / more details. In few iterations by parties interested, we might actually have something tangible which can be offered the rest of community for comments. Deal?

harahu · 2022-11-25T21:48:38.376Z

Zeke Farwell:

Not really. The key conflict is a non-issue since the intermittant water area and the sand or rock can be mapped as two separate objects. For example: river area , rock area . Eventually the remaining waterway=riverbank areas in Norway will get changed to this style.

I hear you say that this is a non-issue, but is it really? I’m a Norwegian mapper, and I’ve used the style you suggest for my own projects where I try to achieve this effect, but it comes with a bitter aftertaste. The technique of mapping with superimposed areas with tag natural=* is not well documented, and does not seem universally accepted. Whether data consumers treat it correctly seems quite hit and miss. I would not translate these cases to the style you suggest before said style:

Is well documented and standardised. For instance, does one use layer=-1 on the area signifying the river/lakebed?
Has gone through a proposal process.

SomeoneElse · 2022-11-26T11:52:21.404Z

A couple of points:

Projects can say what tags they consume by adding an entry at https://taginfo.openstreetmap.org/projects
Of all of the proposed tagging changes over the years, including “deprecations”, as a data consumer I’ve only ever been contacted once (by the parcel locker people) to say that a proposal might affect the data that I’m consuming. Thanks to them for doing that.

ezekielf · 2022-11-27T05:19:16.733Z

For what it’s worth, I don’t think the technique of tagging waterway=riverbank and natural=sand (or bare_rock, shingle, etc) on the same object is well documented or universally accepted either. I wouldn’t be surprised if some data consumers did something odd in both cases. I don’t see a problem with the data modeling though. One feature is an intermittent area of water, another feature is the intermittently exposed sand, rock, or shingle. The areas overlap in the real world, so it makes sense to model them as two overlapping areas in OSM as well. Improving documentation to mention this technique is a good idea though.

ezekielf · 2022-11-27T05:27:59.364Z

You’re certainly correct. Making a statement like this is easy, but doing the real work to make change is difficult. I probably do not have the available time and energy to get started on this in the near future, but I’d be supportive of such an effort.

SomeoneElse · 2022-11-27T11:03:56.961Z

For completeness - here’s an example: https://www.openstreetmap.org/relation/3995755.

From the imagery, it looks like an area of rock that forms a well-defined river channel. It’s probably usually mostly dry with a small stream making its way through (see the Bing imagery) but occasionally a raging torrent.

The question, I guess is whether should be mapped one object or two. Is it only one (something that is sometimes wet and sometimes not), or two (the rock, and the water that’s there part of the year)?

However it gets mapped, there are ways for data consumers to decide what they want to show here. It’s much easier (using pretty much any rendering mechanism) if both sets of tags are on one object, but also possible if they’re not. Personally I don’t think “how easy it should be to render” should be the main factor here - it comes down to “do you think there is one object here or two”.

Adamant1 · 2022-11-27T12:15:35.269Z

Diacritic:

I would like to float the idea of extending the minimum RfC period for proposals

I’m not really sure if the RfC period should be extended, since I assume with most proposals the details have already been discussed on the mailing list and other places beforehand, but I’ve always felt that the two vote period is way to short and should really be extended by at least a week, if not two. It can take almost a week just for everyone to be notified. Let alone can most people read through the proposal, relevant mailing-list discussions, Etc. Etc. and made an informed decision in that short of a time.

(I’m aware that the general rule is “at least” 2 weeks and that it can be extended, but it’s clearly the default and people don’t usually extend it. One reason being that it can look bad on the proposer if they extend the vote period for a proposal that is likely going to be rejected after the 2 weeks ends.)

IanH · 2022-11-27T17:50:21.433Z

SomeoneElse:

Of all of the proposed tagging changes over the years, including “deprecations”, as a data consumer I’ve only ever been contacted once (by the parcel locker people) to say that a proposal might affect the data that I’m consuming. Thanks to them for doing that.

Proposals should include a machine readable description of each set of tag changes. This requires that the all the concepts described in human language are available in a form something that bots and renders can immediately process. This forces the proposal writers to create data object or modifier for each concept being covered in order to be considered complete. The resulting data objects can then be analyzed and used by companion software such as editors and routers.

Minh_Nguyen · 2022-11-27T17:58:23.464Z

The wiki isn’t well suited for machine-readable schemas, unless you’re suggesting data items about proposals, but that could be a challenge to model well. Maybe there could be a voluntary step of creating a pull request against the id-tagging-schema repository, showing how its list of deprecations and suggested upgrades would change. But I’m unsure about making that a required step, since deprecations aren’t always as straightforward as replacing one set of tags with another.

ZeLonewolf · 2022-11-29T05:06:04.707Z

Matija_Nalis:

Zeke Farwell:

We would do well to establish a clear and predictable process for changing a tagging schema, deprecation, and finally full replacement. Even the smallest change is going to annoy someone.

I do agree, but who is going to do the work?
Using passive voice (“we would do well to xxx”) is almost guaranteeing nothing will happen.

Process:

Proposer writes proposal, discusses it widely, and demonstrates consensus with a >75% approval during a 2+ week vote.
Proposer writes up technical procedure for making the change in the map.
Proposer discusses proposed change with the communities involved to ensure there are no objections to making the edits
Proposer makes the edits

There is your process, with no use of the passive voice. It’s also the process that exists today and anyone can use!

In practice, the change from an older tagging scheme to a newer one is rarely a 1:1 substitution and a bunch of work is involved to deal with edge cases, outliers, or cases where the old tag is replaced by more detailed tagging that requires the mapper to make a decision.

The problem isn’t that we don’t have a “process”. The problem (if you can even call it a problem) is that the community rarely agrees on what to do, and most people aren’t willing to put in the effort of listening, technical analysis, persuasive writing and diplomacy needed to do it on a global scale.

This is the reality of an anarchic, community-driven process. Some have suggested that the community process is no longer effective and that it would be better for the OSMF to introduce a formal, bureaucratic process for handling tagging changes. That would certainly “solve” the problem of tagging changes being too hard by appointing an arbiter to decide on such things. However, I suspect that the voting membership of the OSMF would oppose board candidates that would advocate for foundation involvement in tagging issues.

Adamant1 · 2022-11-29T06:12:05.124Z

ZeLonewolf:

That would certainly “solve” the problem of tagging changes being too hard by appointing an arbiter to decide on such things.

Wouldn’t it be more like they are just implementing the process you outlined that has already been decided on by the community? Otherwise, what exactly would they be deciding on that would be out of scope of the already exiting way of doing things?

However, I suspect that the voting membership of the OSMF would oppose board candidates that would advocate for foundation involvement in tagging issues.

I don’t see what the “issue” would be if they are just implementing a tag that the community has already voted on and approved. In a way bureaucrats/arbiters are already involved in tagging issues when for example DWG members revert edits done by people who are using unpopular or unapproved tags. At least in the case of deprecation it would have an actual, wider benefit to the community outside of just tag fiddling to indulge the preferences of a small minority of confrontational users or whatever.

dieterdreist · 2022-11-29T12:30:14.000Z

The problem isn’t that we don’t have a “process”.

the procedure for tag deprecation through the tag proposal process so far didn’t have automatic retagging included, it leads to the tag being labeled “deprecated” in the wiki. Judging from taghistory, there are sometimes also some larger retagging efforts after this labeling, but they are not part of the tag proposal process (often they are not discussed or announced and every case would have to be individually assessed).

dieterdreist · 2022-11-29T12:35:01.000Z

I don’t see what the “issue” would be if they are just implementing a tag that the community has already voted on and approved.

the whole proposal process is about finding and agreeing on tagging definitions, it is not dictating how people have to map, it offers options.

Adamant1 · 2022-11-29T13:04:29.253Z

dieterdreist:

the whole proposal process is about finding and agreeing on tagging definitions, it is not dictating how people have to map, it offers options.

Sure, that’s technically true. But there’s generally an assumption with most people in the community that if a tag is voted on and approved that it depreciates whatever non-approved options exist at the time. Otherwise, it really shouldn’t be called “approval” and there’s no point in talking about depreciation in the first place The fact is that whatever the “technical”, semantics are that “approval makes unapproved tags go down” and there should be a more streamlined, concise process for doing that then currently exists.

The constant meta discussions about the “technicalities” in discussions like this really just holds up the project. It serves no one if it takes 15 years to replace a few instances of a tag to something that is “approved” just because approval isn’t about dictating how people have to map or whatever. Personally, I want to find the chicken shack I’m looking for when I search for it, not have to have find another restaurant because some random user 15 steps down (or up?) the ladder thought it was better to “offer options” instead of going with a tag that everyone else agrees on. I’m sure I’m not the only who feels that way.

ZeLonewolf · 2022-11-29T14:30:17.016Z

Adamant1:

[…] there should be a more streamlined, concise process for doing that then currently exists.

The constant meta discussions about the “technicalities” in discussions like this really just holds up the project. It serves no one if it takes 15 years to replace a few instances of a tag to something that is “approved” just because approval isn’t about dictating how people have to map or whatever. Personally, I want to find the chicken shack I’m looking for when I search for it, not have to have find another restaurant because some random user 15 steps down (or up?) the ladder thought it was better to “offer options” instead of going with a tag that everyone else agrees on. I’m sure I’m not the only who feels that way.

You’re definitely not the only contributor that feels that way. Recent proponents for tagging standards include an OSMF board member and its former chairman. Also, a question on tagging standards was posed to the 11 candidates to the OSMF board, and you can find their answers here.

Unfortunately, anarchy on the project is a fact of life, and a small vocal minority, or even a single vocal contributor, can have enormous impact on what is tagged in the map. For example, the fact that we cannot get rid of service=driveway2 is a testament to the persistence of a single contributor.

Minh_Nguyen · 2022-11-29T18:28:41.325Z

ZeLonewolf:

Process:

Proposer writes proposal, discusses it widely, and demonstrates consensus with a >75% approval during a 2+ week vote.

Proposer writes up technical procedure for making the change in the map.

Proposer discusses proposed change with the communities involved to ensure there are no objections to making the edits

Proposer makes the edits

There is your process, with no use of the passive voice. It’s also the process that exists today and anyone can use!

This is the process that worked for you, but it extends beyond the official process. Should the official process include any of these steps beyond step 1? Do the proposers themselves need to take care of everything, soup to nuts, or could there be a formal option to ask the community to take care of step 4, in case they aren’t confident in their JOSM skills?

Often, proposers need to consider how their proposals fit together coherently and how to stage them strategically. Last year’s street parking refactor didn’t include any kind of mass editing procedure, partly because we anticipated another refactor in short order.

More broadly, at what stage should proposers consult or at least consider software developers, and how much weight should their feedback receive? Proposals typically mention data consumers, but often only openstreetmap-carto, and sometimes only as an afterthought. This is not only an issue with deprecations. Infamously, the transit:lanes proposal enjoyed organic growth for more than three years until editor developers unanimously vetoed it down. It included detailed algorithms for renderers and routers, but the routing engineers at Mapbox were quite unsure how to actually implement the handwavy algorithm in their real router.

By contrast, proponents of street parking refactoring recognized that the wiki doesn’t operate in a vacuum; they proactively reached out to the developers of StreetComplete, the parking lanes viewer, and A/B Street and even implemented a replacement JOSM map style. Maybe a less ambitious refactor can get away with less. For example, my proposal to formally allow units in ele includes a best-effort survey of existing data consumers, but as the proposal already reflects the status quo for the most part, I don’t expect to conduct quite as much outreach to developers ahead of a future vote. (For the record, the proposal is on hold until the U.S. officially deprecates the survey foot. If you think OSM’s deprecation process is torturous…)