Can you search for specific tag deletions throughout history?

I’m are that Overpass has access to “attic” data and can present diffs between dates, but is it possible to search the “attic” for changesets that removed a specific tag while leaving another in place?

My specific use case is that I recently noticed and reverted a change that removed a boundary=administrative tag while leaving behind the type=boundary part on a relation effectively detagging a country’s borders. I know that that specific country has had that happen once before and am wondering how often this sort of thing happens globally. I imagine such tagging errors don’t last long so the current state of the map will likely mask most instances.

In both cases these were brand new users that also did fairly minor edits that seem more typical for a new user.

Is there a tool that will do this without me having to run my own instance of something with the whole history of the planet file?

1 Like

This should be possible with OSMCha, at least if the site would work.

You can specify a date range:

An Area:

And of course the affected tags:

There are many more filters that could be helpful.

But at least currently the page does not work. From my point of view it is a stroke of luck that it loads at all.

2 Likes

I will play around with that but don’t hold out much hope. I originally found out about the edit from an osmcha feed on the bounding box (polygon searches seeming to be broken) but the changeset didn’t seem to be flagged for ‘modification to a place’ from what I noticed so I’m not sure if it is concerned with deletions.

I wrote a tool osm-tag-csv-history which creates CSV files of the changes in tags. If you know your way around unix command line processing, or CSV files, you could figure it out.

specifically, this gets you a list of all changesets where someone deleted a boundary=administrative tag from an object:

osm-tag-csv-history -i HISTORY_FILE.osh.pbf -o - --tag boundary --columns tag_count_delta,key,old_value,changeset_id | grep -oP '(?<=^-1,boundary,administrative,)\d+'

You’d need to figure out how to get the list of objects that had that before.

Not so much I’m afraid, would have to see if one of my old laptops where I decided to “give Linux a go” was in a bootable state.

Relevant change: achavi - Augmented OSM Change Viewer  [attic]
Fix in: achavi - Augmented OSM Change Viewer  [attic]

Overpass did indeed store the information in one of its binary files that the boundary=administrative tag has been removed at 2022-09-13T16:29:45Z. However, there’s no easy (and fast) way to access that information from Overpass QL. Bummer.

I know of some people who keep a list of known boundary relation ids, and check them from time to time using an Overpass query, to see if they still exist, or have changed in the meantime.

Windows Subsystem for Linux might be an option for this sort of thing, or Docker (but Docker is probably more work)

Downloading a 109GB full history planet which gets updated once a week doesn’t seem to me like the most efficient approach for this kind of problem. And local history extracts wouldn’t come with the global visibility you’d need for this use case.

Docker has previously said it will break the VM software I have installed if I install it. I might have another run at WSL although I’m not too keen on running commands I don’t understand on my main machine if I might break something.

@_ᚐᚋᚐᚅᚇᚐ_️ any chance your tool would compile on a raspberry pi? I’ve got one I’ve not had a chance to play with yet and it should be easy to restore if something goes wrong.

As a first pass guess I’ve got a working overpass query for boundaries without boundary= but with admin_level which should normally be an error. Can overpass give a full history file for the results that can be run through a filter? From spot checksmost appear to be false positives (never had a boundary= tag)

Of course this doesn’t catch any ones that have been fixed …

This query would give you the history for a single relation back to Sep 2012, but it takes some time. It’s not exactly the kind of performant query I had in mind for this issue and it wouldn’t scale to a larger number of relations.

timeline(rel,547469);
for (t["created"])
{
  retro (_.val)
  {
    rel(547469);
    out tags meta;
  }
}

One way to significantly speed up this query is to limit the versions to a given time range. The following query would print relation versions created since January 2022:

timeline(rel,547469);
for (t["created"])
{
  if (_.val >= "2022-01-01T00:00:00Z") {
    retro (_.val)
    {
      rel(547469);
      out tags meta;
    }
  }
}

(Maybe also try this overpass turbo deployment, if other instances time out.)

https://api.openstreetmap.org/api/0.6/relation/547469/history would be way faster, and might be acceptable from a usage policy pov, if you’re intending to use it for editing the map.

Er, well you could always use whatever VM software that is! I’m guessing something like Virtualbox, some VMware product, Hyper-V etc…

I think that probably scuppers that approach then. If it’s a one by one thing it may not be any better than using the mapki history service and checking manually.

If Overpass can’t do this well then I think I’m going to have to eek out enough disk space for a VM or get myself a working linux box and do it local.

And this become a level more complicated if you want to look at associated changesets metadata.
I wonder if this would be a possible use case for Ohsome.

From what I can tell in their docs they only support looking at the status of tags in particular snapshots over time rather than being able to search for the changes to tags in particular edits? It might be a way to get element histories for candidate relations though.

If you are interested in the history for one (or a limited number) of nodes/tags/relations use Josm:

Screenshot_20220920_085058

Great way to review what a changeset did change a certain tag and almost instantaneous.

For nodes/ways just select the way and press the History button in the Selection Panel, for relations you have to explicitly select the relation.

1 Like

+1, ctrl+h in josm
. - - . .

@_ᚐᚋᚐᚅᚇᚐ_️
https://community.openstreetmap.org/u/_ᚐᚋᚐᚅᚇᚐ_️
any chance your tool would compile on a raspberry pi? I’ve got one I’ve
not had a chance to play with yet and it should be easy to restore if
something goes wrong.

Maybe. It’s a Rust programme. TBH the biggest issue could be the low CPU speed of a RPi. But sure, try it :slightly_smiling_face:

Exactly, that’s what I meant, it doesn’t scale to a larger number of relations, unless you have another query which would get all relations with changes since a give point in time. That list might be much smaller, and could be a starting point for subsequent queries for one specific relation.

I will try to find out if there’s any way to make the existing data more accessible, but this might take quite some time.