I’m are that Overpass has access to “attic” data and can present diffs between dates, but is it possible to search the “attic” for changesets that removed a specific tag while leaving another in place?
My specific use case is that I recently noticed and reverted a change that removed a boundary=administrative tag while leaving behind the type=boundary part on a relation effectively detagging a country’s borders. I know that that specific country has had that happen once before and am wondering how often this sort of thing happens globally. I imagine such tagging errors don’t last long so the current state of the map will likely mask most instances.
In both cases these were brand new users that also did fairly minor edits that seem more typical for a new user.
Is there a tool that will do this without me having to run my own instance of something with the whole history of the planet file?
I will play around with that but don’t hold out much hope. I originally found out about the edit from an osmcha feed on the bounding box (polygon searches seeming to be broken) but the changeset didn’t seem to be flagged for ‘modification to a place’ from what I noticed so I’m not sure if it is concerned with deletions.
I wrote a tool osm-tag-csv-history which creates CSV files of the changes in tags. If you know your way around unix command line processing, or CSV files, you could figure it out.
Overpass did indeed store the information in one of its binary files that the boundary=administrative tag has been removed at 2022-09-13T16:29:45Z. However, there’s no easy (and fast) way to access that information from Overpass QL. Bummer.
I know of some people who keep a list of known boundary relation ids, and check them from time to time using an Overpass query, to see if they still exist, or have changed in the meantime.
Downloading a 109GB full history planet which gets updated once a week doesn’t seem to me like the most efficient approach for this kind of problem. And local history extracts wouldn’t come with the global visibility you’d need for this use case.
Docker has previously said it will break the VM software I have installed if I install it. I might have another run at WSL although I’m not too keen on running commands I don’t understand on my main machine if I might break something.
@amapanda_ᚐᚋᚐᚅᚇᚐ any chance your tool would compile on a raspberry pi? I’ve got one I’ve not had a chance to play with yet and it should be easy to restore if something goes wrong.
As a first pass guess I’ve got a working overpass query for boundaries without boundary= but with admin_level which should normally be an error. Can overpass give a full history file for the results that can be run through a filter? From spot checksmost appear to be false positives (never had a boundary= tag)
Of course this doesn’t catch any ones that have been fixed …
This query would give you the history for a single relation back to Sep 2012, but it takes some time. It’s not exactly the kind of performant query I had in mind for this issue and it wouldn’t scale to a larger number of relations.
timeline(rel,547469);
for (t["created"])
{
retro (_.val)
{
rel(547469);
out tags meta;
}
}
One way to significantly speed up this query is to limit the versions to a given time range. The following query would print relation versions created since January 2022:
timeline(rel,547469);
for (t["created"])
{
if (_.val >= "2022-01-01T00:00:00Z") {
retro (_.val)
{
rel(547469);
out tags meta;
}
}
}
(Maybe also try this overpass turbo deployment, if other instances time out.)
I think that probably scuppers that approach then. If it’s a one by one thing it may not be any better than using the mapki history service and checking manually.
If Overpass can’t do this well then I think I’m going to have to eek out enough disk space for a VM or get myself a working linux box and do it local.
And this become a level more complicated if you want to look at associated changesets metadata.
I wonder if this would be a possible use case for Ohsome.
From what I can tell in their docs they only support looking at the status of tags in particular snapshots over time rather than being able to search for the changes to tags in particular edits? It might be a way to get element histories for candidate relations though.
Exactly, that’s what I meant, it doesn’t scale to a larger number of relations, unless you have another query which would get all relations with changes since a give point in time. That list might be much smaller, and could be a starting point for subsequent queries for one specific relation.
I will try to find out if there’s any way to make the existing data more accessible, but this might take quite some time.
