London data corrupton

c2r · 2019-08-04T07:47:23.000Z

At https://www.openstreetmap.org/#map=14/51.5510/-0.0173 I’m seeing what appears to be large scale corruption of map data, with a pattern of closely spaced vertical roads being shown. It looks like it might be programaticaly created? Is anyone looking at trying to revert it all?

wambacher · 2019-08-04T07:59:19.000Z

c2r:

Is anyone looking at trying to revert it all?

I think, yes: https://www.openstreetmap.org/user/MacLondon/history

don’t panic

walter, Germany

PaulRichardson · 2019-08-05T15:32:57.000Z

Hello,

This problem is still apparent within central London at zoom >=16 and maps become unusable as a result.

Has the revert been made but perhaps not updated yet or still cached etc?

Thanks,

Paul

SomeoneElse · 2019-08-05T16:32:34.000Z

(for the avoidance of any doubt) - yes, the oroblem was fixed around midday yesterday. Depending on what map you’re looking at, where it gets its data from and how any tiles are cached you might still see a problem, but it’ll get fixed as soon as the problem tiles are refreshed (something that happens periodically as edits are made).

kreuzschnabel · 2019-11-07T21:15:48.000Z

Happened again today in https://www.openstreetmap.org/changeset/76764349, which has been fully reverted by me in https://www.openstreetmap.org/changeset/76773501.

Parts of Oslo were affected too this time. The nodes shifted coincided with nodes of building polygons already mapped in this area: https://www.openstreetmap.org/#map=17/53.13141/50.13179

How on earth could this happen?

–ks

SomeoneElse · 2019-11-07T21:49:24.000Z

kreuzschnabel:

Happened again today in https://www.openstreetmap.org/changeset/76764349, which has been fully reverted by me in https://www.openstreetmap.org/changeset/76773501.

Firstly - thanks for noticing and fixing so quickly!

kreuzschnabel:

Parts of Oslo were affected too this time. The nodes shifted coincided with nodes of building polygons already mapped in this area: https://www.openstreetmap.org/#map=17/53.13141/50.13179

How on earth could this happen?

I orginally suspected a “JOSM node drag” (of which we’ve had a few before) but as Richard on IRC noted, it’s more likely a pre-existing .osm file (with numbers that matched existing nodes) was involved. That might explain how it hit nodes in both London and Oslo from the same

kreuzschnabel · 2019-11-07T22:36:34.000Z

SomeoneElse:

Firstly - thanks for noticing and fixing so quickly!

I couldn’t help it – I was working on the https://www.openstreetmap.org/#map=14/51.8789/9.9066 area which happens to be just on the beeline

Here’s a JOSM screenshot of the London roads’ eastern end:

–ks

wowik · 2019-11-08T05:51:29.000Z

London is doing brexit

alphensebezorger · 2019-11-08T12:06:54.000Z

SomeoneElse:

kreuzschnabel:

How on earth could this happen?

I orginally suspected a “JOSM node drag” (of which we’ve had a few before) but as Richard on IRC noted, it’s more likely a pre-existing .osm file (with numbers that matched existing nodes) was involved. That might explain how it hit nodes in both London and Oslo from the same

I happened to notice that both this recent changeset and the ones in August seem to only move nodes with id’s roughly in the range between 106000 and 118000. Does this suggest a JOSM or OpenStreetMap API bug? Or is there a more likely explanation? Could the JOSM option to upload the data in chunks be related somehow?
Both users seem to use JOSM version 15238, although that could be faked I suppose.

Hopefully a better understanding of the causes of these changes can avoid future problems.

SimonPoole · 2019-11-08T13:43:50.000Z

alphensebezorger:

SomeoneElse:

kreuzschnabel:

How on earth could this happen?

I orginally suspected a “JOSM node drag” (of which we’ve had a few before) but as Richard on IRC noted, it’s more likely a pre-existing .osm file (with numbers that matched existing nodes) was involved. That might explain how it hit nodes in both London and Oslo from the same

I happened to notice that both this recent changeset and the ones in August seem to only move nodes with id’s roughly in the range between 106000 and 118000. Does this suggest a JOSM or OpenStreetMap API bug? Or is there a more likely explanation? Could the JOSM option to upload the data in chunks be related somehow?
Both users seem to use JOSM version 15238, although that could be faked I suppose.

Hopefully a better understanding of the causes of these changes can avoid future problems.

Likely

JOSM or API bug
broken stealth import script

unlikely

actually moving the nodes.

Unluckily it is practically impossible to determine what went wrong without cooperation from the contributor in question (and even then there is no real way to do a proper root cause determination).

Simon

PS: SomeoneElse the thing is that the node ids are extremely low and essentially sequential, aka very very early data and very unlikely to be present in any accidental extract, so likely something is going very wrong with the placeholder ids in one of the three potential culprits.

SomeoneElse · 2019-11-08T13:54:27.000Z

alphensebezorger:

I happened to notice that both this recent changeset and the ones in August seem to only move nodes with id’s roughly in the range between 106000 and 118000.

I’ve just looked at the August one and that includes node IDs over 25,000,000 so I don’t think it’s the same sort of thing. I suspect that that was a “normal” node drag that is all too easy to do in JOSM.

alphensebezorger:

Does this suggest a JOSM or OpenStreetMap API bug?

I don’t think it’s an API bug - that pretty much just does what you tell it to, and if it “occasionally updated the wrong object” we’d know about that by now.

With JOSM you could argue the case that it should at least have displayed a suitable warning dialogue in front of the user, but of course we don’t know that that didn’t happen. With a DWG hat on I often use JOSM to break referential integrity in the data when doing a revert, because “how well things are mapped” will still be better afterwards than before. JOSM’s conflict resolution dialogues are actaully enough of a pain for me to tend to use the perl revert scripts for larger reverts, and patch up the problems afterwards. It’s really a question to which there is no right answer.

alphensebezorger:

Both users seem to use JOSM version 15238, although that could be faked I suppose.

Although faked version numbers of JOSM (and iD) are definitely a thing that spammers et al sometimes use, I have no reason to believe that either user here was using other than a “normal” copy of JOSM.

daganzdaanda · 2019-11-09T00:51:17.000Z

Seems like many of the node IDs involved were very old (6 digits) and the rest were mostly new and mostly in sequence.
Quite a lot of the old node IDs had been deleted/unused for a long time:
https://www.openstreetmap.org/node/106698/history
https://www.openstreetmap.org/node/108772/history
Does the API automatically “recycle” unused old node IDs?

Maybe someone who knows Russian could contact the mapper to ask what plugins or scripts s/he was using for adding the houses in Samara?