Failure to verify result of the edit on the main Mapnik view
This is just the first set of my encounters. And these are just from less than 0.1 percent of their edits; I have spotted many, many, more “oops” attributed to them that I don’t yet to find time to verify, and some are also dangerous for routing. All of these are done without access to heavy-duty anti-vandalism tool.
And I will kindly let you know that, while they “looked” cooperative in their response; it doesn’t do much good, because:
Too many damages had been done.
Faults are found after-the-fact via independent ground verification by me.
The firm does not seem to have a proactive policy to prevent or fix their own errors, and…
They completely lacked proper training necessary to fix the problem themselves; or to make a clean, usable map in the first place.
They don’t use OSM, as seen from their unawareness regarding main Mapnik render.
Their edit lacks ground truth.
Dealing with their edits in a “proper OSM way” (audit, survey, revert, fix, comment) is also actually a huge time sink in my solo mapping expedition, which is billable to no one. I did these anyway as an effort to uncover incompetence and lack of training of Grab’s contractors that would be otherwise be swept under the rug.
The encounters are really eye-opening, and they completely flipped my opinion on armchair mapping and imports from mildly positive to overwhelmingly negative. And yes, “living there” matters.
If there are other companies planing to “involve” with OSM like this, you can count that I will be there to object it.
I’ve created an overpass query so that you can see the extent of the edits made by the Grab / Global Logic Team in case anybody’s interested. I’ve imported it into JOSM and have found a whole bunch of errors already, enjoy!
I noted at our meeting with Grab, was that they clearly reiterated that Global logic had stopped playing with the map.
Of course the problem is now obvious … we keep coming across all those “optimistic road connections” by chance, and on two occasions, I have written to the GL mapper involved (via a changeset comment), only to get no response.
As with the Facebook employees, I expect they have all have moved onto other things.
So, as they are not going to fix their mess, DWG - how difficult will it be to revert all their work in one fell swoop ?
Edit - I just had a look at the Overpass query, and it essentially shows any way they have got their grubby hands on. Most might have just been the moving of a node or a simple realignment of existing way.
As our (or at least my) biggest gripe is the creation of roads that are not there… can the Overpass query be modified to just shows ways existing as Version 1 that were added by GL ? If not, maybe those roads lacking a source as they were too lazy to show us what aerial pic they took the inspiration to dream up that road from !
I’ll start cleaning up CM, but Mishari, you had better make a start on BKK.
Here are some notes that I took from the meeting, any mistakes are mine alone:
Present from Grab were Ajay, Pimlada, Adrian (sorry I didn’t get the name of the others)
OSM Side: jrcarlsen, AlaskaDave, Russ McD and Myself
The overall atmosphere of the meeting was good, Ajay was quite sincere about working with the community and displayed good faith.
Ajay said if we need any high res imagery he’s happy to coordinate with the imagery providers and get it for us.
Grab has also started collecting imagery for Mapillary The idea is both Grab and the community will have a single source of truth.
We agreed on the principle “If you don’t see it, don’t map it” this should eliminate many of the errors. Grab has agreed to go back and rectify problematic edits.
Grab has also agreed to put their Quality Control Standard Operating Procedure document on the Wiki (it should be up soon)
– end note–
I would like to add that the document should give us insights into what is causing the quality issues. If we can work on a modified document that brings the quality up to community standards, there’s no reason why we can’t have a successful collaboration.
Yes, but the document, apart from being written in a grammatically confusing way, does not really address the main problems we have seen…
GL added many roads that clearly did not exist on the ground. They also chose to change the status of roads from what local mappers had already entered. Clearly the validation did not pick that up.
I also see in the Techcrunch article https://techcrunch.com/2018/12/19/grab-maps-osm-thailand-southeast-asia/ they claim 0.03% were wrong. This is misleading as they are basing this number on what local mappers have stumbled across and taken the time to report via a changeset comment. There were hundreds of other errors we simply corrected without reporting due to the time consuming process of writing a changeset comment, and the the fact that these comments are now going unanswered.
Im positive there are hundreds of more “mistakes” yet to be found in Thailand, and while Grab appear to display a willingness to fix these, I have yet to see actual action.
Moving forward, I would want to see the Quality SOP include some concrete rules, such as adding an source (or import tag) to easily identify the work of commercial mappers, and instructions not to change things such as road status & Geometry. I also believe they should be adding fixme tags where they are unsure adding a significant new road actually exists on the ground. Im sure there will be many more rules we can ask for.
So while I appreciate Stereo’s response from the DWG, we can’t agree locally on a total revert, so no point going down that route. There are only two options, one being we fix ourselves, and the other being that Grab instruct GL to go over every edit and delete all connecting roads and status changes that they made. After that, using the recent Grab Mapillary images, they can re-add only those roads where a proven connection can be made.
Exactly, that’s our normal way of handling errors: just fix them when we detect them.
On the other hand, that makes a calculation of an error rate impossible: not every change to some mapped features implies an error.
I prefer to silently correct - I may report in some cases, but overall that’s far to much effort: during a mapping session, an error is corrected in 3 seconds, a report written in 2 minutes.
Thanks for the feedback. I do agree with you but I don’t think it’s possible to establish or articulate all the rules or process in one go. I would prefer that we take the mappers through a PDCA cycle in small areas to experiment with the process before signing off on it and letting Grab expand it to other areas.
Simply put, the cycle is: Grab do edits, we give feedback, it’s implemented as a process, edits are then attempted again. If this is found to be completely hopeless then we can engage DWG but in the meanwhile, it’s worth a gamble and for my experience, produces excellent results.
Bernard, I agree with you, but the point is that we can leverage, so the 2 minutes giving feedback can be applied to hundreds if not thousands of edits.
Ideally, it would be good if Grab can just watch their edits. I wonder if such a tool exists - let’s say I wanted to track if someone made any changes to one of my edits, how would I go about doing this?
Ugh. I admit I avoided looking into the entire fiasco when it was happening, but now, seemingly whenever and wherever I zoom into the map in the Greater Bangkok area, I’ll see fake non-existent streets, bad connections, or other sloppy, patently incorrect additions left by the Grab editors, especially sridhar1. It’s probably too late now, but maybe we really should just have reverted all of their edits back when we had the chance.
As it is related: I did add a prominent Info-Box at the top of the Thailand wiki page to emphasize that Organized Editing has to contact us first and that we ask for high-quality mapping, especially from organized (potentially paid) mappers.