"AI-Based Country Scale Road Import"

Another example that Facebook prefers to work in secret:
They gave a talk with a lot of buzzwords at SOTM us. But declined to have a video recording being made available so that the community can follow their ideas.

http://stateofthemap.us/2016/how-can-ai-help-us-make-maps/

@Facebook: Stop hiding. Open up your communication with the community and share the talk.

As others have stated, the Data Working Group doesn’t give you permission. The community will give you feedback on it. And you haven’t even taken the first step by emailing the imports mailing list!

No, of course not! But in that case there is still more things to map. Buildings (heights, names, postcodes), amenities, landuses, better and more accurate road geometries, road attributes (speed limits, surfaces, lanes, sidewalks, signage, stop signs, traffic lights, road widths, streetlights etc). OSM will never be finished!

It sounds like this Facebook team don’t want to contribute to OpenStreetMap. They appear to want a generic geo database web host. This project is open, we work in the open, we share things in the open, we all work together. If we all don’t have access to the digital globe imagery, then maybe it shouldn’t be used. If you don’t want to discuss your import in the open, and want just find the right high level executives to approve your import, then maybe you don’t really want to contribute to OpenStreetMap.

We shouldn’t throw out the open just because someone says they have nice imagery.

Please work in the open. Please share with us. Please share your imagery. Let’s all work on this project. Stop trying to hide yourself away, and refuse to share your toys. Things are much better if we work together.

As someone who has been active in India for the last 10 years, I can safely say that OSM is struggling here because very few see any point drawing millions of kms of roads across this vast country, most of which are already there on other maps. The majority of contributions of roads in India have come from European mappers, and maybe we should treat that as good as an import as well.

That said, having these roads on the map has acted as a seed for some basic data that makes the map be of use and allow the collection of more valuable data that can only come from local knowledge like streetnames and POIs. In India, we are already seeing that by the way of maps.me contributions from the mobile which is the primary way to access the internet, and would not be possible if there were no roads to begin with.

OSM will be a truly open and global project only when it can build sustainable communities on the ground maintaining the map as stephankn said. If we have access to high quality of open road geometries from FB, it seems like a huge opportunity to improve the quality of the map and boost the utility of OSM, which in turn brings in more editors. Its unreasonable to expect OSM communities in developing nations to form like they have in Europe over beers when basic needs like continuous electricity or safe drinking water is not met. Maybe different regions need different approaches, and this seems like a worthwhile effort if endorsed by the local mappers.

Yes!

Interesting commentary and feedback. I’m always angered when I run into copyright issues because they retard the growth of knowledge. Okay, that’s just the way the world works and I must accept that. I also see the vast amount of mapping that still needs to be done in my primary areas of interest, Thailand and Alaska. I am appreciative of any level of assistance with getting data into OSM because that is truly going to be the proof that the open source model works. When OSM can give Google real competition for offering the best map of the world, we’ll have proved that point. FB could be a valuable ally in helping us reach that goal. Like it or not, we depend on some pretty big outfits to be able to do what we do. Where would we be without Bing?

I have a question about licensing and EULAs. How it is that the FB group can extract information from the DG visual data if that information will be used in OSM, where there are virtually no limits to the further, subsequent uses of that data? Do FB and DG realize that once “their” derived data is in OSM, they have no control of it anymore? That someone may decide to create a map of wheelchair accessible bathrooms in Chiang Mai and turn around and sell that map to anybody with a few baht to spend?

I think we should take a step back, allow the group some latitude (pun intended), let them make a few mistakes, have them participate in this discussion, exchange ideas and advice, and see where it takes us. Only by discussing this important work here will we understand enough about it to feel good about giving the go ahead.

Hi all, I wanted to chime in both with a personal perspective, and one that represents the company I work for, DigitalGlobe.

On behalf of DigitalGlobe, I can confirm that Drishtie’s (Drish’s) statements are accurate and reflect the current agreement between Facebook and DigitalGlobe. Speaking for DigitalGlobe alone, we continue to work with both Microsoft and Mapbox to update satellite imagery available within their respective layers available in the OSM editors. In addition, we are looking at options to publish imagery for OSM editing similar to the imagery that is being used by Facebook.

WRT the copyright comment, I spoke about this at last year’s State of the Map US. In short, DigitalGlobe is a data licensing company. Our goal is to continue to develop, launch and operate really nice satellites so the world continues to benefit from detailed and accurate scientific observations from space. This includes the OpenStreetMap community who have been using DigitalGlobe data by way of Bing and Mapbox for many years. We just want to continue to do that for many more years. You can see my presentation here: https://www.youtube.com/watch?v=1dh60heskmo&index=40&list=PLqjPa29lMiE3eR-gK80irr3xdUiRbIMeg

Last, as an OSM contributor for 5 years, being involved with several State of the Maps both presenting and organizing, I welcome and encourage the work that Facebook is doing to support and improve OSM. We, as a community, must work together; and Facebook has clearly stated it’s willingness to work with us. Let’s continue to be respectful, and acknowledge the commitment Facebook has with OpenStreetMap.

Thanks, Kevin Bullock

Hello all,

Thank you again for all of this feedback. We joined this forum to work with you directly and support high-quality data with your expertise. We are always willing to share and answer any questions you have. My sole focus at Facebook is on OSM, so you will reach me and my team directly if you talk to us through these discussion posts or email us at osm@fb.com (please email if you need a quicker response). Our goal is your goal: to efficiently and accurately create maps powered by our community.

We look forward to continuing to share our derived data so the community can work together to create high-quality maps. We will be sharing sample data with the community next week via the import list. Following that, we’ll share the area we hope to start mapping in Thailand through this forum.

We are currently using the HOT Tasking Manager and iD tool and are working to share our editions/changes to these tools as well. Stay tuned.

Have a lovely weekend wherever you are!
Drish

Thanks to Kevin for clarifying the intentions and philosophy of DigitalGlobe. We are indebted to the people who produce the satellite imagery we use every day and without which we would be lost.

Drish, I wonder if the iD editor is the best one to use for this project. I am not all that familiar with it but the few times I have used it, I found it very limiting. The learning curve is steeper but I think using JOSM will save time and prevent problems in the end. It is very customizable. One can create and use “presets” for example, that make data entry more uniform. In an effort like yours, where many people are adding similar data, you don’t want some of them to tag an object one way while some others choose a different tagging scheme. Also, tagging an object with a preset requires only a couple of keystrokes no matter how many tags are involved.

It’s your project and your decision might need to weigh other important factors. Many of the most prolific contributors to OSM use Potlatch or iD so you are in fine company in that regard. Still, I feel that you might as well be using the most powerful and flexible OSM editor available and that is, unquestionably, JOSM.

Best Regards,

Dave

Another feature of JOSM is the inbuild validator which can detect common mistakes, like crossing ways, not connected way, bad tagging, etc. The Find and To-do plugin can help find and fix errors like this. In computer generated imports like this, you want to be able to automatically detect and fix problems to ensure a high quality of data.

The White Spots on the map of Thailand are still enormously big. Last December in Isaan, I was in places with nothing at all mapped within 10 km from my location. And let’s take a look at the contributors to the map of Thailand: most data were contributed by farangs.
We need some organisations helping us to complete some tasks. Roads, rivers, landuse can be seen on images. Place names, road numbers, shops, hotels, etc. can’t be detected there: we need people on site who will do that. And I hope they’ll more likely be willing to contribute when they see that the map is good (which it isn’t - also Google maps actually isn’t good as it does not differentiate between roads and tracks).
Although the examples shown by Stephan and Dave are very terrible, let’s try to integrate the FB team into the OSM community. With a slower approach, they could get feedback on small change sets, without causing too much disturbances with bad data. Also try to map connected items instead of singular segments. As I see from Dave’s comment, it could be helpful if the FB team tried to work in places where good and recent imagery is available also to us (instead of only to them).

Hello Dave and Rorym,

Thank you for the advice on tooling. As mentioned in our wiki page we do use both JOSM and iD. We totally agree that JOSM is superior for what we are trying to do and are all familiar with both tools. In fact, we started primarily with JOSM for the exact reasons you mentioned :slight_smile: In order to create a better internal process, our engineers have made changes to the iD editor to mirror the cool features JOSM has.

For example here are a few changes we have made to iD that help us be more efficient.

  • The ability to load/save locally generated .osm files much smoother and faster.
  • Customizing colors and highlighting specific point, lines or polygons. For example, we highlight roads so editors can quickly tell the difference between our edits versus what is currently in OSM.
  • Added conflict validation to iD so edits cannot be saved unless all conflicts are resolved.
  • Batch tagging for multiple selected roads
  • Automated suggestion for road connections. So we can easily connect roads in neighbouring tiles when working in the bounding box of a task from the tasking manager.

We are happy to share these and are working on ways to do this in a thoughtful way.

Best,
Drish

These sound great and make powerful editing features that were only available in JOSM more accessible to mappers. Looking forward to a publicly usable fork.

Again FB shows that they are not interested in cooperation with the community. Instead of presenting their ideas and working together with the community on improving the tooling they speak about creating their own toys and a fork.

Besides that:
When will the existing mess be cleaned up? Does Facebook expect the community to clean up after them?
I’m tempted to immediately revert uploads done by RVR007 and others. Removal of rice-field tracks are no big loss.

Duplicates for example in those changesets:
https://www.openstreetmap.org/changeset/46112980
https://www.openstreetmap.org/changeset/46117993

The number of duplicate geometry skyrocketed from non-existent to 10.000 Duplicated in 2017 thanks to technically bad executed Facebook imports.

Given that it’s likely that most of these errors were introduced by Facebook, it would be good to see a response by e.g. Drishtie about this here.

Further up this thread I said:

and that still stands. There are a number of tools around that can “revert everything that hasn’t been since touched by another mapper (e.g. to correct it)”. In this case it’s not quite as straightforward as it might be because non-import accounts used for other mapping activities have been used for the import.

Although I’d be normally very happy for a person who made a buggy import to revert it themselves, given the lack of technical competence demonstrated by Facebook so far I’d personally be somewhat concerned if they did the revert on their own without anyone else checking it. If Drishtie or someone else at Facebook wants to do the revert, then it’s best that they say so here before doing so. If someone from this forum wants to do it, great - but probably best to say so here first to avoid duplication. If not, the DWG can clean up the data (which as I understand it was all imported by mistake), and if I do it I’ll say so here first.

Best Regards,
Andy (DWG)

For those not on the imports mailing list, Facebook have posted to it about this import: https://lists.openstreetmap.org/pipermail/imports/2017-March/004840.html

Hello Andy and Stephan,

Thank-you for your feedback. We are looking into duplicate geometry and will make sure to fix that example you shared and any others we find.

Best,
Drishtie Patel

Hey Stephan – That is useful info. I am wondering how you created this chart? (Could be useful for other mappers).

Also agreed that this should be cleaned up. If you can supply the data (as OSM or GeoJSON) I can make a MapRoulette challenge from it.

Perhaps as a community we can figure out how to help FB do better?

@Stephan I’d also be interested in seeing where you came up with those numbers.

From what I can tell from OSM Inspector, Thailand has barely any issues with duplicate ways (only about 10 ways have issues out of the entire country - in my opinion that’s pretty darn good!).
https://tools.geofabrik.de/osmi/

Out of those 10 ways that have issues only 1 of them is from Facebook and another is from yourself @stephank, you don’t validate before pushing your edits?
http://www.openstreetmap.org/way/474870936

Stephane, next time you try to use a fancy outdated graph to threaten a revert, please try to actually look at the OSM data before you speak.

bye

@Stephan: Have you also considered the cause of the increase duplicate nodes from PokemonGo vandalism.

I was able to find this changeset which was flagged as “Overlapping Highways” (clearly not a Facebook edit).
https://www.openstreetmap.org/changeset/45552933

@Denis: way 474870936 was one of the tagging fixes for Balthus. It fixes only tags. If there had been duplicates before they had not been touched by this. Was not the scope of fixing the tags.

The statistic comes from Osmose checks.

Check for yourself.
http://osmose.openstreetmap.fr/en/errors/?source=5832&item=1230&class=5

I opened the first five nodes on the list:
http://www.openstreetmap.org/node/4689300837
http://www.openstreetmap.org/node/4689300837
http://www.openstreetmap.org/node/4691069588
http://www.openstreetmap.org/node/4691069588
http://www.openstreetmap.org/node/4691071904

Don’t try to blame pokemon users. These mentioned changes all have “#nsroadimport #thailand” listed in the comment. This clearly points towards Facebook.

Again: It is Facebook doing imports. So it’s Facebook to prove they are executing this in a technically correct way. We have enough examples indicating this is not the case!

And don’t try to switch topic. Yes, there are other bad edits in the database. But here we are discussion about those caused by Facebook.

Facebook, you still did not tell use what user names you are using for the import. Why? What do you want to hide?

Could it be that this user also belongs to you? His patterns match what you are doing…

http://www.openstreetmap.org/user/Tanavut