In my UK tool, I’ve only come across a few brands that I need to treat specially, e.g. to ignore certain types of objects that I’m not interested in. (My tool also treats multiple OSM objects that have the same website as being a single branch for the matching. So that allows you to not see e.g. a petrol station attached to a Tesco store as an unmatched OSM object.) There are a few ATP sets where I need to do some processing to change some of the brand:wikidata values to more specific sub-brands, but that’s not too bad.
There are a lots of cases where the branches in OSM do not have consistent amenity=* or shop=* tags and names=* values though - so I’m a bit surprised that matching on them is working so well for you. Perhaps if you had a single big table of sets of “similar” shop/amenity values then that would work with the way you’re doing it. I still think matching with the brand:wikidata values (possibly in addition to other criteria) is the best way to go. If you’re not worried about unmatched OSM objects, then I’d have though you could simply consider anything in OSM with a matching brand:wikidata in addition to the shop/amenity tags you’re already searching on to improve things. (Surely that would allow you to pick up some of the examples I gave of your tool missing matches to OSM objects.)
In terms of possible shop/amenity values for each ATP set, you could try looking at the tagging of existing OSM objects with the right brand:wikidata values. Anything that appears more than once or twice would probably be a good candidate to match on. See the different values for Dunelm at https://osm.mathmos.net/chains/Q5315020/
What I do with the postcodes is use them as a secondary signal for the matching. If there’s a matching postcode then I allow a match over a greater distance (currently infinite). This only gets me into trouble very occasionally if there’s more than one branch of a chain with the same postcode. You could do something similar with website=* matches. It might be dangerous to allow matches on the website over too large a distance though, since someone might have added the wrong website to OSM. At least with UK postcodes, there’s open data on their locations, and I have another tool that flags any addr:postcode=* values that seem to be mis-located.
Doesn’t Daylight distribution have a schema that could be used here, where POIs are grouped into subcategories? I believe it’s “subclass” in that table, this could be nicely reused to make groups on the OSM tags like tourism, food, etc
On a related node, SCEE (StreetComplete “Expert edition” fork) already includes OSMOSE support.
But as noted elsewhere, including ATP/OSM mismatches in OSMOSE is likely more problematic then useful.
You’d also likely need 1000x times more manpower. Perhaps ATP could allow (logged-in - OSM Oauth2? GitHub?) users to rate the quality of individual scrapers (i.e. good / usable / bad) and display that to potential users. Maybe someone should suggest something like that.
Not necessarily more, it can be also done with extensive but expensive tooling. POI companies have this, AI helps. It doesn’t have to be correct, just has to trim down to the highest confidence stuff. Going being to reduce the hours needed.
But effort does have to go in somewhere. My point is that it’s raw data, therefore has the same problems as machine learning derived buildings form satellite imagery and so on, except that POIs also go outdated faster than buildings or roads.
I assume that it is not a problem as we are taking their lat/lon info, not copying Google Maps. And anyway just because I overlay location marker on OSM background (or set of them) it does not make my data becoming ODBL licensed?
Though is that some kind of a blocker? Or potentially worth asking LWG about? I guess that it is in theory possible that they copied opening hours and locations info of own shops from Google Maps?
Correct. You are getting data directly from first party, not some 3rd party like google maps ,thus LWG interpretation allows you to operate on that data (including adding it to OSM).
Correct. Those extra marker locations are part of “Collective Database” (as set in Open Data Commons Open Database License (ODbL) v1.0 — Open Data Commons: legal tools for open data section 1.0 Definitions of Capitalised Words) and is explicitly not considered part of “Derivative Database”, and thus does not need to be republished under ODbL (unless you also publish your geospatial database containing both them and ODbL parts of Collective Database [i.e. OSM data], in which case 4.2 applies).
But in your case, those separate layer of markers is not “tainted” by ODbL of other layer (i.e. OSM basemap).
That’s fine if the first party has worked out the location themselves. But what if ATP is taking the location from an embedded Google Maps widget, and that widget worked out the lat/lon by geocoding a postal address that the first party supplied?
I will do further checks, but cases I have seen is where site has lat lon` and instructs underlying map (Google Maps, OpenStreetMap etc) to show marker at that specific location.
I have found a few storefinders (example Sweaty Betty) where there is no latitude or longitude and a map is produced on their site by feeding the address to google maps - but in that case the ATP spider wont produce a geo location, just an address, so I don’t think this is an issue.
Us:
“Who owns these coordinates? Is it Google? We actually don’t use them for mapping, some people use them for matching. But who owns these coords? Lets spend the next decade deciding, FYI IANAL”
Perhaps ATP could allow (logged-in - OSM Oauth2? GitHub?) users to rate the quality of individual scrapers (i.e. 