I’ve contributed with mapping locally in Sweden for a number of years, and recently I’ve increased my activity, but I’m a bit worried about OSM project as a whole. As an outsider, I think it seems a bit stagnant and unable to adapt to the new situation where there are lots and lots of public high quality map data available.
While the mapping of major cities look quite fine in Sweden, less populated areas are not mapped well at all. It would take tens of thousands of hours to map these areas manually. So of course it should be imported when public sources are available, but who’s in charge of that? How is the data synchronized when the source is updated? And how to merge with all manual work already done? I’m a bit hesitant to contribute with manual mapping as it’s basically a waste of time when a huge import from the government issued high quality geodata would do it all. But now it’s 2020, and here in Sweden the government map data was released in 2015 (if I remember correctly).
There seems to have been some more or less successful attempts in smaller areas to import, some look good, some look like crap with various technical issues and if you ask me needs to be reverted and remade from scratch.
To me this seems like both a large technical challenge and about organization and review. I think it’s obvious that OSM as a project, if it had centralized technical leadership, should have very high priority to import these type of data sources. Imagine all the hours of manual mapping that has been done in Sweden during these 5 years which already exist ready to import. Those mapping hours could be spent in areas where there is no data.
For example, in the Netherlands, buildings and addresses were imported and are kept up to date based on the BAG database: https://wiki.openstreetmap.org/wiki/BAGimport This data has been found to be detailed, well-aligned and is kept up to date by the community.
Anyone can import the data as long they know how to convert it to a OSM usable format, for the bulk imports the community has to be informed of course. I think the bulk of this import happened over 5 years ago. For keeping data up to date, people can leave map notes or report on the forum that an area needs to be updated, and someone who knows how to import the data will make sure to update the area and merge with existing data.
If you have contributed for several years, you are no outsider any more
While there are many contributors that agree with
there are also many that disagree. Some of the reasons are “if we want to use just the same data as everybody else, then importing is a waste of time, just use the original data instead”, “no-one feels responsible for imported data, so it is getting outdated unnoticed”, “it turns away contributors because it looks like there is nothing to contribute any more in the area” and “we devalue the work of mappers by using imported data instead, demotivating mappers by doing this”. Also, there are frequent problems with fine points of the license on public data, and some times the quality of public data is worse than it looked on first sight.
Coming back to your original question: There is no central controlling body in OSM, things only happen if there is someone to make them happen. So if you want to see some data imported, your best bet is to start an import project for that data yourself and to find other mappers that also want to work on this together with you. If you want to go forward with this, check out https://wiki.openstreetmap.org/wiki/Import and the pages linked from there to find out how to proceed.
Not a given. Some of it is absolute rubbish, and a glance at my avatar will suggest I’m passionate about keeping poor-quality data out of OSM, because we can do far better manually. Yes it takes longer, but garbage doesn’t decompose well in this ecosystem, and can be a royal pain to edit.
Long story short, please be aware of the quality (if any) of what you’re trying to import. Enquire locally.
I consider myself a beginner as I’ve mostly only mapped simple things like roads, and fixed alignment errors and tracing issues and such. Only recently I’ve tried to move into making more complete coverage on an area from scratch (as one generally have to start from scratch here, still).
I’m aware that the situation varies wildly in different countries, a strategy that works for one country is not necessarily the best for another.
Here in Sweden the government maps are of very high quality and has been that for a long time, we are a very bureaucratic country which means that the government has spent lots and lots of money to keep high quality detailed maps. It’s hard to find errors in them, except for very new things were the publically released data hasn’t been updated yet.
The OSM data for Sweden on the other hand is actually very poor in 80% of the country, almost no detail and lots of errors small and large. It’s hard to recruit Swedish mappers as they don’t see the point when there’s so high quality maps easily accessible, and when you look into OSM it’s just so far behind, it feels like infinite amount of work just to reach something barely useful, that many lose interest right away. It’s about 50 mappers active in the whole country, and I recently was placed top 15 being active about 10-15 days per year or so.
So obviously the map will not reach any reasonable state in any reasonable time in this country if we rely on manual mapping alone.
But I understand, it’s up to individual mappers to make these type of imports. I’ve looked in the Swedish subsection on this forum which isn’t active, there are some comments about the data but it doesn’t seem to be any organized effort. Maybe I should look into this myself… probably the most reasonable way is a semi-manual import where you manually merge with the existing, so it will still be very time-consuming task, but about 100 x faster than doing everything manual.
It should be said though that the data that has released for public use is not the 100% most detailed, so one still have to map buildings and such. But detailed land cover and roads, railways and lots of naming etc is there.
Compare current OSM data with Bing, and then with “Lantmäteriet Topographic Map” which is the official government map. In the largest city areas it generally looks good, but 70% of the country is smaller cities and rural areas, and there it only occassionally is detailed, and if detailed often with significant alignment errors as it was drawn in a time where there was no good alignment data available.
Do they contain data like POI or are they routable? For Poland, the new Surveyor General who is enthusiastic about OSM and open data, pushed a bill to free the official geodatabases. However the BDOT10k for example does not have these two features mentioned above - it’s suitable mostly for making topographic maps and some GIS analyses (and I presume it’s not as up to date as Swedish maps are).
One of the strengths of OSM lies in its extensible data model. Official maps must adhere to spec and don’t record anything outside of it. Changing the spec is a big deal.
It’s certainly not “everything”, the data that is available for public use is far from all and not the most detailed, so there will still be work to do for manual mappers :-). The data that is there is accurate for the detail level it represents, but you could map more.
I think the most gain is in more rural areas actually, where the density of volunteer mappers is low and there’s lots of features in nature. Lake outlines in OSM Sweden today is often very coarse and outright wrong in many cases, based off old satellite images with low resolution, alignment issues and lots of clouds in them. The government data would fix such issues. I recently mapped a 40x40 km segment with mostly nature, it was about 700 or so separate wetlands in complex shapes. It is tiring to make all those by hand (which I did) knowing that they exist in a public database with high quality.
The data doesn’t contribute that much to cities though, as they haven’t released buildings as far as I remember. So it’s far from just importing and not having to do any work. But it is a good starting point, and for a country like Sweden with lots of nature it does make much of the work.
There is a road database too, I’m not currently familiar how much of that is public, if it’s only the ways and names (like you can see as a layer in the iD editor already), or if there is more. Roads and names would be great to have though, as that often is missing or drawn with poor precision currently due to the limited source material at the time they were first made.
I also think that much of the naming of natural features will need manual work, as I think they are in the government data separate manually placed labels “for the renderer” (it’s made to be ideal in a specific scale), and we don’t want that in OSM. So it will certainly not be just pressing a button, but even with manual review and adjustment of each imported element it will be much faster than drawing them all from scratch.
A thing that I think would be nice is if the OSM database could have some layer support built in so one could make a top level “raw import” centralized for the whole country into an invisible layer (made by some special OSM user with some super-user privileges that can make huge imports), and then individual mappers could use such import layer as a source layer to merge data from into the main database one part at a time. So a mix between import and manual.
To make that workflow possible there would have to be some technical features implemented in the OSM, and as I started this thread I am somewhat worried that the project has not been able or not cared to respond and make use of the potential of these new opportunities that public data does provide.
Much of Scandinavia is sparsely populated so we are short in terms of mapper per km2, and hence the quality of OSM range from great in one area where a frequent mapper is active to near-unusable in another, which is a bit sad map state for developed countries in 2020. However much of Scandinavia (all?) has good public data available too, so if we just can get the most out of that I think the starting point would be much better, and the mappers that actually are active in these countries could more effectively spend their mapping hours on things that actually aren’t available in public sources.
Answering just that one thing, there is no single entity called “the project”. There is no grand plan overseen by one person or group. What there are are lots of individual people (usually working together in informal groups) that do everything in OpenStreetMap.
When you say “the project has not been able or not cared to” you mean “I, and lots of other people like me, have not been able or not cared to”. This is exactly what lyx said yesterday, and he also suggested the way forward: “check out https://wiki.openstreetmap.org/wiki/Import and the pages linked from there to find out how to proceed”.
If you want to do that I’d suggest that you start with a small, concrete proposal of what you want to do and run with that through the process to completion.
I’d stay clear of suggestions like ‘A thing that I think would be nice is if the OSM database could have some layer support built in so one could make a top level “raw import” centralized for the whole country into an invisible layer (made by some special OSM user with some super-user privileges that can make huge imports)’, as that (a) would require a significant amount of work (b) is perfectly doable now with third-party tools and (c) would need a large number of people to be persuaded that this approach was the right one for relatively little or no initial gain.
(apologies to both Margaret Thatcher and Otto von Bismarck for stealing from them in that reply)
The broadly defined people who influence OSM community, myself and you included (top forum posters, local OSM community leaders, WG members, OSMF board, software maintainers) have not done a good job of explaining that to the newcomers. We suck at relaying our culture, and it dawned on me recently after having read so many questions that indicate lack of knowledge of certain core OSM values - not even in "first day" mappers, but in people who are committed to contributing to OSM and stuck around for longer.
I will elaborate on this issue in the near future in a diary post or something. While I will not run for the next OSMF board seat, I'd be curious to see stance of the candidates, maybe even nudge them towards recognizing the issue. Culture is the name of the game.
While I’ve been a casual mapper for several years, I’m just starting to learn about the culture and organization (or lack thereof). I have never been involved in OSM as a community until now in this forum. Just now started to use JOSM. Previously I’ve just mapped myself in the web editor mostly related to my various outdoor activities. I use routing tools that use OSM, and the poor quality of OSM here locally has forced me to fix the map before I can route. So I guess I’m not the typical mapping enthusiast. As a programmer I’ve contributed a lot to various open source projects though, but OSM I’ve done strictly casual mapping so far.
I just listened to Allan Mustard (OSMF chairman) talk from state of the map, and I see that there is a conflict between “traditionalists” and “adapters”. My ideas above are in the “adapters” category so I suppose it’s controversial. But anyway, I’m not that interested in politics, I just want a map that works and is competitive with what’s currently available from other easily accessible sources, and I see that in Sweden there is no real community, there’s just too few of us.
My country is sparsely populated, and we are used having high quality maps available, the Swedish online services already have that, as they generally use government-provided maps, not OSM. I’ve tried to convince others to join me OSM-mapping, but I often get the view that OSM is obsolete now when other maps are so easily available, and when I show OSM they immediately comment on how ugly and empty the map is. Which it is, as the OSM database in Sweden is not that good yet.
I just see that the old-school idealistic decentralized traditionalist way does not really scale to the size OSM is today and the situation we have today with other sources. I see that there is a major risk that OSM will move so slowly forward that other players like Google Maps will through AI and machine learning and other sources pass OSM quality in countries like Sweden if nothing is done, and simply render OSM obsolete for the majority of end users. I think OSM will need to strike some more collaboration with commercial interests so the platform and tools can keep up with the new situation. But that’s just what I think.
If it today is me like a single individual that needs to be the person that does it, okay, then I know. I may actually do it, I like the concept of freely available map with the open license OSM has so it’s a good thing to contribute to, and I think I have the right skill set to be able to get into this, but it will be a huge task which I probably can’t do this year or the next.
Continuing the off-topic even though I haven’t intended to:
I am talking about very basic stuff that both camps probably agree on.
a user makes edits, but doesn’t see them immediately on the default style on osm.org - concludes that there is some moderation approval process going on. In fact some people on FB groups told me this is why they were put off contributing.
something is not available as a preset in iD, hence it can’t be mapped. No knowledge about tag approval process or “use any tags you like”.
no idea that history in OSM data is preserved and nothing is lost except redactions
no idea of the escalation path in case of undesirable edits (changeset comments → local community discussion → DWG)
And so on.
I do not put myself in the camp of the idealists, for instance the apparently lauded imagico’s musings on his blog for me are often an utopia without a clear path forward. Neither do I side with HOT and Facebook, Grab and all the paid mappers who often do more harm than good due to lax QA where we should expect above average effort in case of paid mappers.
I’m the off-topic guy when it comes to forums… When the initial question has already been answered side tracks are fine and interesting I think :), and I have got answer for the initial question.
Over my years of casual mapping I did manage to get everything on the list you mentioned just by using the web editor, however not the the tag approval process.
In fact I have a bit of a problem with that now… I recently have tried to go past my casual mapping merely fixing alignment of roads and shorelines, and actually mapping all the landscape with all its natural features like wetlands, screes, forest, capes and bays, peninsulas, islets, hills etc, and as Sweden have names on pretty much everything I immediately got issues which I thought belonged to basic cartography but OSM still doesn’t support through its default style.
Like naming groups of things with a single name, or naming hills and slopes of varying sizes (text automatically scaled based on prominence) etc. Zooming out the map and most labels just disappear leaving a mostly empty map, as few are tied to an area. Bays and straits can be made as areas, and that works great, but on the ground the support is very limited by the default styles.
Sure I can tag it as I like and hope that sometime in the future some renderer will actually render out the names. But to me this also is a sign of limitations of the decentralized consensus model, which has left some, in my opinion, basic cartography features unimplemented for too long. I think there need to be some centralized cartography expertise that can oversee guidelines and feature set, and dare to implement and recommend features/tags before there is a consensus in this huge diverse community, because the larger it becomes the harder it is to make any decisions at all, and I think OSM shows signs of that. I don’t think there needs to be either anarchy or a dictator, there can be a balance, but I think some more active management would be helpful.
I think it’s too much to demand from the regular mapper to put in work into tagging things that won’t be rendered, and then try to get the tag approved and used in rendering sometime in the future. One have to be extremely committed power mapper which makes huge amount of contributions, otherwise you just get “it’s only 10000 of these objects in the world map, we see no purpose in rendering them”. I know there is the group tag for one thing I want rendered, but it isn’t rendered, so I just tag it in some other way that actually get some visibility on the maps used.
I know that OSM should be seen as a geodata database, not as a rendered map. But I think it is critical in order to engage mappers to have a default rendering that actually shows your work in a good way, which as far as it is possible is competitive with a high quality map made by a professional cartographer.
Is that not quite similar to what already happens for some imports? E.g for the ongoing import of cadastral building outlines in Spain: https://wiki.openstreetmap.org/wiki/Spanish_Cadastre/Buildings_Import/Import_guide. Individual mappers can use the Tasking Manager to download the potential import data for a chosen area, then follow the procedure in the Import Guide to merge with the existing database. I know this is not quite the same as your suggestion as the import data is not an “invisible layer” in OSM itself, but in practice how do you see your suggestion differing from this approach?
Yes maybe it is? Sounds great. I’d like it to be visible also to casual mappers using the web interface and maybe some more centralized and professionalised raw import, as here in Sweden there’s been available public map data in good quality since 2015, but still much of the OSM lacks land cover, and the limited imports that individuals still have made so far lacks in quality, it often looks like this (real example from Sweden):
due to various technical issues the importer did not overcome. It’s often technically difficult thing to do, maybe requiring custom software which few can do. So it seems like our little “community” here in Sweden hasn’t really been up to the task, and I don’t blame us, it’s not easy, it’s a lot of work for one person, and it’s scary to do such a big thing. Most of us don’t want to break things :). So I think at least here it would be better if a professional team (or some really skilled super-user assigned to the task) could work with imports and make them available with a click for volunteers via the standard tools like iD and JOSM, to review and manually merge bit by bit.
Not doing these imports at all, or by letting anyone do them which don’t really have the tools/skill/experience required to do it well resulting in bad and partial imports like we have here in Sweden, I think not only risk having a bad map for a long time, but also waste valuable mappers time which will work on either things we could import, or manually fixing bad imports, instead of actually adding new data that we can’t get from imports.
You’ve posted absolute screeds of stuff both here and on the tagging list, and I can’t fault your enthusiasm, but it’ll only get done if someone wants to do it - that’s how open source works. You clearly want to do it. No one else wants to do it enough to have done so yet. So why not?
OSM is iterative - you don’t have to get everything 100% complete first time. Maybe start with a small extract from the vaunted Swedish official data. Put it up for people to look at. Listen to their suggestions. Revise, improve, and so on.
