Results of the OpenStreetMap Survey

Foundation
, , , , , ,
1

Results of the OpenStreetMap Survey

Between June and July, we conducted a survey that received 461 responses from people across 69 countries.

See the results [here]

My name is Paola, and I am a Design student at the University of Brasília (UnB). I am part of a group that, along with students from the Federal University of Espírito Santo (UFES), is working to make OpenStreetMap (OSM) more accessible and intuitive. Our goal is to apply user-centered design principles to facilitate contributions from everyone, creating a more inclusive and efficient experience for the global mapping community.

The survey was designed to understand the needs and challenges faced by OSM users. We started with a round of interviews with Brazilian mappers to identify their profiles and main pain points. Then, we developed a questionnaire to validate the insights we had gathered. We would like to express our gratitude to everyone who took the time to respond to the survey. Your participation was crucial to advancing this project.

Key Findings

Some of the most relevant findings from the survey were:

  • User profiles: Most respondents have a high level of education, are fluent in English, and have prior experience with Geographic Information Systems (GIS).
  • Use of OSM.org: Many users access the platform to view data, but few utilize the notes feature or resolve them regularly. This indicates potential for improving this functionality to make it more practical and user-friendly.
  • Search accuracy: About 18.6% of respondents found search results to be inaccurate. During usability tests, we observed that finding information can be challenging without the exact term or precise location. Furthermore, 61% of participants supported integrating external data to enhance the search experience.

We acknowledge that we could not cover all OSM functionalities, such as map editing, due to the limitations of MS Forms used in this initial survey. Nevertheless, this step was fundamental to better understanding the community’s needs and outlining paths for future improvements.

Next Steps

Improving the platform requires everyone’s collaboration. Therefore, we invite more users to participate in this process by sharing ideas, feedback, and experiences. If you’d like to discuss the research further or share ideas for the future of OSM, Gustavo will represent us at FOSS4G. We would be delighted to hear from you!

The data from this survey is available in our [GitHub repository] for anyone interested in accessing and exploring the information. Together, we can build a better OpenStreetMap!

13 Likes
2

Results of the OpenStreetMap Survey

Between June and July, we conducted a survey that received 461 responses from people across 69 countries.

See the results [here]

thank you for this, it is really interesting. Just a short question, is it correct that all respondents have at least basic knowledge of English, nobody responded who does not speak English?

1 Like
3

In Openstreetmap as a community we’ve been moving communication and socials to specific platforms and while that makes sense (don’t reinvent the wheel) the osm-specific stuff is definitly lacking.

Notes is basically a way for mappers to contact each other in a geo-way that only osm can provide. But we’re hardly taking advantage of this because all the other tools we use are NOT geo-aware. From this forum to the wiki.

What is really missing is closing that bridge between social and geo. A note is a good example, but better examples are things like the comments on changesets. Both should be a (semi-public) conversation that goes on, and both have that geo feeling to them.

So, we’re missing the notifications that actually open the conversation. How many people didn’t notice they had a comment on their changeset until much later? If ever. It is likely much worse for people that don’t hang out on this forum as we know a lot of people use an email address to sign up but they don’t use that email address.

A geo-enhanced conversation allows users to realize they are not just mapping in a vacuum. There are others nearby that can help them, and that is actually something a lot of people will appreciate.
Geo enhanced means that if you get a response on your changeset from a guy that maps less than 50km from you, you’ll give that a lot more credit than someone that comes from a different continent.

Notes, though. That’s what the survey highlighted. A note is currently just a geo-locked text.
What would be awesome is if you can mark a note as “read”, meaning you don’t have to see it again and again and again as you check or nearby notes.
Being able to quickly follow up makes a lot of sense too.

And notes would also be good have multiple types. For instance in streetcomplete you may get a screen that allows you to make a note when something doesn’t really fit in the categories offered. The application makes it feel like it will just be a mention for future mappers. NOT a task.
But then a local mapper can come by and just mark it as “done” because the task concept is what people think notes are.

Design students may be nice for the devs on the website to talk to about how we can improve some of the simplistic features we have to make them become much more useful.

2 Likes
4

Is that surprising? The survey was in English. People without basic knowledge will simply not participate :wink:

4 Likes
5

And the number of respondents is not statistically representative… so… a preliminary research, not a significative result.
Needs to be improved to be considered relevant.

6

As far as I remember the survey was available in many languages. But still, a basic understand of English is kinda necessary for OSM.

What would be the sample size to be considered statistically representative?

1 Like
7

This research was proposed initially to infer about Brazilian mappers. So, it was divulgated to a large audience (in the WeeklyOSM, e.g.).

If it was presented in (or also in) Brazilian Portuguese, could help to obtain more respondents.

To calculate the significance you would to study Statistics… the previous comment shows that it was not considered by the researchers/students that have lead this survey…

To do a fast calculation (there are applications that do this calculus in the Web for you), with a confidence interval of 95%…

if you have 500 to the Universe (= people in OSM Brazil Telegram group), you have to interview 218 users.

If we consider the daily active members for Brazil (almost 150 in the last week, data from OSMSTATS, Pascal Neis, checked today), you have to interview 108 users.

If you consider 80, that was the lowest level of daily active members for this week, you have to interview 67 user.

The results shows 46 respondents to Brazil…it is still very low, facing our reality.

And there are more people than what is in the Telegram, for example…

Another error is to extrapolate results obtained practically by a local reasearch (see the very low participation of the most of the countries) to the Universe (global OSM, in this case).

The idea of the research is very interesting, but it needs more scientific treatment. What you see is a poor result for most of the considered countries.

If you want, it can indicates the outreach of WeeklyOSM+Commumity OSM+some local Brazilian groups (the means of communication of this survey). But not an overview of the OSM users mean profile. Keep walking.

2 Likes
8

I won’t comment your numbers, because, well… Please use the same tool of yours and run with 8 billion people (entire Earth’s population), same 95% confidence level and 5% error, just to see something.

Just for comparison, the OSM Foundation run some surveys from time to time, and the most complete one is from 2021: 2021 Survey Results - OpenStreetMap Foundation

They (OSMF) hoped to get 1100 responses and managed to get 2958 full responses, with all the advertisement they did. I think a bunch of undergraduate students (no disrespect here!), with no funding or external support, managed a pretty amazing feat, also displaying the results in a more attractive and easy-to-read manner.

1 Like
9

The relationship between population size and sample size is not linear.

Whether or not a lot of effort was put into the project does not impact the validity of the statistical results. So, congrats to the students for a great effort, but the results might not reflect the population.

10

That is exactly my point. Of course it involves a lot of assumptions (normal distribution, random sample etc), but I don’t agree that a n=461 is small nor statically insignificant.

The sample could be bigger? Of course it could. Can we throw the study out the window? I don’t think so. The study has it merits, and we should discuss the results, not the statistics of the thing.

11

In my understanding the post here is trying to make conclusions, which I don’t think are correct. The conclusions that most OSM-users are fluent in English and having experience with GIS and having a higher educational background is highly questionable to me. At least is OSM-users are visitors of OSM.org.

But of course, those results are understandable, considering the circumstances of that survey. You just need to keep in mind while interpreting the results.

Especially if the question regarding English did not came with a option “I don’t speak English”. That’s like giving the option 1,2,3 and 4 and concluding, no one replied 0 :wink:

n=53 are definitely not a significant amount of OSM-users in Germany.

12

As Matheus responded, the survey was translated into many other languages, including Simplified Chinese, and we believed that basic English adequately covered the portion of respondents who do not speak English fluently but know some words. Nevertheless, we will take this into consideration for the next time we conduct the survey.

13

Thank you,
You brought up some great points!

1 Like
14

This is not an official OSMF survey, and we did not receive any support from the organization, nor access to usage data from OSM.org, despite our request. In fact, we could have chosen not to publish the results as we did, but we decided to share our findings openly. Regarding the number of participants in a quantitative design survey, it is worth noting that the number of respondents often has little relevance for achieving meaningful insights in this field.

Perhaps you are not very familiar with design research methodologies. Yes, we started with interviews among Brazilian mappers because it was the most accessible starting point for our group. From these conversations, we crafted the survey questions. If we analyze the profile of those who actively participate in the community, it is evident that this group typically has higher levels of education and income—something we also observe within the Brazilian community.

In any case, we appreciate your interest. We will certainly take this into account if we decide to conduct another survey in the future.

5 Likes
15

It is a bit disheartnening to see practically all comments to critique the number of respondents and nobody talking about how useful the questions were or how interesting the answers.

sampling-bias
sampling-bias1200×1017 146 KB

Yes, even with a sampling bias for the language, you can read the answers that are not related to THAT question and get relevant results out of them that at minimum may be used to aim for further research.

3 Likes
16

I personally like that someone tackled this and is openly sharing their experience. Some assumed stuff, for some the first response seem quite harsh in my opinion, without much suggestions on how to improve.

There is already interesting stuff in there, maybe next time OSMF has time to respond and help and this can be done better and on a bigger scale. It would really be good to see some stuff and share this to various local chapters and organisations.

Thanks @PaolaCardoso and @Gustavo22Soares for sharing your work! :slight_smile:

3 Likes
17

Some people just want to complain instead of doing something, and this is so annoying that it drives amazing people away from the community.

18

Yes, we could have presented the complete redesign with the final solution already in place. However, we chose to pursue an open process because we believe it is fair to allow the community to participate and contribute.

We understand that this data is very important not only for our work but also for the entire community, as you mentioned, even for local chapters. We want to do it again this year, and perhaps with the support of the OSMF, we can gather more responses.

1 Like
19

It is important to note that we are design students, working as volunteers, and we do not receive any funding for this project. Therefore, we have the freedom to simply discontinue it and redirect our efforts to other projects if we feel that this one is not progressing constructively.

20

I appreciate the research and I do my comments in the direction of improve it. And I know you are students and will want to publish it as scientific research result. So, pay attention for statistics.
You could be opened to the critics, as you are in University. It is very natural. Unfortunately, some people does not like be criticized.

1 Like