Following a discussion about a possible quest on StreetComplete to complete cables=* key for power=line, fears rose about errors and mistakes it could generates.
The point is to count the bundles of electrical conductors that compose each power transmission segments (so only big lines).
We found a method to try to assess how high the risk is by setting up a poll:
Even imperfect, this would be a valuable and accessible effort to improve questions that should be asked with what options to answers in the future.
It simulates questions that could be part of StreetComplete against 4 situations that could be found on ground.
We would like to encourage anyone to try to answer to this poll, whatever their experience on the matter can be.
Answers will certainly spot situations in which questions, documentation or illustrations arenât that good.
Apart from StreetComplete, this is also a great opportunity to get a feedback about mapping practices that isnât so often requested.
I find the design and structure of this poll confusing. A better approach might be to present a single image at a time and ask, âHow many cables do you see in this image?â Or, to present an initial "For the purposes of this survey, here is how we define a âcableâ and here are some correctly-annotated examples. Now, for each of these following images, answer, âWhat is the correct value of cables=*?â
StreetComplete users are not supposed to have read the wiki, so a good phrasing of the question is crucial (as well as good translations in all the languages that SC is available in). In your introduction you mention cable bundles, while in the poll title the number of âelectrical conductorsâ is asked for, and in each question âthe number of electrical cablesâ, without being clear what that is. Itâs also confusing that on each page of the questionnaire, there is one picture more than the number of questions. It seems the question is about the first bigger picture and the others are answer option?
If the intention of the poll was to gain understanding of what average StreetComplete user would answer (i.e. to determine if quest would be useful for SC), then @InfosReseaux absolutely should NOT have provided any extra explanation nor linked to the wiki, but only shown the question that would be shown by StreetComplete, e.g. âHow many electrical conductors are seen on this line?â (or whatever question you want to be used as SC Question) without any extra instructions how to answer it besides just reassuring them that there are no right or wrong answers (to reduce the chance of participants looking up correct answers anyway)[1]
In fact, OP shouldâve explicitly warned the users to answer the questionnaire without looking in the wiki or trying to educate themselves on the subject first, and to please skip answering the questionnaire if theyâre already familiar with how to map power infrastructure in OSM (to reduce bias of forum reader which are often much more OSM-tagging-involved than average SC-user).
With such information provided to participants beforehand on how to correctly answer (something StreetComplete users would not be provided with), not only is the survey made completely unusable for determining what average StreetComplete user would answer; but even worse, anybody who read those instructions or visited the wiki is now ineligible to be unbiased participant in the potential future questionnaire too.
Please try to keep with scientific method to avoid introducing (especially such extremely heavy!) biases in the future, if you want the results to be actually useful.
the reassurement was done, but it does not help at all since extra information was already provided and poisoned poll results âŠď¸
Also, the questionnaire seems to use leading questions which are extremely problematic in themselves for this purpose.
If the purpose was to see what StreetComplete users would answer, then it should have the same format as StreetComplete would have â i.e. showing a picture, displaying the quest text âHow many electrical conductors are seen on this line?â and asking the user to input an integer number. Nothing more.
Asking âdo you see 4 cables here yes/no?â (and with 4 extra bold lines added for each cable on the picture) is really not helpful for assessing suitability for StreetComplete, to the contraryâŚ
As that is actually only answering the question âis your eyesight good enough to see 4 extra bold lines in this picture that Iâve madeâ and says nothing about âwould user be able to correctly count the cables from this pictureâ.
But then again, if youâre no longer targeting StreetComplete, but instead want to make graphical introduction for app like (or fork of?) MapSwipe, then such pictures are really good for such introduction.
MapSwipe quests consist of three parts:
first they have a short tutorial on how to determine some data (which pictures found in this questionnaire with emphasized cables would be a really good fit for!)
then there is a short test of several examples to see if the user have understood how to tag it correctly (provided a feedback why some answer is incorrect if users fails to answer correctly)
and only then provides the users with bunch of real quests.
(but, for MapSwipe and similar apps to work, one would need to collect power towers imagery separately, e.g. via Panoramax or something)
Itâs exactly how it is built. On each page, you have got an image with a question and then 4 possible answers, with annotated illustrations. The header has been updated to state this clearly.
I agree with this and the experiment is here to refine that. So itâs not necessary to blame the experiment directly since itâs a step in the whole process.
I will try to make the appropriate clarifications today in the questions, answers and header as well.
Thatâs not the intention, the point is to know if such an image on its own would allow more accurate answers.
We will consider also MapSwipe, thank you for the mention of it.
Just a few insights about pollâs 22 answers so far, thanks to anyone involved.
Good answers are selected more than 70% of answers. 45% of answers are from people that had never contributed on cables=* key.
Even biased by the question or the wiki links, these results still convince me about the relevancy of using annotated images to propose possible answers. Itâs still possible to improve both questions and images to find the most relevant option.
By the way @Matija_Nalis, why isnât this possible to provide SC quest with 10 lines of text explaining what the quest is about? Itâs not about including the whole wiki page, but simple information to state how to count (or how to determine the road surface, or anything else). It doesnât prevent to use clear questions and options, but thatâs also a relevant way to enable users to find answers.
