That’s a good one since a certain billionaire seems to have killed the public availability of t.co links…
4 Likes
This was one of my first thoughts when reading this thread.
But is seems that this is not an issue, as Overpass turbo doesn’t find any such links: http://overpass-turbo.eu/s/1xcB.
5 Likes
Is this an ongoing map roulette task? I mean, does the data get refreshed anytime so that new shortened URLs get added to the challenge?
1 Like
It gets refreshed when challenge creator (disclaimer: me) clicks on Rebuild tasks button, which I plan on doing from time to time; feel free to ping me if you need it before I do it by myself.
Apart from that, there is not yet support for continuous / auto-refresh challenges in Maproulette, although that feature is being discussed (so feel free to add your support there!): Continuous Challenges · Issue #1910 · maproulette/maproulette3 · GitHub
8 Likes
Remember there are a lot of websites that do not open if you are not in their country. Big shops in USA, big and tiny businesses in UK, big shops and government sites in Russia - don’t even give you an answer and you are disconnected by timeout
4 Likes
I thought it would be interesting to run through my http->https bot’s logs and look for websites that are not just non-functional, but don’t even DNS records set up for them. So not just a case of a website being down when the bot queried it, but a website that doesn’t even have the most basic internet configuration in place.
I put together a MapRoulette Challenge from a partial run. I’ll update it next week when the run completes and it should be reasonably easy to keep this updated on a regular (quarterly?) basis if the challenge gets any traction.
4 Likes
So what were the results?
Even a rough percentage from a partial run would be interesting to know.
Cool! It’s showing me a furniture shop in Wales. I confirm the website is down. I search the name on the internet. Their Twitter was last active in 2019, but nothing about them closing. No news articles about it either. I check on Mapillary, no images of that street. Any ideas what else to try?
I could leave a note, but if this is done on a large scale, wouldn’t that effectively be the same as a bot creating the notes?
1 Like
I would have to guess the results are from a run that was 45-55% complete and that generated about 8800 websites that don’t have DNS entries. As the run is randomized, there is a decent chance you can extrapolate ~17,000 entries in the end.
Out of how many in total?
I was doing this for Munich a lot. I do not know how many notes i opened, but i guess several hundreds. And i am overwhelmed with joy of how many support i got through the notes.
With the arguments you are mentioning it is worth opening a note, so that somebody can check the shop locally. I would like to encourage you: Just open a note and put all that info you wrote here into the note text and maybe also add the node/way link of the object to the text. This way it is easy to check your old note if it got resolved meanwhile without paying attention to/closing the note.
Taginfo reports 3.1M instances of the website tag. So about 0.5% of them can’t resolve the address from DNS. Not sure what percentage are failing for other reasons (404s, 503s, reclaimed domain landing pages, etc.) If the challenge gets traction and we see a 10% drop in DNS resolution issues, I’ll commit to broadening the scope to other types of website failures.
1 Like
I’m willing, but past experience tells me the idea is unlikely to gain consensus with the decision makers. It’s not a fight I’m willing to get into, but you have my support and I’ll commit resources to implementing something if it is approved.
I found a similar situation for a cafe. Website doesn’t work (obviously), and I called the phone number listed on the feature to find it is disconnected. I looked at the other tags and one of them was disused:***, so I went ahead and removed the website, phone number and name of the place. And then I marked the task as “fixed” in MapRoulette.
1 Like
if you manually investigated situation but you are not fully sure what should be done, and there is strong indicator that place is gone…
then note creation is definitely fine, in my opinion (unless people processing notes in the given area are unhappy about volume of them and/or you already created many, many notes in a given area)
2 Likes
If you see that bunch of open notes in that area and most of them point out a problem with a POI not existing anymore and are not false positives, then it is really worth to check it locally. I mean, this could also be a motivation for checking things locally if it’s not just one note, but e.g. 5 in a small area.
3 Likes
Definitely, I convinced yesterday my friends to change return route from mountain a bit so I could verify, fix and close one of notes.
But it is not option for me for notes say in Argentina or Moscow (in case of Moscow I was asked to stop opening some types of notes, so I stopped doing this).
Wonder if a set of website:unreachable:* tags would be more palatable. It would be concise, searchable and easily removed when the status is determined. Some tags should probably remain on those valid websites that have found to have acess restrictions.
The resulting set of tags could describe any special requirements required to validate the link. It could be used by automated QA to reduce the number of false positives. All by skipping websites with special requirements it is unable to satisfy. Such as foreign sites only reachable from within that country.
In the MapRoulette challenge, I’m wondering why is there a rectangular area roughly around Germany without tasks?
Did you exclude the area in the link checker? Or is someone else already cleaning up the website tags?
1 Like
I was wondering the exact same thing! I had now attributed it to the “partial” run and wanted to wait until the challenge is updated with the full data.