Bot, detecting invalid websites

From time to time I’m stumbling across invalid website tags. Mostly it is not a matter of misspelling - no, just the formerly valid website changed/disappeared and became invalid.

Wouldn’t it be a good idea to have a bot detecting no longer existing websites?

2 Likes

Osmose has a similar check already, if I remember correctly.

2 Likes

If I created link correctly you can see reports at Osmose (and you can switch off some groups)

Useful fixing is welcome (though not every edit removing Osmose warning is good, some are outright harmful, remember to use your brain)

1 Like

Just an example: The last invalid website I came across was added years ago as
website=https://www.ewe-go.de/privatkunden/elektromobilitaet-unterwegs/

and leads now to

This is the kind of invalid website I would like to have an automated lookup tool for. A possible action after having detected an invalid website address could be: send PN to original editor of that entry and to the last editor of that object. (just a rough first thought).

I followed your link to Osmose, but could not find find any opportunity to select “website” or similar keyword on the left screen list. Could you pls give a further advice?

There is strubbl/osm-link-validator: Download OSM objects of an area via Overpass and validate their links - Codeberg.org but currently only configured for selected regions

search for “URL”

1 Like

is its object detected by Osmose?

No, is not. There is just another Osmose message that the country code for the free 800… service phone number is missing.

Searching for unlimited pops up URLs those that do not conform to the valid URL schema.

I have in mind to let detect URLs that lead to a dead end, where the original website no longer exists.

A thing that should be considered in this occasion, is to not fix “http” to “https”, because even though nowadays browsers force it to https, there are some websites that continue serving in HTTP instead of HTTPS, with the HTTPS version either being a 404 error or serving a different content than the HTTP one (it’s rare, but I’ve seen such cases).

1 Like

then I guess detection request can be added at GitHub - osmose-qa/osmose-backend: Part of osmose that runs the analysis, and send the results to the frontend. · GitHub

2 Likes

That would be great - unfortunately, that’s beyond my capabilities.