Integrating external data (e.g. wikidata) in OSM Wiki

General talk
1

Hi, I’m wondering about the possibility of integrating external data into the OSM wiki, for example from Wikidata, but also from other external sources.

Here’s the context : as part of the “Oh my grid” project , we want to enhance the mapping of high-voltage power grids on OpenStreetMap. We’re doing this directly, but we also aims to produce tools and information so that local communities can do it too.

One of these tools is the creation of country dashboard, to provide an overview of power networks in each country. The aim is to bring together: statistics on OSM data (taginfo, graphs, history, for example from OpenInfraMap, see link below), contextual information on the country (surface area, population, GDP, which can come from wikidata, or elsewhere), and additional unstructured contributions from complementary investigations.

A first attempt at a country dashboard can be seen here: Power networks/Tanzania - OpenStreetMap Wiki

We want to facilitate the circulation of information, and reduce the need for manual maintenance. For the sake of sustainability, we’d like to be able to create these country dashboard directly on the wiki. We’re going to create useful information for osm mappers, we would like this information remains even after the end of the project. We would like to avoid self-hosted solutions that may not last.

But then, there is the problem of integrating external data into the wiki, which isn’t trivial, also for legitimate security reasons. But isn’t it worth thinking about? We could start with using Wikidata data (I tried to use wikibase for it, without success since osm wiki has its own structure that replace wikidata in the wikibase). We’re interested in this for electrical networks, but the concept can certainly be applied to other thematics, which is why it’s worth thinking about more generic tools.

I’ve experimented various solutions on this page (which is kind of a sandbox at the moment) :
https://wiki.openstreetmap.org/wiki/Module:PowerNetworkCountrySheet
I haven’t found any way to make external calls. The best solution I’ve found so far is to hard-code the data (see top of module), which is pretty easy to update if the data is properly structured. But that doesn’t solve the problem of making graphics.

Do you have any thoughts? Is it worth persisting with this strange idea of integrating external data into the wiki? Are there any strategies I could use?

Thanks for your answers.

1 Like
2

I have no right to make more than 3 links in my message, so here is the 4th one : Tanzania - Open Infrastructure Map

This is a nice example of a country dashboard. We could also focus on improving it, but as I told: Personal initiatives, however good they may be, may not last.

1 Like
3

This would normally require the wiki’s system administrators to install the WikibaseClient extension, but that extension is already installed in order for articles to pull in content from the wiki’s local Wikibase instance, where the data items live.

I guess with this approach you’d be interested in federated properties, which are experimental and currently disabled.

How essential is it to have these tables on the OSM Wiki? Both Sophox and QLever can create live reports that join the OSM and Wikidata databases. Then you wouldn’t need to do any maintenance as coverage grows. If you need something more presentable than a raw list of results, a static site hosted on GitHub Pages or Codeberg Pages can call the Sophox or QLever API to build the tables dynamically.

4

Thank you for your answer. I didn’t know Sophox and QLever. I will have a deeper look into it. But reading quickly their description, they don’t allow for direct data integration in the wiki, isn’t it ?

Of course, there is no vital need. Our basic idea is that osm wiki will be more long-term maintened than our custom solutions, and it is more common reference. So we think it is perferably to add information there, if possible. But Github page could be indeed also a good compromise.

5

Not directly, but you can link to a query or its live results.

1 Like