A Panoramax UK instance?

Has there been much discussion about setting up a UK Panoramax instance?

The main things needed seem to be funding, and someone to manage the server. Maybe OSM UK could do this?

Please use this thread to discuss it.

2 Likes

I contribute a fair bit* to Mapillary, so a couple of points from that perspective:

I know some people have objections to contributing to Mapillary on the basis that theyā€™re now owned by Meta (Facebook), but Iā€™ve always seen it as a pretty good symbiotic relationship - I get a good set of tools to help with my mapping, they get a ton of street-level imagery to play computer imaging/AI training withā€ , OpenStreetMap gets an imagery collection to plunder, and Meta picks up the bill for the storage and infrastructure costs.

Iā€™ve kept copies of a lot of what Iā€™ve uploaded (as opposed to the usual upload-and-delete), with the thought that I might want to contribute them elsewhere too at some point - so youā€™d be welcome to them to kickstart a collection.

My files currently total 38TBā€”, so my total contribution to Mapillary is probably 40~50TB. If a single contributor can generate that much content, do not underestimate quite how much the total storage might end up costing!

* 11.2 million images so far: Mapillary
ā€  If they use the imagery for teaching cars how to drive themselves, Iā€™ll be happy that tiny UK country lanes are well represented.
ā€” Iā€™ve 5Mbps upload, so youā€™d have to work out a way of getting them if you want them.

10 Likes

wow, thatā€™s a lot of data, thanks for all your work, and the sobering reminder of the storage requirements :sweat_smile:

OSM-FR say they have around 100TB of storage for 20M pictures, including redundancy & backup. They recommend setting up a server somewhere instead of cloud due to cost, which is fair, but unfortunately thatā€™s not something I can do (if anyone can, thatā€™d be great).

From what I can find, a 20TB drive is around Ā£400+, and OSM-FR have dual 1070TIs for processing, so a UK instance needs some sort of financial backing.

Basically, it looks like we need:

  1. significant startup funding
  2. someone and somewhere to have servers

maybe there are some gov initiatives? if anyone has any ideas itā€™d be great because i have no idea

2 Likes

The other component to this is what is the benefit of offering this. What does it offer aside from not being Mapillary?

1 Like

Thatā€™s a fair point. I guess itā€™s more future-proofing than anything, what if Meta decides itā€™s not worth keeping any more? Or they start charging for API access? Itā€™d always be better to have this data maintained by a non-profit, and federated via the ā€˜geo-commonsā€™ as Panoramax calls it.

But you might be right, the benefits might not outweigh the costs to set it up yet.

2 Likes

Yeah. Iā€™m not against it, and if it isnā€™t worth it right now, it is worth revisiting should circumstances change.

Focussing on the positives though, as thereā€™s always a million reasons not to do something, benefits would be:

  • data ownership/security/control
  • there might be some potential contributors who do not want to use a service provisioned by meta that would now be able to contribute.
1 Like

Thanks for your efforts - Iā€™ve made good use of your uploads to Mapillary in my area!

4 Likes

An active community of developers who are out for new features?

1 Like

also:

  • Avoid an uncontrolled Single Point Of Failure. The list of free services that disappeared in the past or changed their policies is quite long.
  • A full open source software stack (not just a few components)

This is also why OSM-FR proposed the start this project.

excluding backups :wink:

When I proposed the project on the OSM-FR forum, I did a calculation to have an idea for one dense coverage all over France. With redundancy and backup, the total was around 1 PB.

Storage is really the #1 issue and thatā€™s why we chose to have a decentralized sharing. We chose to self-host our instance to control the costs and we use mostly second life hardware, the only exception are SSDs.

Most of our HDD are 8TB now (we started with 3TB ones), and we can get very good deals on this capacity (around or even less than 10ā‚¬/TB). They are not new, so we have redundancy (2 disks in each set of 6 disks, all managed using ZFS).

Our hardware investment for the server is around 3500ā‚¬ so far. Server hosting (colocation) is offered by a Paris area datacenter.

Do not hesitate if you have question, need for advices or help to setup an instance.

The core dev team is working only in english in the gitlab repo and documentations are all available in english.
Do not hesitate to show up on the geocommons forum, english posts are ok even if most of past post are in french.

10 Likes

FYI:

2 Likes

What about a monthly online session in english like the one we currently have in french ?

1 Like