There are a couple of people who are interested in creating a New Zealand Panoramax instance - so I’ve created this post to see if there are other people who are interested and might want to join in.
At this stage its all very early thinking. We would need to look at how we would legally be structured (our own organisation or sitting under another relevant org) as well as how we approach it technically - whether we can get support from related organisations, funding, donations etc.
I haven’t been massively involved in OSM so probably a bit out of the existing networks of people in NZ so if you want to loop me in .. cool.
The engineers behind panoramax say that maintaining a local instance involves “A few hours a week” of work. Is there anyone with linux/server-admin expertise willing to commit to that much work, potentially for decades into the future? If not, there’s a big risk that extremely valuable footage will be lost forever…
OSGeo Oceania is the ideal place for this, as it could be the legal entity behind the service and could manage the finances, whether it’s funded directly by OSGeo Oceania’s existing funds, through the Community Grants program Funding – OSGeo Oceania, or by taking donations or sponsorship earmarked specifically for this service.
Looking at the AU Mapillary leaderboard the top 100 uploaders have contributed 64M images.
In the 16 TB setup, this should be enough to store around 4 million images
That puts things into perspective, Bob is doing something like 500k images per month, so he alone could fill up an instance the size of Argentina’s in under a year.
I don’t think this should stop us though, it’s still a nice idea to get an instance up and running, but I can see that costs could increase quite quickly. I agree with @user10’s about the risks. Unless there’s long term funding commitment in place, I would be upfront to contributors to not rely on it retaining images as a long term archive.
Is there anyone with linux/server-admin expertise willing to commit to that much work, potentially for decades into the future?
I think these kinds of core infrastructure projects, should be managed by a team and backed by the OSGeo Oceania organisation, rather than any single persons pet project, and it helps to de-risk and provides a path for long term continuity.
It could have a working group or special interest group if those are a thing (they are listed here, Oceania - OSGeo , but don’t seem to really exist? OSGeo Oceania could do with better documentation, minutes of recent meetings are not posted and as someone on the outside OO doesn’t seen to do much other than grants, but that is a separate topic), or something else beneath OO. I think it shouldn’t be another org and funded from OO as if the maintainer abandons it OO will not be able to take over and send a new person in.
I have been thinking about how we would sustain an instance and I think it would need a degree of industry support. For that reason I thought it is important to reach out to potential users of imagery such as local authorities, utility operators, logistics / transport operators and academics. My first thought was to pick up the phone to my local council and electricity network operator.
Oceania - OSGeo did seem a logical place to sit under although given that its an Australian Company Limited by Guarantee - I did wonder whether they would need a NZ subsidiary for operations in NZ. I also wonder what position this puts them in regards donations and tax benefits for contributors - I guess that is something that could be solved by me just reaching out and finding out from them.
I think that ultimately given the volume of data there is going to need to be some distributed storage/hosting of some type. I think a big part of the work may be adopting / adapting work from the global level on building a community of contributors and users and the framework through which they interact.
Oceania - OSGeo has the meeting minutes up to March 2026 + all the annual reports.
Yes. @catgirlseraid who posted earlier is on the board too. NZ organisations have sponsored OSGeo Oceania’s conference, as far as I’m aware I don’t think OSGeo Oceania being registered in Australia has been an issue.
Will probably have to do some major thinking on that @ethan. There are some major logistical hurdles (say) second uploading the 24 million images I have already put on Mapillary. The daily load is actually quite staggering too, perhaps 15-40Gbytes per drive day. All done via a $$$ cellular network. Mapillary also has the “nice feature” of using Facebook servers for storage, so it’s quite fast to use. I however only use that rarely for my OSM work, instead poll through my pre-uploads in a FIFO manner. The exceptions mainly about the use of roadsign recognition.
I am however not averse to my images/movies being used for Panoramax. I keep roughly 20TBytes of the capture data with me.
I recently discovered the JXL format as an effective way to reduce photo sizes, my 4.3MB JPG would compress to 3.4MB HEIC, which would compress to 1.4MB JXL all for no perceivable quality difference.
I don’t know what Panoramax does these days, but I wonder if they encode sequences using a video encoding format, as I would expect that could provide better compression given the nature of street level imagery photo sequences.
Given the price increases in hard drives over the last 6 months, it’s a difficult time for anyone to be starting a new Panoramax instance. The cloud providers like Hertzner can only hold on so long before they’ll need to pass on the costs too.
Self hosting is should bring the costs down further, at scale.
Yes even AVIF is a worthwhile conversion. Compressing to a smaller size however also incurs an extra DC energy (and thus time) factor. I already process for 3-4 hours after a similar length drive. (2x i7 laptops and a 300AH battery) Better formats recently came up for discussion on the Mapillary forum, but at the moment AFAIK they only take public/proprietary movie formats and jpg. I remember experimenting with ffmpeg conversion of (mp4) dashcam vids a few years ago where H.264 files unpacked in far less time than H.265 ones for the same resolution. ie the same basic problem. It is actually worthwhile using less video compression during recording IME to reduce artifacts on the extracted jpgs.
Also worth checking whether serving higher compressed images will work smoothly when rolling through a sequence. The decompress/CPU time on the end users device being the concern. Many mobile phones for example cant sustain high CPU usage without overheating.
Whatever the case if anyone wants to try a Panoramax installation I am happy to snailmail some live samples with EXIF geo data. Talking about a full 16GB or 32GB USB stick thing.
I think there are numerous challenges we will need to overcome but some of those should be at the global level (file formats etc) while others will be more local.
Given that I don’t see immediate and immediate huge demand in terms of request traffic for this data - the idea had been whether could we start by self hosting something?
I probably need to start mapping out some of that actions that would be required to get an/instances in place..
