At the moment we are not sure where the imagery will be processed and what will be used for the final hosting. The OpenStreetMap Foundation do have a Dev server, but this “only” has 14TB of available storage.
CD:NGI normally provide the imagery for free. To collect the imagery one provides the “sales” desk @ NGI in Cape Town with an external hard disk and they copy across the imagery over a few days. CD:NGI have the option of charging time and materials, but this has been wavered for us in the past.
We at SAEON have some plans for the WC 25cm imagery that will involve a similar workflow (cogs and titiler). We have an existing server that accepts http range requests we use to host and share cogs. I also have some project funds that could cover the costs of adding additional disk space to the server.
This type of service is probably within the scope of our mandate (we are publicly funded), so I should be able to get this approved.
I would probably need help fetching the data from NGI. It would be a schlep to do it all myself.
Otherwise B or C would work. B might be smarter as we can do the compute on the same rack as the storage.
I would also consider creating a stac catalog of all the cogs. Having both titiler and a stac api that points to the cogs would be the best situation to make the data widely accessible. We could perhaps also host these services.
Let me send some emails to our infrastructure team. If they are amenable we can talk next steps.
The infrastructure team at SAEON is happy with this and we have the disk space available, it is just a matter of provisioning it. We can also run the HTTP server to share the cogs and titiler service. I can work on creating a stac catalog and eventually get a stac api running too.
We’ve not yet been able to collect a copy of the source imagery files from NGI. Logistics of people taking time off work to get there.
The source imagery will then likely be uploaded to OpenStreetMap dev server. Estimate is likely 2 or 3 weeks to upload the imagery, but maybe copies could be sneakernet shared with others in Cape Town to allow faster parallel upload (bittorrent?).
This is great. I would be happy to be part of the sneakernet. I would also like to play with optimising the creation of the COGs.
Once we get around to uploading the COGs to SAEON, we will give you access to the host machine and you will need to connect with our VPN. Rsync will work, not sure about BitTorrent
Let know when the data gathering will commence, and which tiles I should grab. You can reach me at:
glenn at saeon dot ac dot za
too if needed