Hello everyone,
I am planning to build an OpenStreetMap (OSM) tile server to serve vector tiles for an iOS/Android application I’m developing. My hardware consists of an HP Workstation with an Intel i7-8700 and 64GB of RAM, but I need advice regarding the storage solution.
For a full-planet OSM server, is a 1TB NVMe sufficient to start with, or should I begin with a larger drive? Additionally, can I add a second NVMe after a few months, or is it better to install a 2TB NVMe from the beginning?
From a non-technical standpoint: if there is a possibility that you are going to need 2TB in the next year or two, buy it now. The AI companies are driving prices for everything up, especially GPUs, memory, and flash storage.
You would definitely need 2 TB if you wanted to import a full planet into a PostGIS database, but since you say vector tile server, what are your intentions? Planetiler or Tilemaker should be fine with a smaller disk (though YMMV about the RAM, 64 GB might be shaving it awfully close).
If you plan to have a Linux server, you can use LVM (logical volume management) and spread a partition file sytem over several disks. You can enlarge this, shrink, … at any time after.
Doesn’t probably work with a single 2 TB file.
@woodpeck I’m making this outdoor navigation / trail GPS recording app and I want to serve vector tiles to my app, for the tests in the last 2 months I used mapTiler as a tile server, but I use already more than 20k tiles, just for these tests, me and maybe 2 - 3 friends tested the app. The free tier is 100k requests, so I can’t use that for sure if I add the app in appStore/Play Store .. my plan is to offer it for free, I hate advertising, will never add popups/fullscreen/ small ads in my app (but maybe some advanced features would be locked under an small yearly fee if the user base will rise and I can’t serve the tiles anymore from my home 1gbit fiber line)
I used in the last few years wikiloc (paid, yearly) but I needed once to find a track that I recorded an year or two back, and unfortunately wikiloc doesn’t keep the tracks in phone, even if I recorded it myself using wikiloc. I needed that track in a location with no signal and I couldn’t find in my phone, I could oly see ±16 tracks, I recorded hundreds, I was so pissed off that I decided i will make myself an app 
Before wikiloc I used MotionX GPS, loved it , but it’s not available anymore
I already have an dedicated server in a datacenter, but this tile server will be hosted in my home. I will point an subdomain of my app domain to my home server (I have fixed IP , fiber line). The app’s domain is already functional and the Live Sharing function works great, I will host everything the app needs on the server that is hosted in a datacenter, only the tile server I want to host it at my home (I have PV and a few large batteries, so my house won’t be be offline, it wasn’t for years ).
@ToniE Yes, I will use Ubuntu, isn’t that the recommanded OS ?
I already have 1Tb NVME inside this workstation, so if I could just start the server with this it would be great (nvme prices gone crazy, I don’t really want to buy now 2Tb drives at these pumped prices).
So, will the PostgreSQL database be ever bigger than 1Tb? Is the whole database a single file .. there is no way to split the files on multiple drives, other than combining 2 drives in one logical volume ?
You can also use https://openfreemap.org/ preprocessed weekly data for self hosting. ( OpenMapTiles compatible schema )
‘We also provide weekly full planet downloads both in Btrfs and MBTiles formats.’ ( → files.txt )
System requirements
http-host: 300 GB disk space for hosting a single run. SSD is recommended, but not required.
tile-gen: 500 GB SDD and at least 64 GB ram
( openfreemap/docs/self_hosting.md at main · hyperknot/openfreemap · GitHub )
is this a free tile server I haven’t find out yet ?
you do serve vector tiles so I can style them as I want ? In my APP I use MapLibre GL for rendering the maps, I also made an offline map section so users can download an area for offline use (not just yet ready), are there any limits or should I make the app limit the tile requests/second when downloading the tiles for offline use ?
@ImreSamu I’m checking the website just now .. I’ll be back with more questions. thank you
I’m just a fan and user,
so if you have a question, please ask it in the GitHub issues.
By the way, it might be worth thinking through what happens if your app suddenly becomes extremely popular. 
See: OpenFreeMap survived 100,000 requests per second
(I know you’re going to have a go with OpenFreeMap, and that looks like it’ll save you a whole bunch of setup time, but to answer the original question in case that’s relevant to anyone else)
The actual sizes of anything will hugely depend on the detail of the maps that you’re displaying as well as their global coverage. With a database for a raster tile server, it’s fairly straightforward - if you’ve got more things in the database it’ll be bigger, and if you’re creating higher resolution tiles it’ll also be bigger.
For vector tiles “what’s in each tile” is still configurable and may be larger or smaller depending what you choose to put in there and how detailed you want to make it. Some iOS/Android apps might need a fairly stylised and not very detailed map background and might need much less disk space (and be quicker and easier to create, too).
I’m also planing to use Cloudflare between my server and the users of the app, hopefully that will work (I have some doubts regarding cloudflare behaviour I seen on some websites where an “cloudflare I’m not a robot screen” was shown before reaching the actual website, I never liked that behaviour).
Indeed it seems a very good option, I have read just a few minutes details about openfreemap and it seems it’s not a tile server, I still have to serve the tiles to my app, using my server (if I’m wrong, please correct me) but it provides the precompiled vector tiles (I didn’t knew the info in PostgreSQL database needs to be compiled in a vector tile)
In my mind I believed the vector tiles contain ALL the informations available in the OSM database for that specific tile, and by applying the style in users phone using the GPU I only display the informations I want, and yes I want to display lot’s of informations. This was another reason I want to ditch the app I already pay yearly, Wikiloc, the Offline maps are so ugly, and miss some very important informations like all springs (places where you can drink fresh water while away in the wild).
I’m working on my app in my free time, I will try to compile my app using the style from openfreemap to see what happens.
“We also provide weekly full planet downloads both in Btrfs and MBTiles formats.“ this is not familiar to me, as I never used, owned an OSM tile server , I think MapLibre in my app uses pbf tiles
I will have the workstation only in a day or two so only then I can actually test, if indeed openfreemaps is also a vector tile server, and I can host the style on my own website, I will use OFM for sure
anyone knows what style should I use to start with the most similar map tothe opentopomap, or is their style available so I start editing my own style from that style ? I like these colors for the “outdoor type” but these are raster tiles not vector, is there a similar json that uses similar colors to this raster map ?
Thank you
You need an extra data: contour-lines + hillshading
Links:
Related: Outdoors style for vector tiles?
Yes, those are the must-have features in my app. It seems my storage needs are going higher and higher
did anyone combine in one json style the vector tiles from openfreemap and hillshade (terrain RGB) from mapterhorn? Is this doable? Both free with no restrictions as I understood (am I right)?
The code bellow is from MapTiler json, they use an separate layers for contours, as maplibre-contour only works on web, not also for an iOS/Android app, I must also provide that layer too, my question now: is there any free server to serve contour tiles, like openfreemap serve vector tile, mapterhorn serving the hillshade?
or
I should host myself the contour tiles ? (is there a free source where to download countour vectors, or I should use a tool like GitHub - roelderickx/contour-osm: Export contour lines from a file or database source to an osm file to generate the contour from mapterhorn files ?
,"sources":{
"contours":{"url":"https://api.maptiler.com/tiles/contours-v2/tiles.json?key=","type":"vector"},
"landform":{"url":"https://api.maptiler.com/tiles/landform/tiles.json?key=","type":"vector"},
"maptiler_planet_v4":{"url":"https://api.maptiler.com/tiles/v4/tiles.json?key=","type":"vector"},
"outdoor":{"url":"https://api.maptiler.com/tiles/outdoor/tiles.json?key=","type":"vector"},
"terrain-rgb":{"url":"https://api.maptiler.com/tiles/terrain-rgb-v2/tiles.json?key=","type":"raster-dem"}}
If"contours" and "terrain-rgb" are the only missing layers, I will try to create an style using openfreemap for vector tiles and mapterhorn for hillshade, will leave contours for an later update when things will be more clear to me.
Thank you for your support so far!
No, such tiles would be much too heavy. OSM data are filtered and organised in tiles based on a schema : OpenMapTiles, Shortbread, Protomaps Basemap Layers. So you need a style which is consistent with the tiles you use.
Based on my experience, serving vector tiles is not an issue. A full planet pmtiles files is 80 to 120 Gb (depending on the schema), and serving tiles from this file is so easy : PMTiles for MapLibre GL | Protomaps Docs.
The issue is tiles calculation : it requires a lot of ressources. Using planetiler or tilemaker for calculation, you don’t need to import full OSM in a postGIS DB, they can read a planet.osm.pbf export (85 Gb), so disk space is almost not an issue either. But these soft use a lot of memory and cpu, especially to treat planet size exports.
Thankfully, it is possible to find tile files calculated by others:
(I wrote this page on the wiki with some more details but it is in French)
If you like OpenTopoMap, The best thing to do is wait until @derstefan has finished developing its vector version (WIP here).
You will have the same issue with OpenMapTiles, Protomaps or Shortbread vector tiles scheme. They were not designed to store all the elements or tags needed for an outdoor style.
With the help of OSM-France outdoor group, I’m developping an outdoor style for cartes.app. I should warn you that it’s a very long process (I might not have gone ahead with it if I had been warned 
). For each missing element, you have to identify relevant OSM tags, update the tile calculation script, re-run tiles calculation, update the style, add missing icons, … It is endless and very time consuming.
Thank you for your detailed response !
I have started to customize the style based on openfreemap Bright version but I already found 2 limitations, I think the small rivers (streams) are not available in the vector tiles, also the natural=spring are also not available.
Than I checked OpenMapTiles schema and they also do not provide these 2 elements, I haven’t search for other elements I need, as those 2 are already missing and at least the springs it’s a MUST-HAVE for an outdoor experience.
Because of the missing elements in openfreemap I probably need to host the tiles myself, today I will get my hands on the HP workstation I planned to use for this task (I know the i7-8700 is quite old .. but at least the 64Gb RAM will help, I could extend to 96GB as it has one more free ram socket)
I couldn’t find yet someone to offer the PMtiles that also include small streams and also natural=spring, so this means I will need to host them or compute them myself.
This means I will need the full osm database? (If anyone knows a source for pmtiles to include also these “outdoor type” features, let me know)
I’m not planing to regenerate tiles too often, I believe even once/month is good enough.
Does this planet.osm.pbf file contain the “outdoor features“ ? I haven’t seen any schema in that website.
I could also find https://download.geofabrik.de/ where there are individual files for each country. I think there should be a way to compute the tiles for each country individually, so my low resource pc (i7-8700/ 64GB RAM / 1TB nvme) can handle easier the computation needs of one country at a time.
I believe I could regenerate the tiles only for countries that are most used, not the whole word every week, or every month. Now while I type, I imagine exactly this, for the countries where I have most request I could regenerate the tiles weekly and for other countries where there are very few request the regeneration process can be postponed for a month or two .. or more.
I now try JOSM and the downloaded pbf file from geofabrick.de of my country, to see if I see the elements I need, as I don’t know yet if these file contains all elements or not.. but it seems JOSM is not to speedy in my macMini
Thank you again.
No, but it does mean that you may need to create your own schema and host your own tiles. As an example (if Tilemaker fitted your use case), you could start with the Shortbread Schema and example Tilemaker code and add in your new layers - it’s easy to do as you have examples to copy from. I’m sure you could do something similar with other schemas and toolchains.
As an example, here’s mine - it’s likely a bit heavy for what you want, but gives you an idea.
Anyway, it’s better to check with a small country, as @PlayGuide said, it’s a long process to add the right tags, the icons, update the style, check the result, and only when it’s good enough or as you understand the amount of new tags you’ll add, you see how much it would cost for the whole planet.
Yes, this is a full export of the OSM database, with all elements and tags at a given timepoint (no history but you don’t need it for the tiles)
Well, yes you can calculate tiles country per country, but on my experience it is quite complicated (or computer time consuming) to merge several tiles files in 1.
Another solution could be to use OpenFreeMap (or other tile source) for the basemap, and to calculate a tile file only for the missing elements. But I never tried this, so I can’t tell you whether it’s easy in practice or not. I guess you will first have to filter the planet.osm.pbf file for the relevant tags, otherwise the calculation will take just as long.
Do you know pmtiles.io? to check pmtiles files content.
Yes, we did the same for cartes.app starting from OpenMapTiles schema. Our 2 scripts for tilemaker are here.
You don’t need to do this - both tilemaker and planetiler are pretty good at using on-disk storage when generating tiles to minimise RAM consumption. I think your PC should be fine.
Enabling ZRAM and switching to tcmalloc (or similar) will likely help to squeeze more out of the available RAM.
zram with zstd compression is likely best if the RAM is under extreme pressure.
tcmalloc example:
sudo apt-get install libtcmalloc-minimal4
LD_PRELOAD=libtcmalloc_minimal.so.4 osm2pgsql ....