Technical updates to the tile.openstreetmap.org service (OpenStreetMap.org Standard Layer)

I redirected it in Tile disk usage: Difference between revisions - OpenStreetMap Wiki if sysadmins are not maintaining it anymore

2 Likes

I would redirect them to maintained pages covering the same topic or point at external resources

I would - even with the outdated data - keep that page (tile disk usage) as it mentions the date and by that show it’s outdated state.

It would have just been nice to see how that usage (e.g. size of meta tile cache at one of the rendering servers per zoom level) would have evolved over time.

The other data from that table (tiles viewed per zoom level) could probably be digested from the published tile logs but not the (meta)tiles on disk per zoom level.

1 Like

well, it also generates support requests directed at sysadmins

That was not really my intention, but you are right.

So for the moment I self-assigned the job of getting some useful statistics for 2025 towards myself (using the already available public data like e.g. the tile_logs published at planet.osm.org) to update that wiki page in the next days.

Are you ok on not-redirecting it (as somewhat useful - while even outdated - data gets hidden at the moment by redirecting it)? When I first started diving into tile rendering servers several years ago that information was useful and still could be for newbies (and would be even more useful with more up-to-date data)…

Maybe then it would be better to include that information into switch2osm.org into the serving tiles section but for the time being and until that happens I still find that wiki page useful - even with outdated data (e.g. % of world viewed in higher zoom levels, etc.).

yes, I do not feel strongly about it, though maybe I would try to purge outdated info or more clearly indicate its historic status and that it is not supported anymore by sysadmins and they should not be bothered about updates

2 Likes

I’m a bit puzzled by the tile rendering stats. The “tile served by zoom” plot tops out as expected at zoom 19, but the other displays show zoom 20 - is this is a display issue?

Either way, it’s interesting that the current stats for “effort per zoom” appear objectively different to the 2015 figures (now gone?) in which the “effort per zoom” seem to tail off after zoom 17.

Currently, the effort is steadily increasing with zoom level, albeit ~linearly rather than quadratically.

Perhaps there is still some systematic tile scraping going on where zoom 19 tiles are being requested in a bot-like fashion rather than because somebody is seriously interested in zooming all the way in?

Obviously this would be a major argument against increasing the maximum zoom level rendered.

OpenHistoricalMap would also benefit from the fresher tiles. There seems to be a consensus among mappers there that the Standard tiles are useful as a locator tool for finding one’s way around blanker parts of the map, as well as to compare and contrast the two projects. That said, the cached tiles aren’t an urgent problem, as most mappers know how to use OSM too.

1 Like

Support requests are part of the problem, but the other is that people might use the information. The information on the page is outdated, but even if we updated it the numbers would be specific to the OSMF setup and would require knowledge of our even/odd backend split, pre-rendering, and automated tile cleanup.

For anyone who is curious, this is the current data for one of the vector tile servers.

zoom bytes count
0 53281 1
1 116249 4
2 685298 16
3 1454278 64
4 2495873 256
5 58333218 1024
6 156780519 4096
7 535594846 16384
8 1069647741 65536
9 2152974634 262144
10 2564640105 176280
11 3203532584 470408
12 4515519811 1436660
13 5771042740 4605588
14 12692907474 16154156

It changes over time on the high zooms as more editing is done

1 Like

Hello, I didn’t know how to add it when using it before © I apologize for the inconvenience caused by the inability to access attribution information such as contributors to OpenStreetMap! Afterwards, we have followed the Tile usage policy, but still cannot restore access. Is there any way to reply? Please

We need to know your site address to unblock it (this is done manually). Can you share it?

Can’t see anything, not even a demo, without first creating an account?

Not a good start!

Here is why you have been blocked: tracksolidpro.com · Issue #60 · openstreetmap/tile-attribution · GitHub
You were using OpenStreetMap tiles without attributing us. This is against our tile usage policy.

Let us know when you correctly show our attribution, and we will then look at removing the block.

7 Likes

The previous update was well received, so here are the improvements we’ve made over the last month.

1. New Backend Render Servers in Taiwan

We’ve added 2x new backend render servers, generously hosted by Taiwan Digital Streaming Co: azure-01 (currently offline due to a hardware issue) and azure-02. These servers are located in Taiwan, and the CDN now directs a portion of backend traffic for the Asian region to this new pool.

We’re still very keen to expand capacity across Asia, APAC, and Oceania, and we’ll continue to improve load balancing in these regions over time.

2. Improved CPU Performance via BIOS Tuning

We’ve improved CPU performance by tuning the energy_perf_bias setting and related BIOS options - including enabling Turbo Boost and Sub-NUMA Clustering where appropriate.

On one previously misconfigured system, this reduced the osm2pgsql initial import time from 41 hours to just 14 hours - a substantial improvement.

BIOS Settings Before on a misconfigured system:

BIOS Settings After:

3. Resolved IPv6 Backend Performance Issue

We investigated and resolved a frustrating performance issue affecting IPv6 connections to backend servers during high load. Due to a bug in apache, socket sharding was only enabled on IPv4 listeners, leading to sporadically slow uncached tile loads for users accessing tiles when the backend used an IPv6 connection.

As a workaround, we’ve switched all OpenStreetMap.org servers to IPv6-only apache listeners, which avoids the bug and results in noticeably improved miss performance. We have also updated our Fastly CDN configuration to now prefer IPv6 connections to the backend. :nerd_face:

4. Backend Server OS and Database Upgrade

We have now completed the migration of all backend render servers from Ubuntu with PostgreSQL 16 to Debian with PostgreSQL 17. The relevant infrastructure code has also been updated to only test against Debian going forward.

5. Targeted Blocking of Abusive Scrapers

Additional targeted blocking of abusive tile scraper that were disproportionately overloading our tile servers and degrading performance for the vast majority of users.

These requests violated our tile usage policy and consumed an unfair share of resources. Blocking them has helped improve overall service reliability and responsiveness.

6. Improved CDN Backend Pool Balancing

We’ve rebalanced the CDN backend server pools to improve responsiveness during regional daily traffic peaks. We’ll continue to monitor the balancing.

7. CORS Header Added to Error Tiles

We’ve added the appropriate CORS response headers to error tiles, which resolves an issue where these tiles were not being displayed in some cases.

8. CDN Backend now using TLS 1.3

We’ve switched all CDN backend connections to use TLS 1.3, reducing connection negotiation overhead and improving performance.

39 Likes

I still think it’s puzzling how much of the rendering effort is dominated by the maximum zoom level. A harmless explanation would be lots of local businesses showing an OSM map zoomed in at Z19 on their address. An alternative hypothesis is that there is still a lot of systematic scraping going on? Can we distinguish between these hypotheses?

1 Like

It is a combination of there being so many more Z19 tiles, than Z18, than Z17 etc, and some systematic scraping. Only a tiny fraction of Z19 tiles are ever view and therefore rendered.

2 Likes

The apache bug has now been logged in apache’s bug tracker: Log in to ASF Bugzilla (login required)

1 Like

There are 4 times as many Z19 tiles as Z18. From the “tiles served per zoom” data the ratio Z19 / Z18 is about 1.4. So ~35% of the Z19 tiles are being served. This doesn’t look like a tiny fraction to me, or is my maths wrong?

Here is the number of metatiles from 3 of our large servers:

culebre:
Count Zoom
      1 0
      1 1
      1 2
      1 3
      4 4
     16 5
     64 6
    256 7
   1024 8
   4096 9
  16384 10
  65536 11
 262144 12
 164499 13
 424485 14
 835669 15
1398707 16
2903481 17
4789044 18
5306372 19

nidhogg:
      1 0
      1 1
      1 2
      1 3
      4 4
     16 5
     64 6
    256 7
   1024 8
   4096 9
  16384 10
  65536 11
 262144 12
 255181 13
 512959 14
 660860 15
1100518 16
2202034 17
3494925 18
3583741 19

palulukon:
      1 0
      1 1
      1 2
      1 3
      4 4
     16 5
     64 6
    256 7
   1024 8
   4096 9
  16384 10
  65536 11
 262144 12
 352784 13
 484247 14
1170988 15
2202259 16
3215629 17
4775397 18
3382988 19
2 Likes

Still pretty new here: does every step up in the zoom level always halve the x and y dimensions of the area being viewed? Ie. quarter the area being viewed, and therefore quadrupling the number of tiles? So tiles = ceiling(4(zoom - 3))?