Severe disruption to QGIS OpenStreetMap tile.openstreetmap.org users

[Firefishy]

Sorry to QGIS users having trouble loading OpenStreetMap tiles from tile.openstreetmap.org.

We (OpenStreetMap Operations Team) have had to heavily rate-limit QGIS after bulk tile usage risked disrupting service for other users.

We’re working with the QGIS team to distinguish bulk requests from normal interactive use.

Ensure you have the OpenStreetMap XYZ tile resolution set correctly to reduce the disruption you experience (and the load you are placing on the OpenStreetMap infrastructure by ensuring you have the Tile Resolution setting set to “Standard” as per the following screenshots:

[Firefishy]

The QGIS team are working on adding features to help us identify Bulk vs Interactive tile usage, eg: https://github.com/qgis/QGIS/pull/46731

There have also been recent improvements to caching and automatically setting the Tile Resolution settings.

Updating to the latest QGIS version will improve your access to our tile service.

[Firefishy]

During peak hours I am now restricting QGIS to zoom 0 to zoom 18. For now zoom 19 will return 404. Zoom 19 appears to primarily used for bulk downloads / exports, and it places a huge strain on the server’s limited resources.

[Adrian_Shobrooke]

Peak hours for where? Perhaps z19 should return 404 for anywhen & anywhere?

[Firefishy]

European working day and z19 will 404 everywhere if the request is sent to the backend.

[gdt]

First, thanks to the ops team for keeping this stuff working.

I mostly understand what’s going on – the grafana data is really interesting about 0800-1600 CET/DST. I’m less clear on what if anything qgis should be doing incrementally and if I should be doing anything in packaging.

I checked a project and my resolution was “unknown (not scaled)”. I changed it to Standard per above but I wonder if that was a problem. The layer has been in the project for years.

I’m running 3.44.9, and I hope that counts as “recent qgis version”. I don’t know if the “{usage}” PR is in 3.44.x, or if that’s what’s needed to identify use as interactive, so that "export” (?) or “unknown” can be blocked separately. Does the TMS URL need to change?

I’m guessing there are also newer caching settings, larger cache, longer expiry sort of thing. If 3.44.9 or 4.0.2 is not an ok thing to run, please clarify.

Thanks,

Greg

[Firefishy]

Using the tile.OpenStreetMap.org layer in QGIS has become extremely popular in the last few years. The traffic levels have grown to a degree with which we have started to struggle.

QGIS traditionally has not been very good at caching the map tiles, downloaded large swaths of tiles that are never viewed (off-screen, deep zoom for map exports etc) and downloaded 4x as many tiles as required due to incorrect scale setting. QGIS developers have improved some of these in the very latest v3 and v4 releases.

A near future version of QGIS will provide “usage hints” so that we are able to set the correct quality-of-service tier for the request (interactive, bulk-download, export, etc)

As a way for us to limit the extreme traffic we are seeing from QGIS we have intentionally directed all QGIS to a single server (wawel) to ensure our full backend cluster does not get overwhelmed during peak hours.