What Are the Costs VS Utility of the Tile Servers to the OSM Community?

Foundation
1

Moving this discussion from the OSMF listserv, per Andy T.'s good advice:

What Are the Costs VS Utility of the Tile Servers to the OSM Community?

This is a response to Steve C’s comment below. Note that the question isn’t “what are the technical benefits of the tile servers?” That is a different topic. This is about the overall value of the tile servers versus the costs of maintaining them.

C

On Sun, Oct 20, 2024 at 10:15 AM Steve Coast wrote:

On Oct 20, 2024, at 5:19 AM, Mikel Maron wrote:
In 2023, we organized a successful fundraising campaign Fundraising | OpenStreetMap Blog (the total announced results were £373,000, not sure if that’s the source of the $300k figure).

Yes, and that is genuinely great, but the question is what do we get for the money?

The major output remains the free global tile server, where something like 99% of the usage is by third parties and not OSM. We create our own funding requirement and dedicate our sysadmin time to running this thing despite completely free alternatives now existing (e.g. https://openfreemap.org/ ) and then celebrate the “growth” of our “service”.

We could use these resources to, for example, complete the map.

It seems cheaper and more effective marketing to run a free OpenStreetMap coffee and hot dog stand in Times Square, New York City.

Best
Steve

4 Likes
2

Claim here seems to be that map tiles widely used for example in OSM editors such as JOSM, on osm.org page, in QGIS and on many sites that typically attribute OpenStreetMap are literally worthless. And that it brings nothing at all to OpenStreetMap community and that running random hot dog stand at slight loss would be as helpful.

This claim seems clearly and blatantly wrong.

Lesser claim that it is not maximally effective use of resources is more viable but specific alternative would need to be mentioned. Comparing specific real work (with all its warts and deficiencies that will be on any real activity) to platonic ideal of ideal and not specific spending is not helpful. Let’s say we shut down tile servers: how this funds and resources be used?

Also, [citation needed] for that 99% claim. Is it based on anything or is it comedic exaggeration pretending to be serious statistics?

14 Likes
3

MOROW GitHub - chrisdebian/MOROW: "Maintaining Our Rights of Way"- An initiative, initially for the UK, to encourage use, discovery, mapping and reporting of Public Rights of Way (PRoW).

As a fledgling app that will ‘give back’ to OSM, I could certainly make use of the tile server, and I’m sure I’m not alone:

Is it possible to look at the access logs, to get an understanding of who/ what is using the server?

Many thanks,

Chris
chris_debian

4 Likes
4

https://planet.openstreetmap.org/tile_logs/hosts-2024-10-19.csv

3 Likes
5

More important than the stats I would point to A Year of Infrastructure Progress: Site Reliability Engineer 2023/2024 Update | OpenStreetMap Blog

Notably absent is any prominent mention of the tile service.

7 Likes
6

I cannot fathom this question could come from anyone other than long-time OSM contributor, to whom OSM being a thing is such a paradigm, they cannot grasp someone else’s way of thinking.

The argument is anecdotal, but hopefully non-controversial: most people do not know what OSM is, period. Its actual presence in life is not even close to the brand exposure, which I find as crucial to finding new contributors, big and small. I am yet to meet a person outside of my Internet bubble, who said something along the lines: ‘Oh, I totally recognise OSM, that’s where my Snapchat / Facebook / other map sources its information from!’. The current website is not a good representation of data richness, as it’s not interactive, but after 20 years of the project’s existence, I am yet to find a website that would present this data: a) better b) not disappear after a handful of years (recent example being Qwant Maps). I am baffled that when trying to onboard someone, I am struggling to point them to anything other than osm.org for them to see how great of a resource I just described.

The current inability of OSM to present itself on its own is probably a great news for some of its corporate users, who overwrite its branding with their own (and we know them), but IMHO not great to the project overall. My stance is, that while the Foundation’s stated purpose is not to complete with these commercial projects, there’s really not one in this area. Current OSM is, I believe, too weak. Can we really hope that we’ll get potential contributors by pulling them straight into editor, saying ‘It’s good for everyone, trust me’. Google doesn’t seem to have to convince POI owners to add and update themselves on their own. I am yet to hear about one, who has done so for OSM, because they saw a Copyright tag in their app’s settings.

Please make a case for your argument of ‘completing the map’. For now, I do not believe making any editor 15% better would give us a better community return long-term, if that’s what you aim for. Not going to go into depth about the clearly American-centric comment; the website is available (mostly) worldwide.

11 Likes
7

Vector tiles related news ( OSMF )

Tile server - wish list ( ~ utility )
related to the E-TOP10 task:
“Top Ten Tasks - Engineering Working Group”

  • |1.1|Localized map rendering
    • ~ multiple languages support
  • |1.2|OAuth login to wiki
  • |1.3|OSM API deleted items call
  • |1.4|List of changeset discussions for a user
  • |1.5|Better tools for welcoming new users
  • |1.6|Area datatype
  • |1.7|Improved activity/history tab
  • |1.8|Clickable POIs on the frontpage
  • |1.9|Pedestrian and bike routing on areas
3 Likes
8

The biggest value of the tiler server to me is that it shows extremely fast updates of the underlying data, which is rare if not unheard of completely for commercial map providers.

In any product use case I would rather use something from MapTiler or Stadia or set up my own Protomaps system while trying to apply some available map style, but in QGIS, apps like EveryDoor, and other environments where I want to quickly get a look at OSM as a basemap that tile layer is great.

If MapTiler or others had a vector or even raster basemap that also provider hourly or faster syncs to OSM with free use for some limit of requests I would however probably stop using the OSM endpoint entirely.

I think the foundation should actually encourage someone to do that, or should setup a very sklle service of their own that would ask for a subscription above a very modest amount of use, in a way that would help generate a little funding.

8 Likes
9

If you look at the CSV Simon linked to, and do the maths, you’ll see that 79.7% of all tile requests are NOT openstreetmap.

These include global corporations, porn sites, hobby sites, you name it.

I’m merely saying that MAYBE we shouldn’t be running a free tile server for the entire world when

  1. you can now get this for free https://openfreemap.org/
  2. We don’t have infinite resources

We could keep the tile server for anonymous people using openstreetmap.org and free up 80% of the bandwidth, capacity and money for other things.

It’s arguable the tile server should really only be for mappers who are logged in, in which case it will probably drop 99.99%, but I wouldn’t go that far today.

How does this make any sense to give free tiles to everyone? It doesn’t. It would make more sense to run a free hotdog stand, because then at least we’d put logos on the coffee cups and get some marketing value out of it.

5 Likes
10

For the not so old timers (I suppose I have to admit to nearly being one): there has always been tension between, in the extremes, a faction lobbying for essentially no tile service at all, Steve being one of them, and the other extreme the “free tiles for all” supporters.

And there are some good arguments on both sides. However while historically this was one of the main things that operations was buzzing around and a major cost to the OSMF in more than one way, that is currently far less so.

It is likely that some of the same questions (for example should we be competing with commercial services) will become more current again when the vector tile service is launched, so I suppose we should stay tuned.

9 Likes
11

Point of order - the tile service is now competing with free services, not commercial services.

12

Point of order: we don’t, lots and lots of sites are blocked (or have been told to go elsewhere).

3 Likes
13

Simon - blocking the rest should be no big deal then, and, free up resources spent managing all the blocking… Do we really want to pay people to manage blocking and choosing who gets free tiles this week?

14

I’m not going to repeat all the pros and cons from the last 20 years, but one thing is clear: the only action that would actually remove substantial effort would be limiting access to only openstreetmap.org.

If you go any further than that (aka OSM related sites, apps and so on), you end up with having to admin in the other direction, without the benefit of the marketing effect of running the 2nd most recognizable map tile service in the world.

16 Likes
15

Good points above on what tiles are about and the actual minimal resource cost to maintain them. From what I understand, it would be more of a pain to restrict them further than any gains in maintenance.

The latest iteration of this debate came up in discussion of corporate donations to OSMF

Yes, and that is genuinely great, but the question is what do we get for the money?

This is off-topic on the current thread (?), but I do want to quickly stub out responses to a very good question, that thread out several other discussion items as well.

OSMF has raised funds specifically to support our core infrastructure and improve services and user experience. That means money spent on our SRE (A Year of Infrastructure Progress: Site Reliability Engineer 2023/2024 Update | OpenStreetMap Blog), vector tiles development, iD maintenance, and several other operations and engineering related work.

Looking forward, OSMF does need to prioritize where future technical investments are made. That itself takes effort. I’m not at liberty to discuss all the details yet, but there are plans to address this.

On promotion of the project, there is a lot more we can do. This year we celebrated OSM’s 20th birthday and the CWG built a hub and campaign to get support the community celebration around the world. Outside of the community, OSM should be better known. There are small things we can do – like implement plans to make the /copyright page a better welcome (Update Copyright Page · Issue #5132 · openstreetmap/openstreetmap-website · GitHub), to demanding better recognition for Overture as they take a lot of thunder for data distribution, to getting OSM placed more in the bigger societal conversation (like https://www.nytimes.com/2024/07/10/opinion/google-maps-driving-apps-flaws.html).

One area that overlaps with the technical direction is modernizing distribution of OSM data in cloud native formats. Map engineers are buzzing about what this is opening up, and reaching for Overture because they make it easy and document well. No reason OSM shouldn’t be the test bed of choice for this next iteration of geospatial data distribution.

7 Likes
16

Let’s say we do this - how this funds and resources be used?

Is it better use than current tile server use, including in JOSM and in ways promoting OSM?

(And even if that is better use than public tile server - is there maybe other thing worth cutting first?)

2 Likes
17

I am on holiday this week in Southern Turkey, but here is my 1 Turkish Lira’s worth as a OSM Operations member:

We publish logs for the tile service, do take a look: Index of /tile_logs
I don’t or need to keep tabs, but I think our biggest tile user is QGIS.

The automated processing of the logs ironically would be our highest ongoing cost, if it were not for the free AWS credits we use to run the processing. See AWS costs discussion here a year ago.

Fastly kindly provide the CDN for tile.openstreetmap.org for free. Their service is amazing and it offloads 90%+ of all requests. Cache hit % would be much higher if we wanted to serve users less fresh tiles.

In 2024 we have spent only around £1000 on the rendering servers, we upgraded RAM and CPUs in 2 of the render servers, see ticket 1, ticket 2 on GitHub ops issues. We also replaced a failing DIMM. 2023 was minimal too, with minor upgrades. Modern hardware is blazingly fast. 1 North American render server is sponsored by AWS, 2 render servers are sponsored in Australia. See hardware.osm.org for the full list. We do pay for rack space and power for the racks in Amsterdam and Dublin, but we have lots of spare space in the racks, hosting tile render servers does not crowd out anything. Power is purchased in blocks and we use minimal amounts. Bandwidth is not metered and our uplinks are not oversubscribed. We are hoping to be able to announce a very large reduction in our hosting costs later this year or early next year, work-in-progress.

Ops admin time is minimal, the server setup is fully managed by chef configuration as code, 1 or 1000 servers is nearly same amount of effort. Very minor initial server setup / import is manual. We occasionally make some manual tweaks to regional load balancing, but failover is automatically handled by monitoring and health checks. Attribution blocks are automatically managed by OpenTofu triggered by a public GitHub tile attribution issue tickets. There may be a small opportunity cost, but we are mostly volunteer run, and definitely open to new things.

OSMF paid for minutely vector tile development work this year, and we recently got it up and running. The service currently running dark as a semi-private service. I expect public announcements when ready. Minutely updated vector tiles demo - #125 by pnorman

I am back from holiday next week Friday, if there is a strong desire for a full cost breakdown, I can produce one then.

(Typed from phone. Will add links once at a real computer)

46 Likes
18

I suspect the entire operation of the OSFM could be funded, and then some, with a very modest subscription fee along these lines.

1 Like
19

Just two quick notes:

About “marketing”. Being known as the world’s second largest tile server to people who use tile servers doesn’t get OSM anything other than the responsibility of that. We don’t even ask people to give money when they use it, or even ask them to thank us.

The point of marketing anything is to leverage a return that can benefit the operation. There are a lot of things we should market about OSM, and being the world’s largest free tile server is not anywhere on the list.

About “free” donations. Just because something is donated to us, doesn’t make it free. Such resources and good will also have value. It is great that most of the costs, in this case, are underwritten - that may well be the end of the story, though I do think a complete report on an annual basis would be good practice.

In any case, saying something is “free” so we don’t have to worry about “cost” is to skirt the issue. In a nonprofit, you always have to ask - could this free resource or good will be applied to another, different area that also needs attention. It’s good fiscal discipline to audit “free” (i.e. donated) resources alongside all other costs. If nothing else, it means that redundancy can be built into the system in case the “free” resource goes away.

4 Likes
20

I agree that it is a mistake to associate “free” with “worthless”.

But this donations mentioned here are not really fungible: if we kill tile servers, then we cannot really use donated CDN for something else. And it would not be replaced by a cash donation. Rather, we would not even get CDN.

The same for the donated rendering servers: we would not get servers that can be used for any purpose we want, we would not get them at all.

It is hard to estimate but I am pretty sure that benefits of tile servers are greater than costs. Even if we put repeated refuting of that 99% claim into costs.

(this is not changing fact that this can be better leveraged for marketing and fundraising)

Donated resources used to run tile servers in overwhelming part were donated because we run tile servers, and without public tile servers we would not get them. (And it applies for any reasonable estimates of work hours value and hardware and service donations value)

27 Likes