Minutely updated vector tiles demo

I’ve put up a demo page showing my work on minutely updated vector tiles. This demo is using my work for the tiles and the Versatiles Colorful stylesheet. There are some technical details on my diary but I wanted to expose the work to a wider audience.

These tiles are updated minutely but may lag a few minutes behind due to various levels of caching. This won’t be an issue later on as it’s just cache settings that need changing, not the backend.

I’m interested in feedback. The part that I’m working on is just the generation of tiles so if you know the issue is with the style you’re better off opening an issue there. The advantage of vector tiles is that if you don’t like the style, you can use your own.

40 Likes

Looks like there’s a CORS issue when trying to grab https://demo.tilekiln.xyz/shortbread_v1/tilejson.json, might be why I’m not getting anything rendered?

Well that error has gone away, not sure if a quick fix or my end bugging out before, but it looks good now.

Not sure whether the cache is holding longer than expected but I’m not seeing Changeset: 148232861 | OpenStreetMap reflected at https://pnorman.github.io/tilekiln-shortbread-demo/#16.74/-34.677331/138.678987 yet, even after 10 minutes since first loading and over an hour since the change. Still shows the two changed roads as the same as other secondary roads.

Hi Paul,

It’s beautiful :slight_smile:

Request for the readers:

If someone has already put together a docker-compose based setup that only requires specifying a small BBOX of interest, which automatically updates the data and generates tiles within it, I would be happy to focus on testing, multilingual support, and possibly even operating a small test server similar to this. So, if anyone has a setup like this already configured, could you please share it?

Is there a straightforward way to tell which changeset triggered the latest update of a particular tile, or its timestamp? Is this something one can query in Prometheus as well? That could be useful for when, say, someone drains one of the Great Lakes by accident, since the changeset bbox could technically be elsewhere.

4 Likes

There seems to be a rendering problem with roads and other linear features at the border of two tiles:

grafik

grafik

2 Likes

This change is now showing properly, could the cache be lasting up to two hours, or is there a rendering delay as well at the moment?

1 Like

Thanks. This is a problem with insufficient buffer.

1 Like

Another find:
A (curved) way with many nodes close to each other produces rendering artifacts:



https://pnorman.github.io/tilekiln-shortbread-demo/#19/49.5879206/11.0348144
https://pnorman.github.io/tilekiln-shortbread-demo/#19/49.5837502/11.0252494

3 Likes

That’s a great debugging feature, helps us find those grandiosely over-noded ways :wink:

21 Likes

Thanks. If you could give a link to the url for the demo it would help to find the location and zoom you’re seeing the problems on. This problem is a bit trickier. Any map layer starts to exhibit problems if you zoom in enough, and if you have a very finely noded curved way it will always experience jumps like this if you zoom in enough because there is coordinate rounding. It looks like this is happening on z19, which is equivalent in scale to z20 on a typical raster map. I’ll look at what scale we’re targeting and increasing the extent.

There shouldn’t have been - everything was working fine when I checked it. Fastly is set to a 5 minute TTL and Varnish to 5m, so the most there should be is about 15 minutes.

There’s a few possibilities

  • an exceptionally big change came in and that took longer because many tiles needed generating
  • the tile wasn’t indicated as having changes by osm2pgsql, because the algorithm is known to not be perfect, and then a later change came in that touched that tile

Added the links to my post.

No. Changesets objects aren’t in the minutely data diffs and the last remains of them are stripped out early in the processing.

I will be adding last-modified headers for cache reasons, but this is not the same as when the changes were made.

No. Per-tile metrics are not possible to store in Prometheus.

1 Like

My usual issue with vector tiles is geometry simplification being too aggressive, but since that doesn’t seem to be the case here, that’s something going right :slight_smile:

1 Like

There’s no true simplification going on here at all. There is a resolution reduction and de-duplicating step (ST_AsMVT) but that is adjusted by the resolution used and should hold up well till the scale of z19 raster tiles.

If you try zooming in quickly on a slow connection you will see very simplified lines but this is better than rasters where you see a blur.

I also find many tile sets oversimplify. I understand why because it’s an easy way to reduce tile size but it is frequently overdone. This is best seen at a zoom like 13.99 where the tiles are 1024 pixels wide.

I do expect to need to introduce some simplification but because I’m aiming at mappers it adjusts the priority of reproducing the original geometries compared to performance and tile size.

4 Likes

I call that quite ambitious; I guess the interests of mappers – though less in numbers – in the minutes presented vary much more than the interests of consumers. I might be wrong though.

I’m not aiming to meet the needs of every mapping-related use case that mappers have. No one service can do that. But minutely updates are really only of use to mappers so the biggest single feature is for them.

4 Likes

Indeed this is showing that more nodes for rounded ways is not always better.

There seems to be some issue with rendering railways, service roads et cetera (which I assume is because of those tags not being covered in the style yet). Here’s an instance of that in Sealdah, India.


2 Likes