Problematic Retry-After=0 on Nominatim

General talk
1

I accidentally made too many requests to nominatim[dot]openstreetmap[dot]org (forgot to sleep my thread between requests) and got rate limited, which is expected according to the usage policy.

I wanted to implement a retry system when my software gets HTTP 429s, based on the Retry-After header, however it seems that the header is not implemented by the server. Here’s the response headers of my offending requests:

connection=close
content-length=449
server=Varnish
retry-after=0
content-type=text/html; charset=utf-8
accept-ranges=bytes
date=Thu, 14 May 2026 13:47:07 GMT
via=1.1 varnish
x-served-by=cache-par-lfpb1150034-PAR
x-cache=MISS
x-cache-hits=0
x-timer=S1778766428.558017,VS0,VE5
strict-transport-security=max-age=300
alt-svc=h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400

I first opened an issue on the nominatim issue tracker, but this issue is more an operations issue rather than a software issue.

Since I see that nominatim server is behind a varnish server, could the varnish server set the retry-after header? When I got 429s they lasted about 5 mins. I am sure the server operators know the exact value to put there.

I’m writing on the forum because it seems to be the only way to report issues related to the nominatim server, according to Nominatim’s About & Help page.

1 Like
2

Maybe you should install your own nominatim instance or use a third party provider with defined specs instead of asking the volunteers that are running a public service paid for by donations to help you in trying to technically (ab)using the osmf nominatim instance to the max according to the usage policy? I doubt that this is the intention or spirit of the usage policy…

4 Likes
3

@philt3ren Please consider paying for the service. nominatim.openstreetmap.org is run by volunteers and funded by donations. Your usage is heavy use. nominatim.openstreetmap.org is not the right service for you.

Nominatim is free and open source software. You can either set it up yourself or pay someone for you to do it. There is a list of (paid) alternatives to Nominatim on the OSM wiki. Some offer the same API as nominatim.openstreetmap.org, others use the software but offer a different API.

If you get HTTP 429 from the free service, you should not write software dealing with the limits of the service and continue stealing computing time from the community.

2 Likes
4

I am not trying to abuse the nominatim server, but correct me if I am.

My usage of the nominatim server is to use the reverse endpoint on some pictures I have to get the location of where they were taken.

I have a script to do that and I run that script only once every 6 months, making 50 requests every time it is ran.

When I ran my script yesterday, it made too many requests sequentially (because something broke on my end) and I got rate limited, which is perfectly normal.

When investigating why I got rate limited to fix that, I started to look at the HTTP headers the server replied to know when it would be okay for me to make new requests. This is when I saw the retry-after header was set to 0, meaning that it’s okay for the HTTP client used to make a new request after 0 seconds, according to the HTTP spec: Retry-After header - HTTP | MDN.

What I was trying to say by writing on the forum was: If the nominatim server sets the retry-after header correctly, then http clients should respect it to not overload the server.

Sorry for the trouble everyone. And please let me know if my usage is really abusive.

5 Likes
5

Reading your first quote again, this is actually really not ideal if this header got returned by the Fastly CDN:

So for your use case, just implement the sleep with a few seconds (as you have probably already done) between each request.

And yes, you are right, doing 50 requests every 6 month should be all fine with the usage policy as long as you don’t hammer them but let those requests dribble instead (which should be sufficient for those image tagging job)…

1 Like
6

I wonder where it should be reported. This retry-after=0 should be at least dropped, if I understood it meaning correctly.

Hopefully pinging @lonvia is acceptable?

1 Like
7

gh→osm→operations→issues or - if you can guess the VSL currently used → a pr at gh→osm→chef (but that VSL stuff is - with reason - non-public).

All this probably came from the recent need to put Fastly in front of Nominatim as well in those never-ending waves of AI scrapers and the like.

(and lonvia has seen the report via gh→osm-search→nominatim but reacted with “not planned” as the original post does not state clearly that VSL is returning a retry-after=0 but rather sounded as if no retry-after header is present in the 429 response.

2 Likes