Overpass API performance issues

I imagine there might be differing query strategies depending on the size of the bbox. But someone would have to read the source code to figure that out, and I haven’t looked at it.

Please have a look for Query_Filter_Strategy. In particular in bbox_query.cc, it does switch if the area is bigger than one square degree. That heuristics might need to be adjusted or even refined in newer times, but I have not done any analysis about that recently.

There is another independent mechanism kicking in for very large amounts of objects (millions of ids). This is solely to avoid that the interpreter tries to keep all ids of all building=yes in memory, which would usually surpass available memory. It then resorts to full planet scan in a very high level view.

@lonvia

is

As a broad guideline to stay within safety margins, users are expected to send a maximum of about 10000 requests per day and keep their download volume below about 1 GB per day.

from Commons still true?

Should it be removed from Overpass API - OpenStreetMap Wiki ?

I listed there new rules that I am aware of.

That limit is fine for a one-off use of Overpass. If you set something up that uses the Overpass API regularly, then divide those numbers by 100.

The servers can do about 1.5 million requests per day at the moment. It’s hard to estimate the number of users in the days of botnets. But if we work with the 30k users that Roland has estimated, then you can do the math of what constitues a fair share: 50 request per day or less.

Please also note that limits should always be counted towards an application. If you have an app or website, then the usage counts towards the sum of requests made by all your users.

I edited wiki to mention that.

To put it another way, if you’re using more than about 150 seconds of query processing time total for all your queries in a single day, you’re using more than your share.

(That’s based on the 30k users estimate.)

BTW, FairwayMapper has still some empty spots. Allotments is fairly small and usage requires passing authorization header, but I can confirm it works well.

Thanks @Mateusz_Konieczny! As an FYI, we’ve increased the free tier allocation to 2000 requests per month per user from 500. Modest still, but I’m still feeling out the edges of what our hardware is capable of - I am hopeful I can bump up the number of slots once I’ve built a bit more confidence that it’s not all going to come crumbling down :slight_smile:

If anyone has any useful experience of semi-realistic load testing of overpass queries I’d be keen to chat - might help me get to the conclusion faster.