Overpass API performance issues

Technically you could run it on your laptop if it is beefy enough (or if it is not, but you don’t care about whole planet but a smaller region like your own country)[1]

It took me about 5 hours walltime (4 and half of them waiting for downloads of full preprocessed planet to finish, which I used productively elsewhere), and running that docker container was not too complicated (most of the work was reading the docs, setting up HTTPS certificates and redirection[2] and securing it a little) on Debian Trixie.

apt install docker.io docker-cli nginx certbot python3-certbot-nginx # Debian Trixie as example...

certbot --nginx -d overpass.example.com # or however you like to get your HTTPS certificates

mkdir -p /home/overpass/db # or wherever you have some ~300GB free (you can skip those and "-v" volume mapping options below, if you have enough space in /var/lib/docker. I didn't) 

chmod 777 /home/overpass/db # you really should protect it better, but if you're the only one ever able to access it, it might be acceptable 

docker run -d \
    --restart unless-stopped \
    -e OVERPASS_META=no \
    -e OVERPASS_MODE=clone \
    -e OVERPASS_STOP_AFTER_INIT=false \
    -e OVERPASS_DIFF_URL=https://planet.openstreetmap.org/replication/minute/ \
    -e OVERPASS_ALLOW_DUPLICATE_QUERIES=yes \
    -p 127.0.0.1:80:80 \
    -v /home/overpass/db:/db \
    --name overpass_world \
    wiktorn/overpass-api # and wait some hours while it finises

(you can change that OVERPASS_META to yes or even attic, but it is going to be slower and use much more disk space) and creating /etc/nginx/sites-available/overpass (and symlinking to sites-enabled) with something like:

server {
    listen 443 ssl;
    server_name overpass.example.com;

    ssl_certificate /etc/letsencrypt/live/overpass.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/overpass.example.com/privkey.pem;

    location / {
        # basic (but easily fooled) protection against random bots (i.e. only allows connection if used as  custom server on overpass-turbo.eu or openstreetbrowser.org). But IP locking below is much preferred.
        #if ($http_referer !~* "^https?://(www\.)?(overpass-turbo\.eu|openstreetbrowser\.org)/") {
        #    return 403;
        #}

        # those three lines are optional but really recommended to more securely lock access to your IP(s) only, lest AI bots hammer you to death too, or some exploit hits you. You should change them to your IPs, obviously.
        allow 192.0.2.123;     
        allow 127.0.0.1;
        deny all;

        proxy_pass http://127.0.0.1:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

#server {
#    listen 80;
#    server_name overpass.example.com;
#    return 301 https://$host$request_uri;
#}

And then service nginx restart, clicking Settings on https://overpass-turbo.eu and changing overpass API URL to https://overpass.example.com/api/ (you obviously replace overpass.example.com everywhere with your server FQDN).

(I might have missed some small step, but hopefully nothing too critical).

But now I feel as I king of the speed (it is running only my own simple queries) and no errors finally :smiley:


Next stop: find out why it doesn’t seem to be minutely updating:

Update: Updated the command to fix updates, can be verified with:

% curl --noproxy "*" -s "http://localhost/api/interpreter?data=\[out:json\];node(1);out;"  | grep timestamp
    "timestamp_osm_base": "",

As others have noted, it is best not to interrupt that docker while it’s updating, or it might get stuck and then you’ll either have to learn how to fix it manually, or just drop the docker container and start all over again.


  1. it might save you some effort, as you IIRC might avoid CORS HTTPS requirements and whole of nginx setup, if your overpass host is on “http://localhost”, but I haven’t actually tried it ↩︎

  2. after finding out that unencrypted HTTP isn’t going to work with https://overpass-turbo.eu frontend ↩︎

2 Likes