Hi,
is anybody experiencing problems with minutely diffs in Overpass API since yesterday?
My update process for Overpass API downloads minutely diffs up to 006/177/786.osc.gz but then just hangs and doesn’t continue (no CPU load).
The only thing I find in the log is
2024-07-17 18:23:19: updating from 6177699
2024-07-17 18:23:43: updating to 6177786
Can someone help me how to proceed?
Not overpass, but minutely diff processing on this server hasn’t been interrupted.
A quick search finds this and this. You might want to check what versions of the relevant bits of software you’re using and see if there are any issues there.
I’ve had this issue too. It’s likely a problem with the diff files or a network issue. Check if the server is down or slow. Manually download the 006/177/786.osc.gz file and see if it’s corrupted. Restarting your update process from the last successful diff might help.
That’s the first things I did. The download process starts again from the last applied diff, downloads 87 files and stops again at 786. There’s nothing special with these files, using any other starting point for updates gives the same result.
update_from_dir gets started but doesn’t do anything. It looks like the dispatcher was somehow blocked for write accesses. Regular requests were served fine, but updates weren’t. A full restart of Overpass (instead of just the update process) seems to have solved the issue
Hi,
thank you for reporting the issue. It did not occur on any instance under my control, so I don’t know what happened there precisely.
Some things that might help:
The standard output of apply_osc_to_db.sh or fetch_and_apply.sh print the exact error message. If you have that then look into that.
Look into the $DB_DIR/database\.log or $DB_DIR/transactions\.log for older versions. The last dozen lines containing the string write (usually write(), but older versions have variations) give the exact point where the update stopped.
The file sizes of the diff I see are:
-rw-r--r-- 1 overpass overpass 51051 Jul 16 13:09 /srv/overpass/diffs/006/177/690.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:09 /srv/overpass/diffs/006/177/690.state.txt
-rw-r--r-- 1 overpass overpass 105670 Jul 16 13:10 /srv/overpass/diffs/006/177/691.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:10 /srv/overpass/diffs/006/177/691.state.txt
-rw-r--r-- 1 overpass overpass 117959 Jul 16 13:11 /srv/overpass/diffs/006/177/692.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:11 /srv/overpass/diffs/006/177/692.state.txt
-rw-r--r-- 1 overpass overpass 81651 Jul 16 13:12 /srv/overpass/diffs/006/177/693.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:12 /srv/overpass/diffs/006/177/693.state.txt
-rw-r--r-- 1 overpass overpass 111872 Jul 16 13:13 /srv/overpass/diffs/006/177/694.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:13 /srv/overpass/diffs/006/177/694.state.txt
-rw-r--r-- 1 overpass overpass 75558 Jul 16 13:14 /srv/overpass/diffs/006/177/695.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:14 /srv/overpass/diffs/006/177/695.state.txt
-rw-r--r-- 1 overpass overpass 90557 Jul 16 13:15 /srv/overpass/diffs/006/177/696.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:15 /srv/overpass/diffs/006/177/696.state.txt
-rw-r--r-- 1 overpass overpass 155686 Jul 16 13:16 /srv/overpass/diffs/006/177/697.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:16 /srv/overpass/diffs/006/177/697.state.txt
-rw-r--r-- 1 overpass overpass 48715 Jul 16 13:17 /srv/overpass/diffs/006/177/698.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:17 /srv/overpass/diffs/006/177/698.state.txt
-rw-r--r-- 1 overpass overpass 81046 Jul 16 13:18 /srv/overpass/diffs/006/177/699.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:18 /srv/overpass/diffs/006/177/699.state.txt
-rw-r--r-- 1 overpass overpass 48086 Jul 16 13:19 /srv/overpass/diffs/006/177/700.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:19 /srv/overpass/diffs/006/177/700.state.txt
-rw-r--r-- 1 overpass overpass 41789 Jul 16 13:20 /srv/overpass/diffs/006/177/701.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:20 /srv/overpass/diffs/006/177/701.state.txt
-rw-r--r-- 1 overpass overpass 151133 Jul 16 13:21 /srv/overpass/diffs/006/177/702.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:21 /srv/overpass/diffs/006/177/702.state.txt
-rw-r--r-- 1 overpass overpass 49315 Jul 16 13:22 /srv/overpass/diffs/006/177/703.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:22 /srv/overpass/diffs/006/177/703.state.txt
-rw-r--r-- 1 overpass overpass 103384 Jul 16 13:23 /srv/overpass/diffs/006/177/704.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:23 /srv/overpass/diffs/006/177/704.state.txt
-rw-r--r-- 1 overpass overpass 143269 Jul 16 13:24 /srv/overpass/diffs/006/177/705.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:24 /srv/overpass/diffs/006/177/705.state.txt
-rw-r--r-- 1 overpass overpass 76148 Jul 16 13:25 /srv/overpass/diffs/006/177/706.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:25 /srv/overpass/diffs/006/177/706.state.txt
-rw-r--r-- 1 overpass overpass 126297 Jul 16 13:26 /srv/overpass/diffs/006/177/707.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:26 /srv/overpass/diffs/006/177/707.state.txt
-rw-r--r-- 1 overpass overpass 54756 Jul 16 13:27 /srv/overpass/diffs/006/177/708.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:27 /srv/overpass/diffs/006/177/708.state.txt
-rw-r--r-- 1 overpass overpass 39782 Jul 16 13:28 /srv/overpass/diffs/006/177/709.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:28 /srv/overpass/diffs/006/177/709.state.txt
-rw-r--r-- 1 overpass overpass 65462 Jul 16 13:29 /srv/overpass/diffs/006/177/710.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:29 /srv/overpass/diffs/006/177/710.state.txt
-rw-r--r-- 1 overpass overpass 109989 Jul 16 13:30 /srv/overpass/diffs/006/177/711.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:30 /srv/overpass/diffs/006/177/711.state.txt
-rw-r--r-- 1 overpass overpass 152865 Jul 16 13:31 /srv/overpass/diffs/006/177/712.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:31 /srv/overpass/diffs/006/177/712.state.txt
-rw-r--r-- 1 overpass overpass 46616 Jul 16 13:33 /srv/overpass/diffs/006/177/713.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:33 /srv/overpass/diffs/006/177/713.state.txt
-rw-r--r-- 1 overpass overpass 58794 Jul 16 13:33 /srv/overpass/diffs/006/177/714.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:33 /srv/overpass/diffs/006/177/714.state.txt
-rw-r--r-- 1 overpass overpass 79740 Jul 16 13:34 /srv/overpass/diffs/006/177/715.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:34 /srv/overpass/diffs/006/177/715.state.txt
-rw-r--r-- 1 overpass overpass 49756 Jul 16 13:36 /srv/overpass/diffs/006/177/716.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:36 /srv/overpass/diffs/006/177/716.state.txt
-rw-r--r-- 1 overpass overpass 70428 Jul 16 13:36 /srv/overpass/diffs/006/177/717.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:36 /srv/overpass/diffs/006/177/717.state.txt
-rw-r--r-- 1 overpass overpass 95109 Jul 16 13:38 /srv/overpass/diffs/006/177/718.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:38 /srv/overpass/diffs/006/177/718.state.txt
-rw-r--r-- 1 overpass overpass 46247 Jul 16 13:39 /srv/overpass/diffs/006/177/719.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:39 /srv/overpass/diffs/006/177/719.state.txt
-rw-r--r-- 1 overpass overpass 88289 Jul 16 13:40 /srv/overpass/diffs/006/177/720.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:40 /srv/overpass/diffs/006/177/720.state.txt
-rw-r--r-- 1 overpass overpass 73560 Jul 16 13:41 /srv/overpass/diffs/006/177/721.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:41 /srv/overpass/diffs/006/177/721.state.txt
-rw-r--r-- 1 overpass overpass 84632 Jul 16 13:42 /srv/overpass/diffs/006/177/722.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:42 /srv/overpass/diffs/006/177/722.state.txt
-rw-r--r-- 1 overpass overpass 79363 Jul 16 13:43 /srv/overpass/diffs/006/177/723.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:43 /srv/overpass/diffs/006/177/723.state.txt
-rw-r--r-- 1 overpass overpass 41418 Jul 16 13:44 /srv/overpass/diffs/006/177/724.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:44 /srv/overpass/diffs/006/177/724.state.txt
-rw-r--r-- 1 overpass overpass 125368 Jul 16 13:45 /srv/overpass/diffs/006/177/725.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:45 /srv/overpass/diffs/006/177/725.state.txt
-rw-r--r-- 1 overpass overpass 54286 Jul 16 13:46 /srv/overpass/diffs/006/177/726.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:46 /srv/overpass/diffs/006/177/726.state.txt
-rw-r--r-- 1 overpass overpass 85419 Jul 16 13:47 /srv/overpass/diffs/006/177/727.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:47 /srv/overpass/diffs/006/177/727.state.txt
-rw-r--r-- 1 overpass overpass 46661 Jul 16 13:48 /srv/overpass/diffs/006/177/728.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:48 /srv/overpass/diffs/006/177/728.state.txt
-rw-r--r-- 1 overpass overpass 29426 Jul 16 13:49 /srv/overpass/diffs/006/177/729.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:49 /srv/overpass/diffs/006/177/729.state.txt
-rw-r--r-- 1 overpass overpass 39108 Jul 16 13:50 /srv/overpass/diffs/006/177/730.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:50 /srv/overpass/diffs/006/177/730.state.txt
-rw-r--r-- 1 overpass overpass 39119 Jul 16 13:51 /srv/overpass/diffs/006/177/731.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:51 /srv/overpass/diffs/006/177/731.state.txt
-rw-r--r-- 1 overpass overpass 70601 Jul 16 13:52 /srv/overpass/diffs/006/177/732.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:52 /srv/overpass/diffs/006/177/732.state.txt
-rw-r--r-- 1 overpass overpass 65527 Jul 16 13:53 /srv/overpass/diffs/006/177/733.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:53 /srv/overpass/diffs/006/177/733.state.txt
-rw-r--r-- 1 overpass overpass 36066 Jul 16 13:54 /srv/overpass/diffs/006/177/734.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:54 /srv/overpass/diffs/006/177/734.state.txt
-rw-r--r-- 1 overpass overpass 88983 Jul 16 13:55 /srv/overpass/diffs/006/177/735.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:55 /srv/overpass/diffs/006/177/735.state.txt
-rw-r--r-- 1 overpass overpass 62543 Jul 16 13:56 /srv/overpass/diffs/006/177/736.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:56 /srv/overpass/diffs/006/177/736.state.txt
-rw-r--r-- 1 overpass overpass 81419 Jul 16 13:57 /srv/overpass/diffs/006/177/737.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:57 /srv/overpass/diffs/006/177/737.state.txt
-rw-r--r-- 1 overpass overpass 63108 Jul 16 13:58 /srv/overpass/diffs/006/177/738.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:58 /srv/overpass/diffs/006/177/738.state.txt
-rw-r--r-- 1 overpass overpass 101858 Jul 16 13:59 /srv/overpass/diffs/006/177/739.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 13:59 /srv/overpass/diffs/006/177/739.state.txt
-rw-r--r-- 1 overpass overpass 54145 Jul 16 14:00 /srv/overpass/diffs/006/177/740.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:00 /srv/overpass/diffs/006/177/740.state.txt
-rw-r--r-- 1 overpass overpass 82152 Jul 16 14:01 /srv/overpass/diffs/006/177/741.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:01 /srv/overpass/diffs/006/177/741.state.txt
-rw-r--r-- 1 overpass overpass 88365 Jul 16 14:02 /srv/overpass/diffs/006/177/742.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:02 /srv/overpass/diffs/006/177/742.state.txt
-rw-r--r-- 1 overpass overpass 115865 Jul 16 14:03 /srv/overpass/diffs/006/177/743.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:03 /srv/overpass/diffs/006/177/743.state.txt
-rw-r--r-- 1 overpass overpass 56127 Jul 16 14:04 /srv/overpass/diffs/006/177/744.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:04 /srv/overpass/diffs/006/177/744.state.txt
-rw-r--r-- 1 overpass overpass 58549 Jul 16 14:05 /srv/overpass/diffs/006/177/745.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:05 /srv/overpass/diffs/006/177/745.state.txt
-rw-r--r-- 1 overpass overpass 43692 Jul 16 14:06 /srv/overpass/diffs/006/177/746.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:06 /srv/overpass/diffs/006/177/746.state.txt
-rw-r--r-- 1 overpass overpass 64759 Jul 16 14:07 /srv/overpass/diffs/006/177/747.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:07 /srv/overpass/diffs/006/177/747.state.txt
-rw-r--r-- 1 overpass overpass 86205 Jul 16 14:08 /srv/overpass/diffs/006/177/748.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:08 /srv/overpass/diffs/006/177/748.state.txt
-rw-r--r-- 1 overpass overpass 91401 Jul 16 14:09 /srv/overpass/diffs/006/177/749.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:09 /srv/overpass/diffs/006/177/749.state.txt
-rw-r--r-- 1 overpass overpass 97956 Jul 16 14:10 /srv/overpass/diffs/006/177/750.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:10 /srv/overpass/diffs/006/177/750.state.txt
-rw-r--r-- 1 overpass overpass 124887 Jul 16 14:11 /srv/overpass/diffs/006/177/751.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:11 /srv/overpass/diffs/006/177/751.state.txt
-rw-r--r-- 1 overpass overpass 50587 Jul 16 14:12 /srv/overpass/diffs/006/177/752.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:12 /srv/overpass/diffs/006/177/752.state.txt
-rw-r--r-- 1 overpass overpass 91082 Jul 16 14:13 /srv/overpass/diffs/006/177/753.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:13 /srv/overpass/diffs/006/177/753.state.txt
-rw-r--r-- 1 overpass overpass 93600 Jul 16 14:14 /srv/overpass/diffs/006/177/754.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:14 /srv/overpass/diffs/006/177/754.state.txt
-rw-r--r-- 1 overpass overpass 48703 Jul 16 14:15 /srv/overpass/diffs/006/177/755.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:15 /srv/overpass/diffs/006/177/755.state.txt
-rw-r--r-- 1 overpass overpass 47624 Jul 16 14:16 /srv/overpass/diffs/006/177/756.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:16 /srv/overpass/diffs/006/177/756.state.txt
-rw-r--r-- 1 overpass overpass 56790 Jul 16 14:17 /srv/overpass/diffs/006/177/757.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:17 /srv/overpass/diffs/006/177/757.state.txt
-rw-r--r-- 1 overpass overpass 65867 Jul 16 14:18 /srv/overpass/diffs/006/177/758.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:18 /srv/overpass/diffs/006/177/758.state.txt
-rw-r--r-- 1 overpass overpass 55419 Jul 16 14:19 /srv/overpass/diffs/006/177/759.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:19 /srv/overpass/diffs/006/177/759.state.txt
-rw-r--r-- 1 overpass overpass 106001 Jul 16 14:21 /srv/overpass/diffs/006/177/760.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:21 /srv/overpass/diffs/006/177/760.state.txt
-rw-r--r-- 1 overpass overpass 39080 Jul 16 14:21 /srv/overpass/diffs/006/177/761.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:21 /srv/overpass/diffs/006/177/761.state.txt
-rw-r--r-- 1 overpass overpass 35843 Jul 16 14:22 /srv/overpass/diffs/006/177/762.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:22 /srv/overpass/diffs/006/177/762.state.txt
-rw-r--r-- 1 overpass overpass 74254 Jul 16 14:23 /srv/overpass/diffs/006/177/763.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:23 /srv/overpass/diffs/006/177/763.state.txt
-rw-r--r-- 1 overpass overpass 72944 Jul 16 14:24 /srv/overpass/diffs/006/177/764.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:25 /srv/overpass/diffs/006/177/764.state.txt
-rw-r--r-- 1 overpass overpass 146980 Jul 16 14:25 /srv/overpass/diffs/006/177/765.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:26 /srv/overpass/diffs/006/177/765.state.txt
-rw-r--r-- 1 overpass overpass 221544 Jul 16 14:27 /srv/overpass/diffs/006/177/766.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:27 /srv/overpass/diffs/006/177/766.state.txt
-rw-r--r-- 1 overpass overpass 93571 Jul 16 14:28 /srv/overpass/diffs/006/177/767.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:28 /srv/overpass/diffs/006/177/767.state.txt
-rw-r--r-- 1 overpass overpass 43610 Jul 16 14:29 /srv/overpass/diffs/006/177/768.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:29 /srv/overpass/diffs/006/177/768.state.txt
-rw-r--r-- 1 overpass overpass 39697 Jul 16 14:30 /srv/overpass/diffs/006/177/769.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:30 /srv/overpass/diffs/006/177/769.state.txt
-rw-r--r-- 1 overpass overpass 121911 Jul 16 14:31 /srv/overpass/diffs/006/177/770.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:31 /srv/overpass/diffs/006/177/770.state.txt
-rw-r--r-- 1 overpass overpass 36580 Jul 16 14:32 /srv/overpass/diffs/006/177/771.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:32 /srv/overpass/diffs/006/177/771.state.txt
-rw-r--r-- 1 overpass overpass 61884 Jul 16 14:33 /srv/overpass/diffs/006/177/772.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:33 /srv/overpass/diffs/006/177/772.state.txt
-rw-r--r-- 1 overpass overpass 27375 Jul 16 14:34 /srv/overpass/diffs/006/177/773.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:34 /srv/overpass/diffs/006/177/773.state.txt
-rw-r--r-- 1 overpass overpass 94619 Jul 16 14:35 /srv/overpass/diffs/006/177/774.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:35 /srv/overpass/diffs/006/177/774.state.txt
-rw-r--r-- 1 overpass overpass 107774 Jul 16 14:36 /srv/overpass/diffs/006/177/775.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:36 /srv/overpass/diffs/006/177/775.state.txt
-rw-r--r-- 1 overpass overpass 91087 Jul 16 14:37 /srv/overpass/diffs/006/177/776.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:37 /srv/overpass/diffs/006/177/776.state.txt
-rw-r--r-- 1 overpass overpass 86817 Jul 16 14:38 /srv/overpass/diffs/006/177/777.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:38 /srv/overpass/diffs/006/177/777.state.txt
-rw-r--r-- 1 overpass overpass 67872 Jul 16 14:39 /srv/overpass/diffs/006/177/778.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:39 /srv/overpass/diffs/006/177/778.state.txt
-rw-r--r-- 1 overpass overpass 121406 Jul 16 14:40 /srv/overpass/diffs/006/177/779.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:40 /srv/overpass/diffs/006/177/779.state.txt
-rw-r--r-- 1 overpass overpass 88675 Jul 16 14:41 /srv/overpass/diffs/006/177/780.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:41 /srv/overpass/diffs/006/177/780.state.txt
-rw-r--r-- 1 overpass overpass 60845 Jul 16 14:42 /srv/overpass/diffs/006/177/781.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:42 /srv/overpass/diffs/006/177/781.state.txt
-rw-r--r-- 1 overpass overpass 97991 Jul 16 14:43 /srv/overpass/diffs/006/177/782.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:43 /srv/overpass/diffs/006/177/782.state.txt
-rw-r--r-- 1 overpass overpass 40002 Jul 16 14:44 /srv/overpass/diffs/006/177/783.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:44 /srv/overpass/diffs/006/177/783.state.txt
-rw-r--r-- 1 overpass overpass 95846 Jul 16 14:45 /srv/overpass/diffs/006/177/784.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:45 /srv/overpass/diffs/006/177/784.state.txt
-rw-r--r-- 1 overpass overpass 67350 Jul 16 14:46 /srv/overpass/diffs/006/177/785.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:46 /srv/overpass/diffs/006/177/785.state.txt
-rw-r--r-- 1 overpass overpass 175219 Jul 16 14:47 /srv/overpass/diffs/006/177/786.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:47 /srv/overpass/diffs/006/177/786.state.txt
-rw-r--r-- 1 overpass overpass 368531 Jul 16 14:48 /srv/overpass/diffs/006/177/787.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:48 /srv/overpass/diffs/006/177/787.state.txt
-rw-r--r-- 1 overpass overpass 133194 Jul 16 14:49 /srv/overpass/diffs/006/177/788.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:49 /srv/overpass/diffs/006/177/788.state.txt
-rw-r--r-- 1 overpass overpass 54826 Jul 16 14:50 /srv/overpass/diffs/006/177/789.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:50 /srv/overpass/diffs/006/177/789.state.txt
-rw-r--r-- 1 overpass overpass 65641 Jul 16 14:51 /srv/overpass/diffs/006/177/790.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:51 /srv/overpass/diffs/006/177/790.state.txt
-rw-r--r-- 1 overpass overpass 44121 Jul 16 14:52 /srv/overpass/diffs/006/177/791.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:52 /srv/overpass/diffs/006/177/791.state.txt
-rw-r--r-- 1 overpass overpass 25166 Jul 16 14:53 /srv/overpass/diffs/006/177/792.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:53 /srv/overpass/diffs/006/177/792.state.txt
-rw-r--r-- 1 overpass overpass 54969 Jul 16 14:54 /srv/overpass/diffs/006/177/793.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:54 /srv/overpass/diffs/006/177/793.state.txt
-rw-r--r-- 1 overpass overpass 60863 Jul 16 14:55 /srv/overpass/diffs/006/177/794.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:55 /srv/overpass/diffs/006/177/794.state.txt
-rw-r--r-- 1 overpass overpass 121714 Jul 16 14:56 /srv/overpass/diffs/006/177/795.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:56 /srv/overpass/diffs/006/177/795.state.txt
-rw-r--r-- 1 overpass overpass 42898 Jul 16 14:57 /srv/overpass/diffs/006/177/796.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:57 /srv/overpass/diffs/006/177/796.state.txt
-rw-r--r-- 1 overpass overpass 69529 Jul 16 14:58 /srv/overpass/diffs/006/177/797.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:58 /srv/overpass/diffs/006/177/797.state.txt
-rw-r--r-- 1 overpass overpass 48241 Jul 16 14:59 /srv/overpass/diffs/006/177/798.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 14:59 /srv/overpass/diffs/006/177/798.state.txt
-rw-r--r-- 1 overpass overpass 58635 Jul 16 15:00 /srv/overpass/diffs/006/177/799.osc.gz
-rw-r--r-- 1 overpass overpass 86 Jul 16 15:00 /srv/overpass/diffs/006/177/799.state.txt
Hi @drolbr,
fetch_osc_and_apply.log just shows the regular updating:
2024-07-16 13:18:44: updating from 6177698
2024-07-16 13:18:44: updating to 6177699
2024-07-16 13:18:49: update complete 6177699
2024-07-16 13:18:49: updating from 6177699
2024-07-16 13:18:59: updating from 6177699
2024-07-16 13:19:09: updating from 6177699
2024-07-16 13:19:19: updating from 6177699
2024-07-16 13:19:29: updating from 6177699
2024-07-16 13:19:39: updating from 6177699
2024-07-16 13:19:49: updating from 6177699
2024-07-16 13:19:50: updating to 6177700
and then nothing until my restart.
database.log - the last entry about write operations is the one trying to update to 6177700, consistent with replicate_id=6177699.
2024-07-16 13:20:34 [6150] read_finished of process 30285.
2024-07-16 13:20:34 [30285] read_finished() end
2024-07-16 13:20:50 [30290] write_start() start version='2024-07-16T13\:19\:14Z'
2024-07-16 13:20:50 [6150] waited idle for 166 cycles.
2024-07-16 13:20:50 [6150] waited idle for 1 cycles.
apply_*.log doesn’t exist.
transactions.log doesn’t show anything about any update.
Software version is osm-3s_v0.7.62 I downloaded 2 weeks ago.
Hi @drolbr,
in the meantime I had the same problem twice again.
The first one was related to a short network outage the fetch_osc_and_apply script didn’t recover from.
The second one seems to be due to a file not downloaded due to a server error:
2024-09-07 22:36:50 URL:https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/minute/006/252/122.state.txt [86/86] -> "/tmp/osm-3s_update_MBrJLC/006252122.state.txt" [1]
https://osm-planet-eu-central-1.s3.dualstack.eu-central-1.amazonaws.com/planet/replication/minute/006/252/122.osc.gz:
2024-09-07 22:36:56 ERROR 500: Internal Server Error.
gzip: stdin: unexpected end of file
Check State
Apply
Reading XML file ...Parse error at line 1:
no element found
To restart the updates I had to shut down the dispatcher. If I restart the update script only, it hangs after printing Apply.
The problem in both cases seems to be a file not downloaded on the first try.
This seems to be different to the separate fetch and apply scripts - here the fetch part would just continue to loop trying to download the next diff file.