I recently started fiddling with OSM2pgsql and Osmium to really work with OSM data. I am just wondering how to work with the following: assuming I want, for instance, only nodes which are bars (amenity=bar) into my postgres database, how do I best keep this database updated (weekly, for instance)?
I have, so far, worked mainly using the following workflow:
Download planet.osm file, filter with osmium-tags-filter, import into postgres database with osm2pgsql. However, as you may have identified, I have not integrated replication into this workflow and I would like to. My question though: how can I find the diff particularly for the file that I filtered with osmium? I do not necessarily care about all the changes, of course, just changes to (in this case) nodes with amenity=bar.
Can’t you use the same filters via osmium-tags-filter on the osm change files (.osc files)? and use the filtered change file as input in your replication process?
It’s sure a bit overhead as you are filtering out a lot of information at the change file level but I would give this a try.
Can’t you use the same filters via osmium-tags-filter on the osm change files
No, that generally does not work for many reasons. For one, you will not get the information that the tag you are filtering for has been deleted. There is no way to filter change files by themselves. So don’t do that.
If you are fine with weekly updates anyway, then just redo your whole process once a week and re-import a fresh database. You can use pyosmum-up-to-date to keep a local planet file updated, so you don’t have to download the whole planet file again each time. Then do the filtering again, drop the old database and re-import from scratch.
If you need minutely updates and/or want to use larger amounts of the OSM data (not just one type of amenity but, say, all highways around the world), you’ll have to keep a complete osm2pgsql updatable database around.
Thank you, both, for the extensive responses. I think Jochen’s suggestion works best then indeed, since I am fine with sticking to weekly updates. Since pyosmium makes it easy to keep the planet file up-to-date – which was my main concern: having to do the whole download on a regular basis – I think this solution is sufficient for my use case.