Thanks to your tips, I got the following workflow
1st, I fetch changes from the planet OSM and apply them on the local *.osm.pbf, clip data by bounding polygon to filter out unnecessary data and save updated extract in "$data_file.new" file.
osmosis \
--read-replication-interval workingDirectory=$DATA_DIR/replication_in \
--simplify-change \
--read-pbf file="$data_file" --b \
--apply-change \
--bounding-polygon file="$poly_file" cascadingRelations=yes clipIncompleteEntities=true \
--write-pbf file="$data_file.new"
2nd, I compare two files, old and new, deriving changes and writing them in API DB
osmosis \
--read-pbf file="$data_file.new" \
--read-pbf file="$data_file" \
--derive-change \
--write-apidb-change host="$POSTGRES_HOST" database="$POSTGRES_DB" user="$POSTGRES_USER" password="$POSTGRES_PASSWORD" validateSchemaVersion=no
3rd replace old pbf with new one and repeat the cycle.
*Is it possible to combine steps 1 and 2 into one run?
Also, I ran into a few hiccups that I hope the community can help resolve.
- when omitting
--simplify-changeoption for fetching diffs and applying them,osmosisthrows error that several versions of one feature appear in the change stream. However, I would like to have all versions and not just the latest version from the diff for the objects in the local DB. - This may be a side effect of the previous issue, but for objects being removed, the
visible=falseattribute is written to the previous version of the object, and the latest version with removal is skipped. However, I would like to have all the versions without gaps.
Here is an example.
Version 1 and 2 is a feature creation and modification
Then, version 3 is a feature removal, but it was skipped, instead version 2 got visible=false value
The feature was restored (ver 4). You may see that version 3 is missing.
And final deletion of the feature (ver 5) is missing as well as ver 3
I would appreciate any advice to help resolve these issues.