Handling Road Segment Changes and Historical Data Tracking in OSM Integrations

We are integrating with a system that will send us OSM maps with its road segment data every two weeks. The total size of these OSM files is around 100GB, meaning we will receive full maps each time.

We have found a way to generate update OSM files for the storage system where these maps will be stored (PostgreSQL + PostGIS).

osmosis --read-pbf file=old_file.osm.pbf --read-pbf file=new_file.osm.pbf --write-pbf file=update_file.osm.pbf

However, the main challenge is that we need to track changes in road segment characteristics with each update. Since we plan to extract up-to-date data for specific time periods in the future, we need to somehow capture what data was changed and at what moment in time.

Has anyone already faced this issue or perhaps found existing tools that could help with this?

Hi,

Not answering your question, but to make it clear to other readers is that OSM maintains chronology for all objects from the moment they are created. The unfortunate part is, if e.g. a road is split for reasons of say different restrictions for a section, the chronology is by default inherited by the segment with the most nodes, albeit, certainly in my editor of choice you get to choose which part has to get that history.

Avoiding showing mapper id’s the right hand side of this window shows who done it, when, what tags were changed/added, deleted.

(not sure how v1 to v11 are identical for the second last tag and happened on the last, 11th edit)

There area mappers who’re seemingly agnostic about the want of OSM, a many mappers ethos, to do all to retain the chronology, and seem to prefer deleting things and remapping, turning objects into V1 again, Certainly in the JOSM editor there’s no need. Redo an object which is often much quicker than the arduously dragging nodes around, adding more, just select to object to be replaced with new and hit Shft+Ctrl+G to transfer all the tagging and history. Does of course not work when splitting ways. Only one part gets the history.

Disastrous it becomes when a certain ‘easy’ apps is used for on-the-road surveying. Update a crossing, add all the details and each detail is added in a separate change set, ending you up with V5,V6, V7. Do a road, add paving, if it’s lit, how many lanes, if oneway… all adding a version and timestamp each. At least another on-the-road concatenates the changes into a single edit and makes a description of all the tags that were updated for the object.

(thread hijack in the air, the moderatore may split this off as OT, a serious versioning bloathing to me).

@SekeRob
Thank you for your response! This is a good option for investigation to understand how the temporal change history is implemented.

Our main point is that we want to store OSM data in PostgreSQL + PostGIS and be able to query it. However, when applying an OSM file update, it simply overwrites the existing data.

I think you’ll need to explicitly append the data - see for example this page which links to some examples.