Automated incremental Bus Stop (GTFS) updates

SafwatHalaby · 2018-06-11T16:37:22.000Z

addr:number changed to addr:housenumber in https://www.openstreetmap.org/changeset/59727988

Sanniu · 2018-06-15T10:12:37.000Z

Osmose now jumps on "suspicious tag combination
highway together with addr:* "

anonymous_gushdan_mapper · 2018-06-15T10:15:25.000Z

I don’t think the address should be saved in the bus stop node in OSM. This might cause duplicate addresses when there’s already an address node (or tag) on a nearby building.

SafwatHalaby · 2018-06-17T10:05:05.000Z

I apologize for the problematic tagging. I am not very familiar with address tagging and should have studied this further first.

What do you propose? It is possible to revert this and put “addr:housenumber” and “addr:street” in the description tag. But I would have preferred something which allows the clients like Osmand to use the addresses even in areas where no one has tagged the houses. That would be much better for usability.

SafwatHalaby · 2018-06-17T10:14:35.000Z

Is it OK to keep “addr:street” and put the housenumber in the description?

Osmose now jumps on "suspicious tag combination
highway together with addr:* "

Isn’t Osmose wrong here? (Assuming we keep addr:street only)

SafwatHalaby · 2018-06-20T12:11:58.000Z

Since address duplication is dangerous and may cause unknown behavior in client address lookup, I have decided to move addr:housenumber to gtfs:addr:housenumber in the meantime, even before the discussion is finished. Please feel free to post your opinions on what should be done with addr:housenumber and addr:street.

Changesets: #60009323, #60010547

SafwatHalaby · 2018-08-18T14:02:10.000Z

I’m investigating ways to increase the “bus factor”. Currently, when fetching new MOT GTFS files, the script needs to compare them with the files fetched the previous run. This means that if I lose both my PC and my PC backup, or if I ever get hit by a bus, running the script properly would be a bit fiddly, because one would have to reconstruct the lost file.

I would like to make the script completely stateless locally. This should be technically possible: Fetch latest bus stop changeset by SafwatHalaby_bot, and use that as the “old gtfs file”. This would make the script solely dependent on OSM servers, and not on any local hard drive.

This might be a major overwrite, so while we’re at it, I would like to rewrite the script such that it does not depend on JOSM. This should make running it headless natural and pave the way for complete automation.

dsh4 · 2018-08-23T17:54:32.000Z

Replying to an old point:

SafwatHalaby:

What would be the suitable OSM equiavlan of “רציף”? E.g.

According to https://wiki.openstreetmap.org/wiki/Tag:public_transport%3Dplatform , ref= should be used for that. I’m not sure whether that tag is supposed to go on the highway=bus_stop node or on something else.

Glad to hear of the planned bus factor improvements.

SafwatHalaby · 2018-09-20T12:23:09.000Z

There is a semantics issue here. The thing called “platform” in the Wiki is essentially a bus stop, we already use the “ref” tag for bus stop numbers.

The thing called a “ratzeef/platform” in the MOT GTFS is better translated into a “terminal number”, available only for some bus stops, mainly in central bus stations. So far I couldn’t find a Wiki entry which discusses it.

anonymous_gushdan_mapper · 2018-09-23T12:47:17.000Z

I don’t think you can call it “terminal number”, the רציף/platform in the GTFS is a platform just like a train station platform.

A bus terminal is what we call מסוף אוטובוסים in Hebrew. The Tel Aviv Central Bus Station or Arlosoroff Terminal are bus terminals. They don’t have numbers.

The “platform” in the GTFS file is there so that transit navigation apps will point users to the right platform for their bus. The GTFS file does not have the accurate coordinates for each platforms, so it uses the same coordinates for all and the same name for all, for example “Tel Aviv Central Bus Station 7th Floor/Platforms” ( https://www.openstreetmap.org/node/1803094757 ) is one node in OSM and one stop_code in the GTFS, but it has many lines in stops.txt with different stop_id for each, and the difference is just the stop_id and the platform number.

Considering this, I don’t see how keeping the “platform number” is useful to OSM in any way, and I suggest you ignore this in your automated imports, and only have one OSM node per GTFS stop_code.

SafwatHalaby · 2018-11-24T17:10:34.000Z

anonymous_gushdan_mapper:

Considering this, I don’t see how keeping the “platform number” is useful to OSM in any way, and I suggest you ignore this in your automated imports, and only have one OSM node per GTFS stop_code.

Agreed. The “platform number” is currently ignored.

The deletion of stops is slowly making the OSM dataset deviate from the MOT dataset. I propose preventing users from deleting stops. Non existing stops would have to be reported upstream to MOT, just like renaming. Comments?

As a reminder, here are the current conflict handling rules: https://wiki.openstreetmap.org/wiki/User:SafwatHalaby/scripts/gtfs#Information_for_mappers

I am essentially proposing replacing “You may delete a bus stop” with “you may not delete a bus stop directly. Contact MOT instead”.

SafwatHalaby · 2018-11-24T17:20:05.000Z

I just completed a bus stop update in #64848970.

This is a normal update, except that it’s bigger than usual, because the last update was 4 months ago.

Click here to see the full history, or here for more info regarding bus stop updates.

Sanniu · 2018-11-24T20:52:56.000Z

SafwatHalaby:

I am essentially proposing replacing “You may delete a bus stop” with “you may not delete a bus stop directly. Contact MOT instead”.

“Contact MOT” is not enough - the exact web address/email should be added, otherwise this will be useless…

Reuven_Kost · 2018-11-27T17:26:08.000Z

@SafwatHalaby Great job You do! Thank you
I believe the last changes are good improvements, relaying on MOT/GFTS data is more clever, although it has its own mistakes.

What about adding the bus numbers (Mispar Ha Kav) as a new tag to each bus stop?

It will give useful information for passengers and trip planners
The data is available as route_short_name in routes.txt in GFTS database.

dsh4 · 2018-11-30T18:42:33.000Z

@Reuven Bus numbers shouldn’t be mapped as new tags on the bus stops but as new relations of which the bus stops are members; see the schema: https://wiki.openstreetmap.org/wiki/Buses#Bus_routes

Your suggestion makes sense; in fact, I made a similar suggestion in this very thread some time ago; see discussion starting at https://forum.openstreetmap.org/viewtopic.php?pid=682596#p682596

SafwatHalaby · 2018-12-07T07:22:08.000Z

To summarize the main objection raised by some of the mappers:

The MOT route data cannot be relied upon, so an import would include the stops a bus goes through, but not the exact route or the street names.

The route=bus relation mandates routes, so an import must invent a new relation and not use route=bus.

The import needlessly bloats the map and complicates the import/conflict resolution code without adding much practical value to OSM, “because importing the bus stop data without the roads and routes / because importing the data in any form” isn’t of much use to OSM.

Sanniu:

“Contact MOT” is not enough - the exact web address/email should be added, otherwise this will be useless…

Of course! The exact address can already be found at the page, and if the deletion change is ever added it would also link to the address.

SafwatHalaby · 2018-12-07T07:30:59.000Z

SafwatHalaby:

Since address duplication is dangerous and may cause unknown behavior in client address lookup, I have decided to move addr:housenumber to gtfs:addr:housenumber in the meantime, even before the discussion is finished. Please feel free to post your opinions on what should be done with addr:housenumber and addr:street.

Changesets: #60009323, #60010547

https://forum.openstreetmap.org/viewtopic.php?pid=704210#p704210

Pinging this old issue. Suggestions appreciated.

Sanniu · 2019-02-25T08:29:53.000Z

Hi SafwatHalaby, are you going to run your great bot? I really miss bus-stop updates last couple of months…

SafwatHalaby · 2019-03-02T18:11:23.000Z

Hello! Sorry for the infrequent updates. I had been unable to perform the runs weekly in the past few months, but starting next week the regular updates should return.

With that said, I would like to re-write the code ASAP. (It will likely be re-written in Java). The re-write should make it cleaner, allow full automation, and make it locally stateless. I aim for minimal dependencies (preferably only the standard Java library) so that it can run for a very long time without maintenance.

SafwatHalaby · 2020-09-28T09:54:42.000Z

Script has been executed.

Changesets:
https://www.openstreetmap.org/changeset/91500256
https://www.openstreetmap.org/changeset/91502029