I would like to get/poll periodically the latest OSM Notes modifications (creation, comments, close, reopen) for the whole world. What are the possibilities?
I only see the notes/search option from the API where I can specify the “from” parameter.
Is there any other option? Websub (PubSubHubbub). Something for real time but without impacting the API with periodic requests? Or even with a Messaging System (IRC, Matrix, Telegram)?
https://resultmaps.neis-one.org/osm-notes has RSS feed for each country. You set up in your application, or service.
I want to have other kinds of filtering to do analysis on notes. That’s the reason I want something directly from OSM.
Also, the precision of the country boundaries in that page is not very accurate: https://twitter.com/angoca/status/1475298343203090433
I am using a standard rss feed reader with a manual URL construction from a bounding box like this:
http://api.openstreetmap.org/api/0.6/notes/feed?bbox=8.1706155,51.8144663,8.5269845,51.9737924
A feature missing for me in Notes would be “subscribe”. Without otherwise interacting with a Note i would like to subscribe, to get future updates. Same for Changesets. Sometimes changeset comments go back and forth and i’d like to be updated on comments.
Commenting a “subscribe” to a Note/Changeset is a workaround.
Flo
Nope. You can simply hit the “subscribe” button on changesets even if they’re not yet closed, and you will receive a notification for every comment.
If once-a-day updates are good enough for your case, you can periodically grab https://planet.openstreetmap.org/notes/planet-notes-latest.osn.bz2 (~230 MB) and parse that XML to extract whatever information you need (filtering by comment timestamp, and any other rules you might have, should be trivial).
Additionally to that maximum customizeability, it does not create load on other servers, you do not depend on their acceptable usage policy (or speed / availability), etc.
In fact, I would like something like “real time”. To be notified once a change in a note is done.
Parsing the planet is what I have already done, but this process is very slow (processing the whole XML takes a time) and this daily periodicity is not what I am looking for.
(Pull notification) I can create a RSS for the whole world, and polling it each minute, but this impact the API, because many of the feeds will not contain any new value. This operation is started form the requestor. My view is that if everyone is polling the API, the server has to respond these requests. I want something real time without huge impact, or unnecessary CPU cycles. I would like something different that: “Hey API, is there anything new in this area?”
(Push notification) I would like to “receive” the notification from the API, once I have subscribe to it. To receive from the server when there is something new. This operation will be started from the server.
Probably, the current API status does not provide subscription like this, and I will have to use the API by polling it periodically. It could eventually be part of the API 0.7.
I think we’ve talked about this already. The API 0.6 notes search endpoint provides start and end timestamps. Why can’t you use those? If there are too many notes, simply make the time interval smaller, and query again.
I know that the wiki pages states that the “q=” paramter is mandatory. I don’t find this in the source code (openstreetmap-website/app/controllers/api/notes_controller.rb at master · openstreetmap/openstreetmap-website · GitHub), and the endpoint also works without it: https://api.openstreetmap.org/api/0.6/notes/search
I’d say, this is very very unlikely to happen for a number of reasons.
The API 0.6 notes search endpoint provides start and end timestamps
True, e.g. this seems to successfully return 20-minute range of Note updates earlier today:
https://api.openstreetmap.org/api/0.6/notes/search?from=2022-10-26T13:20:00Z&to=2022-10-26T13:40:00Z
One should take care of following API Usage policy which states (among other things):
The editing API is provided in order to edit the map data, not for read-only purposes or projects
However other AUP suggestions (like using dumps + diffs) in AUP do not work for Notes if one need better than 24-hour granularity (as seems to be the case here), because I don’t think we have minutely diffs for Notes (only for map data).
Maybe if one calls it reasonably rare (e.g. once every 5 minutes instead of every second) it would be often enough for @AngocA usecase, and one might ask ops if that would be considered acceptable usage? (due to alternatives being non-existent)
I’m not sure if it’s exactly what your looking for, but in my opinion ENT8R’s NotesReview is pretty good for reviewing notes, regardless of if they were recently opened or not. I think you can search for notes by specific users to.
It is a nice tool. However, unless I’m very wrong, it is simply calling API 0.6 mentioned above and displaying it on the map. So, it has all the limitations of the API (usage policy, max 100 notes, etc)
Oh, bummer. I wasn’t aware that’s what it was doing. Your probably right though 
You can set the Notes API limit to 10000 in every call, but I doubt that’s gonna solve OP’s request.
Yesterday I checked discussions-latest (5.4 GB, DON’T DOWNLOAD UNLESS YOU WANT TO), but there are no Notes in there as well, only changeset discussions.
So … I guess without asking someone like @Firefishy if a downloadable XML of all Notes (and a nightly Diff or something) could be provided on planet.openstreetmap.org (which I would be more than happy to see!), this might not be solvable. Removed for reasons of stupidity. Thanks for pointing out the obvious @mmd.
K