I am interested in experimenting with Nominatim by creating fictitious OSM objects / addresses / places / boundaries / relations under different sets of conditions and then seeing how Nominatim search results vary under those differing experimental conditions.
I understand that uploading experimental objects which don’t exist in the real world into OSM database (even if done temporarily for a few minutes) constitutes as “bad editing” similar to vandalism.
Is there a method where I may conduct this little Nominatim experiment without corrupting OSM data? Another fellow mapper pointed me to master.apis.dev.openstreetmap.org, but this site doesn’t seem to be working properly - my uploaded edits don’t appear to be captured in its database.
I’ve just tried adding something to the dev server and it looks like it updated OK: https://master.apis.dev.openstreetmap.org/way/4306340585 . Nothing renders data from there, and I don’t think there’s a Nominatim instance pointing at it either, so I don’t think that it’d be useful for your purposes. Instead you might need to:
Your proposed method appears to be the most conscientious solution - I get to conduct my experiment exactly the way I want it done without infringing upon the consensual norms of the OSM community.
My worry is I’m not technologically competent enough to successfully install a working local instance of nominatim on my pc and getting it to work seamlessly with local data, but I will give it a go anyway.
In the meantime, is there any other known site that runs on OSM engine which allows uploading of fictitious data which includes the nominatim search engine as well?
I have used the “sudo apt install” command to obtain the necessary prerequisite libraries (Libosmium, Protozero, etc…). I then installed Osmium Tools using “sudo apt install osmium-tool” command. I am now stuck at the instructions to “build Osmium using CMake”. The instructions say “cd osmium-tool” but I can’t seem to find the osmium-tool folder.
You can now also simply add your test data to the existing installation:
create the new data in JSOM and save it as an osm file (e.g. extra.osm)
use osmium-tool to get rid of the negative ids for new objects: osmium renumber -s 10000000,10000000,1000000 -o extra_renumbered.osm extra.osm
load data into your local Nominatim installation: nominatim add-data --file extra_renumbered.osm
and integrate it into the search index: nominatim index
If the data is recent enough, you can even use this approach to change data. Download the area in question in JOSM, change some of the data (do not add new data, the negative ids will break the import) and save again to a local file. Don’t renumber but import the resulting file directly into Nominatim. This only works as long as you have relatively recent data in your Nominatim database because JSOM saves all the data it gets from the OSM database. Therefore you do not only get the changes you made in JOSM but also all the changes made in OSM in the meantime but only in the area you downloaded. So there might be conflicts between old and new data that lead to odd results from search.
None. Simply load the file that you have saved with JOSM directly into Nominatim.
BTW you can download single objects in JOSM (File->Download Object). If you do that, make the changes you want and save to a file, you should be able to make very precise changes. Much better than when downloading an entire area.
Guys, tq so much for your step-by-step guidance. I am feeling very blessed.
I’m now trying to conduct my experiments, and I hope you don’t mind guiding me step-by-step again…
I have downloaded maldives-latest.osm.pbf from Geofabrik and imported this database into nominatim using the command: nominatim import --osm-file maldives-latest.osm.pbf 2>&1 | tee setup.log
I opened a new blank layer in JOSM and created a fictitious country (country XYZ), divided it into 4 regions (N, S, E, W regions), populated these regions with towns and cities (some as nodes and some as enclosed ways), and placed a amenity=fast_food test object (named “KFC-exprmt”) in one of the towns; and saved this database as experiment.osm
I merge this experimental database with the Maldives database using the commands: osmium renumber -s 10000000,10000000,1000000 -o experiment_renumbered.osm experiment.osm , then: nominatim add-data --file experiment_renumbered.osm , then: nominatim index
I now queried the database with the command nominatim search --query "KFC-exprmt" and get the expected result: `“KFC-exprmt, Town A, N region, country XYZ”
I now want to move this “KFC-exprmt” object to a new location and repeat the nominatim query - this process will be repeated again and again and again as I record down the different search results under different conditions. Question #1: After I move object “KFC-exprmt” to its new location in JOSM, I save and overwrite the file experiment.osm - and then I do what? (Because the imported db in Nominatim is maldives-latest.osm.pbf). Is there an efficient workflow where I can just move->save->query->repeat?
Question #2: I have downloaded Nominatim-UI, and I’m trying to get it to work with my experimental data. I pointed the Nominatim_Config.Nominatim_API_Endpoint to the installation folder and I get the message “Error parsing JSON data” (see image below). What am I doing wrong here?
change/move the data as required (do not add new data!)
save to file again
import the file with changed data directly into Nominatim( add-data + index), this will update your existing data in the database
Question #2: I have downloaded Nominatim-UI, and I’m trying to get it to work with my experimental data. I pointed the Nominatim_Config.Nominatim_API_Endpoint to the installation folder and I get the message “Error parsing JSON data” (see image below). What am I doing wrong here?
Nominatim-UI needs a webserver to work with. Set Nominatim_Config.Nominatim_API_Endpoint = http://127.0.0.1:8088. Then run nominatim serve to run the built-in test server.
Please refer to the Nominatim UI section of the docs how to run the UI. You need to have two webservices running in the end: one for the API and one for the UI.
As mentioned before: the problem with new data in JOSM is that the objects get negative IDs. Nominatim cannot handle those. That is what the renumber is for. It changes the IDs to positive ones. However, we would not want to have existing objects renumbered because then they are not the same object anymore. That is why you must not mix new and existing data in one file.
That means if you want more new data, simply create another file with new data in JOSM and renumber once again. But (and this is important!) this time using different start values (the values given with parameter -s). Otherwise the file ends up with the same object IDs and the existing fake data will be overwritten in unpredictable ways.
Looks like the tools disagree on file contents. It should also work to change the original file and then run renumber with exactly the same start values -s. In theory the renumbered IDs should be deterministic. I have not tested if that works in practice.
