I want to have a routing server on the localhost which could server requests to give user distance between any two cities with the latitude and longitude given.
I have been trying for this about a month. I tried an open source Routing Projecthttps://github.com/DennisOSRM/Project-OSRM but to extract the whole planet out of it, it just take too too much RAM and CPU resources. It was not able to compile on a 64 GB machine even.
Then I tried YOURS with Gosmore believing that it will surely help. While I was able to find the distances between some of the datasets only in api/dev (api/1.0 wasn’t able to calculate distances much efficiently) in their official website but when I tried to deploy it on the local machine server, sometimes I get error that “Unexpected error occure in Gosmore” or sometimes “the distance between two places is > 1000 Km”.
Since I don’t want that my routing server should get limited with only 1000 Km distance, now I am looking for more better Routing engine that I could deploy on my localhost and start the service. Can somebody please help me in this case so that I can efficiently serve the distances and time of travel ( time of travel doesn’t matter much to me ) to the users without any limitations.
Gosmore doesn’t limit routing to 1000km, it’s just that it takes more time and memory and I don’t have the server resources for that, so I limited the maximum distance to 1000km
I think you’re pretty much stuck to OSRM or Gosmore. The first being much faster but requiring immense amounts of memory to generate the hierarchy and not much routing flexibility, the second being slower but much more flexible and runs on modest hardware even for routing database generation).
Yup, can you tell me how much resources and time it will take if I run gosmore on whole planet ? May be we could manage it.
You listed in YOURS Api users that even Apple uses it (or correct me if I am wrong), Is there app also limited by 1000 km thing ?
I issued the same URL to yournavigation’s api/dev and on my localhost api/dev, it seems that both are different versions or the code that you run behind the website is different than the code that is supplied because I am getting an ‘Unexpected Gosmore error’ for most of the cities when trying to access the distance between two places via api/dev ( api/1.0 seems to old to provide me good results) . So is it like that the code is different or is it some error on my side.
And also
Do you know of any other efficient routing server that could help me make it run on the whole planet or do any sort of adjustments like : I can parse the data individually for continents and then merge them or something like that.
Actually I start getting the unexpected gosmore error after I increased that distance variable to millions, before that I was getting the same ‘1000 km error’.
Is this also possible that I can get the distance through this CGI interface without using YOURS at all ? Because only distance and timeoftravel is what I need, but currently CGI interface only gives me result without distance and time. Is there any way. I am desperate for it ?
As you can see from the source code of YOURS (gosmore.php) the distance is calculated from the route nodes using the Great Circle Distance Formula, so, yes you can ‘get’ the distance using the output from Gosmore, though you have to calculate it yourself as Gosmore does not directly provide this metric. The wiki page for Gosmore states that the remaining traveling time is the 5th field in the output.
I checked the gosmore through CGI interface and they are perfect, atlast it seems that its the YOURS that is saying that unexpected gosmore error occurs. I debugged the code little with my little PHP knowledge and it seems that error occurs because in the code $node=0.
I ended up in writing my own python script to calculate the distance from the gosmore output. I kept on adding the great circle distance between two lat and lon values. (But still there is a little variation i can see from the desired output. )
btw, is my approach correct to calculate distances this way. I am having little doubt in it because I can see a difference of about 5-6 Kms for a particular query that I issued to both YOURS and my own script.
PS : YOURS should be extended from PHP support to python so that it can make use of its multithreading and do thing blazingly fast than PHP.
Well, it will depend on your Great Circle Distance Formula implementation and the chosen route (which could differ between your Gosmore installation and mine. You can see YOUR’s implementation of GDCF in gosmore.php (function: calculateDistance).
Well any webserver will do the multithreading for PHP, so I don’t see the advantage of Python here.
Just one more thing :
Can I tell gosmore to make use of more RAM and more CPU and do things fast. I mean if I host gosmore on a very powerful machine, do I have to do some additional tweaking with the code to instruct it to use more resources or it does automatically.
And sometimes I am also getting this error :
Do you know what does this mean :
“Error: Mmap of dnull for routing arrays”
thank you very very much for your useful replies so far.
Not as I know it. Gosmore has a very simple companion application for Linux that will try to keep the entire routing database in memory (if you have enough, e.g. 16+ GB) so that Gosmore doesn’t have to wait for disk I/O.
If you want to get the fastest routing using Gosmore, I recommend a CPU with the fastest single threaded performance you can afford. This means an Intel Sandy Bridge or newer architecture at the highest clock speed. If you don’t expect too many simultaneous requests then you can use a dual core, but otherwise get a high-end quadcore i5. Don’t bother with hypertheading as Gosmore won’t be able to use it unless you run at least 6 gosmore processes simultaneously but that requires a heck of a lot of ram for good performance (e.g. 48+ GB) in which case you can also start thinking about using OSRM, but then the CPU is overkill again, so…
For long distance routing with Gosmore I would choose CPU/RAM combo’s something like this:
dualcore (2 threads): 16+
quadcore (4 threads): 32+
quadcore+hyperthreading (4-8 threads): 48+
(16 GB for the routing database + N * 4 GB)
You’re welcome.
BTW, like I said earlier, Gosmore is much more flexible then OSRM: A single route database for gosmore contains up to 8 completely different routing profiles which you can tweak during runtime using stylesheets with different routing parameters, this effectively gives you many more routing profiles.
I tried to build the whole planet with gosmore into .pak file in amazon aws machine with following specification :
Large Instance 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of local instance storage, 64-bit platform
and unfortunately the process got killed after making 2 GB of gosmore.pak and process running for continously about 380 minutes. I checked the syslog, they say :
Jul 8 04:26:32 ip-10-32-11-89 kernel: [19889293.358724] Out of memory: Kill process 13587 (gosmore) score 963 or sacrifice child
Jul 8 04:26:32 ip-10-32-11-89 kernel: [19889293.358743] Killed process 13587 (gosmore) total-vm:7514580kB, anon-rss:7367896kB, file-rss:80kB
The CPU/RAM combo that you gave above was for routing process or is this hold also for this rebuild process ?
The rebuild process shouldn’t take as much RAM, 4 GB should be enough afaik. The CPU is not very important unless you want the fastest updates ofcourse…
Eh, no. I just planned a route (which looked perfectly fine) of almost 3000km, but which took over a minute to be calculated. The return route was calculated in 12 seconds, so it was a case of the routing database not being cached in memory.
I had a similar problem. The rebuild may run on 4 GB RAM but you do need a large swap file. If it runs out of swap it will give a memory error. I’ve rebuilt planet.osm on a local machine with 8 GB RAM and 16 GB swap. It ran over the weekend.
(But still there is a little variation i can see from the desired output. )
