Removing all but highways with osmosis

alswearengen · 2011-06-04T12:43:28.000Z

Hi,

I’m currently trying to remove all elements except highways (relations, buildings, railways, rivers, etc.) from a pbf file of size germany. I want to do that with osmosis 0.39. But it fails even for a single task like removing all relations. I’m doing this with this command:

./osmosis --read-pbf file="file1" --tf reject-relations idTrackerType=BitSet --un --write-pbf file="file2"

edit: I forgot the output of osmosis :


SCHWERWIEGEND: Thread for task 1-read-pbf failed
java.lang.OutOfMemoryError: Java heap space
    at java.util.ArrayList.<init>(ArrayList.java:112)
    at org.openstreetmap.osmosis.core.filter.common.ListIdTracker.ensureListIsSorted(ListIdTracker.java:87)
    at org.openstreetmap.osmosis.core.filter.common.ListIdTracker.get(ListIdTracker.java:150)
    at org.openstreetmap.osmosis.tagfilter.v0_6.UsedNodeFilter.complete(UsedNodeFilter.java:133)
    at org.openstreetmap.osmosis.tagfilter.v0_6.TagFilter.complete(TagFilter.java:142)
    at crosby.binary.osmosis.OsmosisBinaryParser.complete(OsmosisBinaryParser.java:34)
    at crosby.binary.file.BlockInputStream.process(BlockInputStream.java:37)
    at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:37)
    at java.lang.Thread.run(Thread.java:662)
04.06.2011 14:25:56 org.openstreetmap.osmosis.core.Osmosis main
SCHWERWIEGEND: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
    at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
    at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
    at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
    at org.codehaus.classworlds.Launcher.main(Launcher.java:31)

I also tried to set

export JAVACMD_OPTIONS=-Xmx1024M

but it seems not to be working or enough memory.

I learned that “–un” is the problem here but I need the not used nodes to be removed.

Any hints?

Thanks,
Al

Marqqs · 2011-06-04T16:29:24.000Z

Hello & welcome!

Unfortunately I cannot help you with Osmosis but there are other users who certainly can. I just lack experience with Osmosis, sorry.

In case you do not succeed using Osmosis for this task, there is an alternative. But - and that’s a real disadvantage - you cannot write PBF as output. Nevertheless, if you want to try, here is an example:

o5mfilter bremen.osm --keep"highway=" >streets.osm

You can save a lot of time for filtering if you take .o5m for input instead of .osm format:

osmconvert bremen.osm.pbf --out-o5m >bremen.o5m
o5mfilter bremen.o5m --keep"highway=" >streets.osm

If you want to get just major streets, try this:

o5mfilter bremen.o5m --keep"highway=primary =secondary" >streets.osm

If you need just names and nothing else, take this command:

o5mfilter bremen.o5m --keep"highway=primary =secondary" --keep-tags="highway= name=" >streets.osm

and so on…

alswearengen · 2011-06-06T10:01:57.000Z

That works just fine, thank you.

I have one more problem now. My source files contain nodes from clipped ways so they don’t belong to anything useful. I know they result from my bounding-box operations (“completeWays=no”).

I would like to remove them as well. My approach with osmosis should handle this with the --un parameter. Anyone has a hint to just remove such nodes from an osm-file?

Marqqs · 2011-06-06T14:41:57.000Z

alswearengen:

That works just fine, thank you.

I have one more problem now. My source files contain nodes from clipped ways so they don’t belong to anything useful. I know they result from my bounding-box operations (“completeWays=no”).

I would like to remove them as well. My approach with osmosis should handle this with the --un parameter. Anyone has a hint to just remove such nodes from an osm-file?

I’m not sure I got that right. There are nodes from outside of your original bounding box? You could exclude these nodes from the file with Osmosis (–bb option). Or you could use osmconvert for this purpose. For example:

osmconvert europe.osm.pbf -b=10,50,11,51 --drop-brokenrefs --out-o5m >my_region.o5m

alswearengen · 2011-06-06T14:54:36.000Z

I’m doing a bounding-box command with osmosis with

completeWays=no

The ways with nodes inside and outside of the bbox are dropped but the nodes of these ways that are inside the bbox stay. These are the nodes I want to remove.

I hope this is understandable

Marqqs · 2011-06-06T16:05:08.000Z

alswearengen:

I’m doing a bounding-box command with osmosis with
completeWays=no
The ways with nodes inside and outside of the bbox are dropped but the nodes of these ways that are inside the bbox stay. These are the nodes I want to remove.

I hope this is understandable

Ok, I think I got you now. Unfortunately, I don’t know haw to do this with Osmosis. Maybe anyone else could help?

Why do you want to exclude these ways completely? Wouldn’t it be better to accept the ways and remove every node of them which lies outside the bounding box?