[osmosis-dev] Osmosis update failed: Pipeline entities are not sorted or contain multiple versions of a single entity

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[osmosis-dev] Osmosis update failed: Pipeline entities are not sorted or contain multiple versions of a single entity

David Niklas
Hello,

I'm getting an out of order error from osmosis and I'm not
certain what to do about it.
I got the update from an offical mirror, so osmosis should
not have any problems with it.
I am using the latest release of osmosis as far as I know.
I have looked at the xml change file and there are 2 entities with an
identical value for the "id" attribute, but different values "version"
attribute.

Thanks!


% nice -n15 lbzip2 -kdc planet-160718.osm.bz2 | nice
-n5 /home/me/Documents/programs/osmosis-bin/bin/osmosis --read-xml-change
file=406.osc.gz --buffer-change bufferCapacity=12000 --fast-read-xml
file=/dev/stdin compressionMethod=none --buffer bufferCapacity=12000
--apply-change --buffer bufferCapacity=12000 --write-xml file=/dev/stdout
| pv -arb | nice -n15 lbzip2 -c9z > planet.osm.bz2 Dec 26, 2017 10:31:47
AM org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.45
Dec 26, 2017 10:31:47 AM org.openstreetmap.osmosis.core.Osmosis run INFO:
Preparing pipeline. Dec 26, 2017 10:31:47 AM
org.openstreetmap.osmosis.core.Osmosis run INFO: Launching pipeline
execution. Dec 26, 2017 10:31:48 AM
org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline executing,
waiting for completion. Dec 26, 2017 10:35:06 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion SEVERE: Thread for task 1-read-xml-change failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
has occurred, aborting. at
org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:162)
at
org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:218)
at
org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:305)
at
org.openstreetmap.osmosis.core.buffer.v0_6.ChangeBuffer.process(ChangeBuffer.java:48)
at
org.openstreetmap.osmosis.xml.v0_6.impl.ChangeSourceElementProcessor$ChangeSinkAdapter.process(ChangeSourceElementProcessor.java:144)
at
org.openstreetmap.osmosis.xml.v0_6.impl.NodeElementProcessor.end(NodeElementProcessor.java:139)
at
org.openstreetmap.osmosis.xml.v0_6.impl.OsmChangeHandler.endElement(OsmChangeHandler.java:94)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at
org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown
Source) at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown
Source) at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source) at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at
org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at
org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) at
javax.xml.parsers.SAXParser.parse(SAXParser.java:195) at
org.openstreetmap.osmosis.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:90)
at java.lang.Thread.run(Thread.java:748)

Dec 26, 2017 10:35:06 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion SEVERE: Thread for task 2-buffer-change failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Pipeline entities
are not sorted or contain multiple versions of a single entity, previous
entity type=Node, id=7040756, version=5 current entity type=Node,
id=7040756, version=6. at
org.openstreetmap.osmosis.core.sort.v0_6.SortedDeltaChangePipeValidator.process(SortedDeltaChangePipeValidator.java:58)
at
org.openstreetmap.osmosis.core.buffer.v0_6.ChangeBuffer.run(ChangeBuffer.java:84)
at java.lang.Thread.run(Thread.java:748)

Dec 26, 2017 10:35:06 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion SEVERE: Thread for task 3-fast-read-xml failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to read
XML file /dev/stdin. at
org.openstreetmap.osmosis.xml.v0_6.FastXmlReader.run(FastXmlReader.java:102)
at java.lang.Thread.run(Thread.java:748) Caused by:
org.openstreetmap.osmosis.core.OsmosisRuntimeException:
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
has occurred, aborting. at
org.openstreetmap.osmosis.xml.v0_6.impl.FastXmlParser.readOsm(FastXmlParser.java:436)
at
org.openstreetmap.osmosis.xml.v0_6.FastXmlReader.run(FastXmlReader.java:97) ...
1 more Caused by: org.openstreetmap.osmosis.core.OsmosisRuntimeException:
An output error has occurred, aborting. at
org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:162)
at
org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:218)
at
org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:305)
at
org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.process(EntityBuffer.java:48)
at
org.openstreetmap.osmosis.xml.v0_6.impl.FastXmlParser.readOsm(FastXmlParser.java:422) ...
2 more

Dec 26, 2017 10:35:06 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion SEVERE: Thread for task 4-buffer failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
has occurred, aborting. at
org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:162)
at
org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:218)
at
org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:305)
at
org.openstreetmap.osmosis.set.v0_6.impl.DataPostboxSink.process(DataPostboxSink.java:45)
at
org.openstreetmap.osmosis.core.sort.v0_6.SortedEntityPipeValidator.process(SortedEntityPipeValidator.java:68)
at
org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.run(EntityBuffer.java:84)
at java.lang.Thread.run(Thread.java:748)

Dec 26, 2017 10:35:06 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion SEVERE: Thread for task 5-apply-change failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An input error
has occurred, aborting. at
org.openstreetmap.osmosis.core.store.DataPostbox.checkForInputErrors(DataPostbox.java:174)
at
org.openstreetmap.osmosis.core.store.DataPostbox.consumeCentralQueue(DataPostbox.java:239)
at
org.openstreetmap.osmosis.core.store.DataPostbox.hasNext(DataPostbox.java:441)
at
org.openstreetmap.osmosis.set.v0_6.ChangeApplier.run(ChangeApplier.java:186)
at java.lang.Thread.run(Thread.java:748)

Dec 26, 2017 10:35:06 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion SEVERE: Thread for task 6-buffer failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An input error
has occurred, aborting. at
org.openstreetmap.osmosis.core.store.DataPostbox.checkForInputErrors(DataPostbox.java:174)
at
org.openstreetmap.osmosis.core.store.DataPostbox.consumeCentralQueue(DataPostbox.java:239)
at
org.openstreetmap.osmosis.core.store.DataPostbox.hasNext(DataPostbox.java:441)
at
org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.run(EntityBuffer.java:83)
at java.lang.Thread.run(Thread.java:748)

Dec 26, 2017 10:35:06 AM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks
failed. at
org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92) at
org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
at org.codehaus.classworlds.Launcher.main(Launcher.java:47)

68.1MiB [ 348KiB/s] [ 348KiB/s]
%

_______________________________________________
osmosis-dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/osmosis-dev
Reply | Threaded
Open this post in threaded view
|

Re: [osmosis-dev] Osmosis update failed: Pipeline entities are not sorted or contain multiple versions of a single entity

Sarah Hoffmann
Hi,

On Wed, Dec 27, 2017 at 08:42:45PM -0500, [hidden email] wrote:

> Hello,
>
> I'm getting an out of order error from osmosis and I'm not
> certain what to do about it.
> I got the update from an offical mirror, so osmosis should
> not have any problems with it.
> I am using the latest release of osmosis as far as I know.
> I have looked at the xml change file and there are 2 entities with an
> identical value for the "id" attribute, but different values "version"
> attribute.

You need to filter those multiple versions before applying a
change to an osm file. Try --simplify-change on the change file[1].

Kind regards

Sarah

[1] http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--simplify-change_.28--simc.29

>
> Thanks!
>
>
> % nice -n15 lbzip2 -kdc planet-160718.osm.bz2 | nice
> -n5 /home/me/Documents/programs/osmosis-bin/bin/osmosis --read-xml-change
> file=406.osc.gz --buffer-change bufferCapacity=12000 --fast-read-xml
> file=/dev/stdin compressionMethod=none --buffer bufferCapacity=12000
> --apply-change --buffer bufferCapacity=12000 --write-xml file=/dev/stdout
> | pv -arb | nice -n15 lbzip2 -c9z > planet.osm.bz2 Dec 26, 2017 10:31:47
> AM org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.45
> Dec 26, 2017 10:31:47 AM org.openstreetmap.osmosis.core.Osmosis run INFO:
> Preparing pipeline. Dec 26, 2017 10:31:47 AM
> org.openstreetmap.osmosis.core.Osmosis run INFO: Launching pipeline
> execution. Dec 26, 2017 10:31:48 AM
> org.openstreetmap.osmosis.core.Osmosis run INFO: Pipeline executing,
> waiting for completion. Dec 26, 2017 10:35:06 AM
> org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
> waitForCompletion SEVERE: Thread for task 1-read-xml-change failed
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
> has occurred, aborting. at
> org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:162)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:218)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:305)
> at
> org.openstreetmap.osmosis.core.buffer.v0_6.ChangeBuffer.process(ChangeBuffer.java:48)
> at
> org.openstreetmap.osmosis.xml.v0_6.impl.ChangeSourceElementProcessor$ChangeSinkAdapter.process(ChangeSourceElementProcessor.java:144)
> at
> org.openstreetmap.osmosis.xml.v0_6.impl.NodeElementProcessor.end(NodeElementProcessor.java:139)
> at
> org.openstreetmap.osmosis.xml.v0_6.impl.OsmChangeHandler.endElement(OsmChangeHandler.java:94)
> at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
> at
> org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown
> Source) at
> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown
> Source) at
> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
> Source) at
> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
> Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
> Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
> Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at
> org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at
> org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
> at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) at
> javax.xml.parsers.SAXParser.parse(SAXParser.java:195) at
> org.openstreetmap.osmosis.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:90)
> at java.lang.Thread.run(Thread.java:748)
>
> Dec 26, 2017 10:35:06 AM
> org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
> waitForCompletion SEVERE: Thread for task 2-buffer-change failed
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: Pipeline entities
> are not sorted or contain multiple versions of a single entity, previous
> entity type=Node, id=7040756, version=5 current entity type=Node,
> id=7040756, version=6. at
> org.openstreetmap.osmosis.core.sort.v0_6.SortedDeltaChangePipeValidator.process(SortedDeltaChangePipeValidator.java:58)
> at
> org.openstreetmap.osmosis.core.buffer.v0_6.ChangeBuffer.run(ChangeBuffer.java:84)
> at java.lang.Thread.run(Thread.java:748)
>
> Dec 26, 2017 10:35:06 AM
> org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
> waitForCompletion SEVERE: Thread for task 3-fast-read-xml failed
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to read
> XML file /dev/stdin. at
> org.openstreetmap.osmosis.xml.v0_6.FastXmlReader.run(FastXmlReader.java:102)
> at java.lang.Thread.run(Thread.java:748) Caused by:
> org.openstreetmap.osmosis.core.OsmosisRuntimeException:
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
> has occurred, aborting. at
> org.openstreetmap.osmosis.xml.v0_6.impl.FastXmlParser.readOsm(FastXmlParser.java:436)
> at
> org.openstreetmap.osmosis.xml.v0_6.FastXmlReader.run(FastXmlReader.java:97) ...
> 1 more Caused by: org.openstreetmap.osmosis.core.OsmosisRuntimeException:
> An output error has occurred, aborting. at
> org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:162)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:218)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:305)
> at
> org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.process(EntityBuffer.java:48)
> at
> org.openstreetmap.osmosis.xml.v0_6.impl.FastXmlParser.readOsm(FastXmlParser.java:422) ...
> 2 more
>
> Dec 26, 2017 10:35:06 AM
> org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
> waitForCompletion SEVERE: Thread for task 4-buffer failed
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error
> has occurred, aborting. at
> org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:162)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:218)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:305)
> at
> org.openstreetmap.osmosis.set.v0_6.impl.DataPostboxSink.process(DataPostboxSink.java:45)
> at
> org.openstreetmap.osmosis.core.sort.v0_6.SortedEntityPipeValidator.process(SortedEntityPipeValidator.java:68)
> at
> org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.run(EntityBuffer.java:84)
> at java.lang.Thread.run(Thread.java:748)
>
> Dec 26, 2017 10:35:06 AM
> org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
> waitForCompletion SEVERE: Thread for task 5-apply-change failed
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: An input error
> has occurred, aborting. at
> org.openstreetmap.osmosis.core.store.DataPostbox.checkForInputErrors(DataPostbox.java:174)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.consumeCentralQueue(DataPostbox.java:239)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.hasNext(DataPostbox.java:441)
> at
> org.openstreetmap.osmosis.set.v0_6.ChangeApplier.run(ChangeApplier.java:186)
> at java.lang.Thread.run(Thread.java:748)
>
> Dec 26, 2017 10:35:06 AM
> org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
> waitForCompletion SEVERE: Thread for task 6-buffer failed
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: An input error
> has occurred, aborting. at
> org.openstreetmap.osmosis.core.store.DataPostbox.checkForInputErrors(DataPostbox.java:174)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.consumeCentralQueue(DataPostbox.java:239)
> at
> org.openstreetmap.osmosis.core.store.DataPostbox.hasNext(DataPostbox.java:441)
> at
> org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.run(EntityBuffer.java:83)
> at java.lang.Thread.run(Thread.java:748)
>
> Dec 26, 2017 10:35:06 AM org.openstreetmap.osmosis.core.Osmosis main
> SEVERE: Execution aborted.
> org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks
> failed. at
> org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
> at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92) at
> org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
>
> 68.1MiB [ 348KiB/s] [ 348KiB/s]
> %
>
> _______________________________________________
> osmosis-dev mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/osmosis-dev

_______________________________________________
osmosis-dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/osmosis-dev