[osmosis-dev] Osmosis java.lang.IndexOutOfBoundsException

Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[osmosis-dev] Osmosis java.lang.IndexOutOfBoundsException

Simon Nuttall
Daily builds of Great Britain cycling data have been working fine
until now, when the following error occurs:

Nov 15, 2019 1:42:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.46
Nov 15, 2019 1:42:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new
driver class is `com.mysql.cj.jdbc.Driver'. The driver is
automatically registered via the SPI and manual loading of the driver
class is generally unnecessary.
Nov 15, 2019 1:42:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Nov 15, 2019 1:42:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Nov 15, 2019 1:42:01 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion
SEVERE: Thread for task 1-read-pbf failed
java.lang.IndexOutOfBoundsException: Index:0, Size:0
        at com.google.protobuf.IntArrayList.ensureIndexInRange(IntArrayList.java:263)
        at com.google.protobuf.IntArrayList.getInt(IntArrayList.java:131)
        at org.openstreetmap.osmosis.osmbinary.Osmformat$DenseInfo.getUid(Osmformat.java:6277)
        at crosby.binary.osmosis.OsmosisBinaryParser.parseDense(OsmosisBinaryParser.java:119)
        at org.openstreetmap.osmosis.osmbinary.BinaryParser.parse(BinaryParser.java:124)
        at org.openstreetmap.osmosis.osmbinary.BinaryParser.handleBlock(BinaryParser.java:68)
        at org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:135)
        at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInputStream.java:34)
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)
        at java.base/java.lang.Thread.run(Thread.java:834)

Nov 15, 2019 1:42:01 AM
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion
SEVERE: Thread for task 5-buffer failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An input error
has occurred, aborting.
        at org.openstreetmap.osmosis.core.store.DataPostbox.checkForInputErrors(DataPostbox.java:174)
        at org.openstreetmap.osmosis.core.store.DataPostbox.consumeCentralQueue(DataPostbox.java:244)
        at org.openstreetmap.osmosis.core.store.DataPostbox.hasNext(DataPostbox.java:441)
        at org.openstreetmap.osmosis.core.buffer.v0_6.EntityBuffer.run(EntityBuffer.java:83)
        at java.base/java.lang.Thread.run(Thread.java:834)

Nov 15, 2019 1:42:01 AM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more
tasks failed.
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
        at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
        at org.codehaus.classworlds.Launcher.main(Launcher.java:47)



This is the osmosis call:

/usr/local/bin/osmosis --read-pbf /tmp/great-britain-latest.osm.pbf
--tag-filter accept-ways highway=* aerialway=* route=ferry
man_made=pier,jetty leisure=* landuse=*
amenity=parking,bicycle_parking access=* --tag-filter reject-ways
highway=motorway,bus_guideway,construction bicycle=use_sidepath
--bounding-polygon file="/tmp/great-britain.cs.poly"  --buffer
--write-apidb dbType=mysql populateCurrentTables=no host=localhost
database=planetExtractOSM191115 user=import validateSchemaVersion=no

Do you have an idea of what the problem could be?

I'm guessing it might be an osmid has exceeded a threshold of some
sort - but I don't know where to start debugging this.

Simon

_______________________________________________
osmosis-dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/osmosis-dev
Reply | Threaded
Open this post in threaded view
|

Re: [osmosis-dev] Osmosis java.lang.IndexOutOfBoundsException

hvalentim
Same thing happening here (osmosis v. 0.47). Suddenly:

osmosis --read-pbf file=/home/folder/osm/portugal-latest.osm.pbf --buffer
--tag-transform file=/home/folder/osm/folder/tag-transform.xml --write-pbf
file=/home/folder/osm/portugal-latest-transformed.osm.pbf

is ending with:

nov 15, 2019 11:13:03
org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager
waitForCompletion
SEVERE: Thread for task 1-read-pbf failed
java.lang.IndexOutOfBoundsException: Index:0, Size:0
        at com.google.protobuf.IntArrayList.ensureIndexInRange(Unknown Source)
        at com.google.protobuf.IntArrayList.getInt(Unknown Source)
        at
org.openstreetmap.osmosis.osmbinary.Osmformat$DenseInfo.getUid(Osmformat.java:6606)
        at
crosby.binary.osmosis.OsmosisBinaryParser.parseDense(OsmosisBinaryParser.java:119)
        at
org.openstreetmap.osmosis.osmbinary.BinaryParser.parse(BinaryParser.java:124)
        at
org.openstreetmap.osmosis.osmbinary.BinaryParser.handleBlock(BinaryParser.java:68)
        at
org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:135)
        at
org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInputStream.java:34)
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)
        at java.base/java.lang.Thread.run(Thread.java:834)
(...)



--
Sent from: http://gis.19327.n8.nabble.com/Osmosis-Development-f5338089.html

_______________________________________________
osmosis-dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/osmosis-dev
Reply | Threaded
Open this post in threaded view
|

Re: [osmosis-dev] Osmosis java.lang.IndexOutOfBoundsException

Frederik Ramm
In reply to this post by Simon Nuttall
Hi,

I think this was a problem of the Geofabrik download server. We
accidentally generated our "cleaned" files (the GDPR conformant file
without any user data) in a different way last night, completely
dropping the UID/Username fields, instead of replacing them with 0 and
empty strings as we did before.

This tripped up osmosis and also older versions of osm2pgsql (<= 0.94).

The pbf files are replaced with old-style, compatible files now.

Nightly updates and bz2 files were not affected since they don't use the
PBF format.

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
osmosis-dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/osmosis-dev