NPE with mkgmap 3320

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

NPE with mkgmap 3320

UliBaer
With the latest versions (tried, back until 3316) of mkgmap i'm getting a NPE with a new setup on an older 32Bit XP Machine:

java.lang.NullPointerException
        at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:700)
        at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:56)
        at java.util.HashMap.compareComparables(Unknown Source)
        at java.util.HashMap$TreeNode.treeify(Unknown Source)
        at java.util.HashMap.treeifyBin(Unknown Source)
        at java.util.HashMap.putVal(Unknown Source)
        at java.util.HashMap.put(Unknown Source)
        at uk.me.parabola.imgfmt.app.net.TableA.addArc(TableA.java:93)
        at uk.me.parabola.imgfmt.app.net.NOD1Part.addNode(NOD1Part.java:229)
        at uk.me.parabola.imgfmt.app.net.RoadNetwork.splitCenters(RoadNetwork.ja
va:239)
        at uk.me.parabola.imgfmt.app.net.RoadNetwork.getCenters(RoadNetwork.java
:250)
        at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:231)
        at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:120)
        at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:82)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:253)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:249)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Exiting - if you want to carry on regardless, use the --keep-going option
Number of ExitExceptions: 1

It is in this code segment:

        public int compareTo(RoadDef other) {
                // sort by city name - this is used to group together
                // roads that have been split into segments
                if(other == this)
                        return 0;

                // TODO: look at what this is doing...
                if(city != null && other.city != null)
                        return city.getName().compareTo(other.city.getName()); <--- Here happens the NPE
                if (hashCode() == other.hashCode())
                        return 0;
                else if (hashCode() < other.hashCode())
                        return -1;
                else
                        return 0;
        }

This happens when running with "--max-jobs=2" (it is a core2duo machine) but when running with "--max-jobs=1" the error does NOT occur... :?

Best regards, Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

UliBaer
...and now for the interesting part:

Declaring the offending method synchronized, e.g.

public synchronized int compareTo(RoadDef other) {
...
}

solved the problem for me.
I don't think this is a general solution, cause the main problem probably lies a lot deeper... :?

Best regards, Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

UliBaer
Last night this error happened again at the same spot.
The remedy is still the same: Run only one thread or declare the method synchronized.

I still think, something is not right there...

java.lang.NullPointerException
        at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:700)
        at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:56)
        at java.util.HashMap.compareComparables(Unknown Source)
        at java.util.HashMap$TreeNode.treeify(Unknown Source)
        at java.util.HashMap.treeifyBin(Unknown Source)
        at java.util.HashMap.putVal(Unknown Source)
        at java.util.HashMap.put(Unknown Source)
        at uk.me.parabola.imgfmt.app.net.TableA.addArc(TableA.java:86)
        at uk.me.parabola.imgfmt.app.net.NOD1Part.addNode(NOD1Part.java:229)
        at uk.me.parabola.imgfmt.app.net.NOD1Part.subdivideHelper(NOD1Part.java:
305)
        at uk.me.parabola.imgfmt.app.net.NOD1Part.subdivide(NOD1Part.java:266)
        at uk.me.parabola.imgfmt.app.net.RoadNetwork.splitCenters(RoadNetwork.ja
va:249)
        at uk.me.parabola.imgfmt.app.net.RoadNetwork.getCenters(RoadNetwork.java
:256)
        at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:231)
        at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:120)
        at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:82)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:253)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:249)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3344!

UliBaer
Forgot: With actual version 3344.
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3344!

Gerd Petermann
Hi Uli,

seems that you are the only user with this problem,
and I don't see an explanation for the problem.
Please provide some details about your JRE
and the java options that you use.

Gerd

> Date: Wed, 5 Nov 2014 09:57:30 -0800

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] NPE with mkgmap 3344!
>
> Forgot: With actual version 3344.
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/NPE-with-mkgmap-3320-tp5813472p5823285.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3344!

UliBaer
Hi Gerd,

i use the following command to start mkgmap:

start /belownormal /b /wait java -XX:+AggressiveOpts -XX:+AggressiveHeap -XX:+ParallelRefProcEnabled -Xmx1400m -jar ..\bin\mkgmap.jar --max-jobs=2 --generate-sea --country-name="%country%" --family-id=%FID% --mapname=66%FID%001 --overview-mapname=66%FID%000 --series-name="OSM-%country%" --family-name="OpenStreetMap: %country%" --style-file=..\bin\styles\ --style=%style% --description="%country%" -c ..\bin\template_basic.args -c template.args %TypFileToAdd%


I now use Java version 8, but the first error occurred with 7.xx, don't remember the exact version:

C:\Tmp>java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b18)
Java HotSpot(TM) Client VM (build 25.25-b02, mixed mode, sharing)

C:\Tmp>


When i used Java 7 i used the following command line to start mkgmap:

start /belownormal /b /wait java -Xmx1444m -jar ..\bin\mkgmap.jar --max-jobs --generate-sea --country-name="%country%" --family-id=%FID% --mapname=66%FID%001 --overview-mapname=66%FID%000 --series-name="OSM-%country%" --family-name="OpenStreetMap: %country%" --style-file=..\bin\styles\ --style=%style% --description="%country%" -c ..\bin\template_basic.args -c template.args %TypFileToAdd%

The error does not occur every time, and can be cured with single threading.

Best regards, Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3344!

UliBaer
BTW:
Since there is a check against City object being null before the compare method:


// TODO: look at what this is doing...
if(city != null && other.city != null)
        return city.getName().compareTo(other.city.getName()); <--- Here happens the NPE
if (hashCode() == other.hashCode())
        return 0;


I suspect the getName()-function from City object to return <null> in some cases.
I'm @ work right now and can't explore, but i remember City.getName() checking for Label != <null> but then calling Label.getText(), which does *not* check for <null>.

Best regards, Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3344!

Gerd Petermann
Hi Uli,

yes, you are probably right.
BTW: This seems to be related to a new HashMap implementation in Java 8.
In my environtment (with Java 7), the compareTo() method is never called
which could explain why nobody else has reported the problem.

Gerd

> Date: Thu, 6 Nov 2014 00:58:06 -0800

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] NPE with mkgmap 3344!
>
> BTW:
> Since there is a check against City object being null before the compare
> method:
>
>
> // TODO: look at what this is doing...
> if(city != null && other.city != null)
> return city.getName().compareTo(other.city.getName()); <--- Here
> happens the NPE
> if (hashCode() == other.hashCode())
> return 0;
>
>
> I suspect the getName()-function from City object to return <null> in some
> cases.
> I'm @ work right now and can't explore, but i remember City.getName()
> checking for Label != <null> but then calling Label.getText(), which does
> *not* check for <null>.
>
> Best regards, Uli
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/NPE-with-mkgmap-3320-tp5813472p5823349.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

Gerd Petermann
In reply to this post by UliBaer
Hi Uli,

I am able to reproduce a similar error when I use a TreeMap
instead of a LinkedHashMap
for RoadDef objects in TableA. I think the problem was introduced with
the merge of the unicode branch (r3076) which
introduced the new encText field in Label and a possible text==null .

  @Steve:
I am not sure what to do here. A method like City.getName() should
probably used the getEncText() method if getText() returns null?
But getEncText() returns a char[] while getText() returns a String,
so that seems to be inefficient :-(

Gerd


> Date: Wed, 5 Nov 2014 09:56:06 -0800

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] NPE with mkgmap 3320
>
> Last night this error happened again at the same spot.
> The remedy is still the same: Run only one thread or declare the method
> synchronized.
>
> I still think, something is not right there...
>
> java.lang.NullPointerException
> at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:700)
> at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:56)
> at java.util.HashMap.compareComparables(Unknown Source)
> at java.util.HashMap$TreeNode.treeify(Unknown Source)
> at java.util.HashMap.treeifyBin(Unknown Source)
> at java.util.HashMap.putVal(Unknown Source)
> at java.util.HashMap.put(Unknown Source)
> at uk.me.parabola.imgfmt.app.net.TableA.addArc(TableA.java:86)
> at uk.me.parabola.imgfmt.app.net.NOD1Part.addNode(NOD1Part.java:229)
> at
> uk.me.parabola.imgfmt.app.net.NOD1Part.subdivideHelper(NOD1Part.java:
> 305)
> at
> uk.me.parabola.imgfmt.app.net.NOD1Part.subdivide(NOD1Part.java:266)
> at
> uk.me.parabola.imgfmt.app.net.RoadNetwork.splitCenters(RoadNetwork.ja
> va:249)
> at
> uk.me.parabola.imgfmt.app.net.RoadNetwork.getCenters(RoadNetwork.java
> :256)
> at
> uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:231)
> at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:120)
> at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:82)
> at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:253)
> at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:249)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.lang.Thread.run(Unknown Source)
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/NPE-with-mkgmap-3320-tp5813472p5823284.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

Gerd Petermann
Oops, sorry, r3076 was the merge of the sort branch, not the unicode branch

Gerd


From: [hidden email]
To: [hidden email]
Date: Thu, 6 Nov 2014 10:42:03 +0100
Subject: Re: [mkgmap-dev] NPE with mkgmap 3320

Hi Uli,

I am able to reproduce a similar error when I use a TreeMap
instead of a LinkedHashMap
for RoadDef objects in TableA. I think the problem was introduced with
the merge of the unicode branch (r3076) which
introduced the new encText field in Label and a possible text==null .

  @Steve:
I am not sure what to do here. A method like City.getName() should
probably used the getEncText() method if getText() returns null?
But getEncText() returns a char[] while getText() returns a String,
so that seems to be inefficient :-(

Gerd


> Date: Wed, 5 Nov 2014 09:56:06 -0800

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] NPE with mkgmap 3320
>
> Last night this error happened again at the same spot.
> The remedy is still the same: Run only one thread or declare the method
> synchronized.
>
> I still think, something is not right there...
>
> java.lang.NullPointerException
> at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:700)
> at uk.me.parabola.imgfmt.app.net.RoadDef.compareTo(RoadDef.java:56)
> at java.util.HashMap.compareComparables(Unknown Source)
> at java.util.HashMap$TreeNode.treeify(Unknown Source)
> at java.util.HashMap.treeifyBin(Unknown Source)
> at java.util.HashMap.putVal(Unknown Source)
> at java.util.HashMap.put(Unknown Source)
> at uk.me.parabola.imgfmt.app.net.TableA.addArc(TableA.java:86)
> at uk.me.parabola.imgfmt.app.net.NOD1Part.addNode(NOD1Part.java:229)
> at
> uk.me.parabola.imgfmt.app.net.NOD1Part.subdivideHelper(NOD1Part.java:
> 305)
> at
> uk.me.parabola.imgfmt.app.net.NOD1Part.subdivide(NOD1Part.java:266)
> at
> uk.me.parabola.imgfmt.app.net.RoadNetwork.splitCenters(RoadNetwork.ja
> va:249)
> at
> uk.me.parabola.imgfmt.app.net.RoadNetwork.getCenters(RoadNetwork.java
> :256)
> at
> uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:231)
> at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:120)
> at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:82)
> at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:253)
> at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:249)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.lang.Thread.run(Unknown Source)
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/NPE-with-mkgmap-3320-tp5813472p5823284.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list [hidden email] http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

UliBaer
Hi Gerd,

this is good news! I'll use my single threaded workaround for the time being.
When a patch or a different approach is committed, i will run again multi threaded.

I was sure that the error also happened with Java 7, but now i'm not so sure anymore.
Maybe, i had already switched to Java 8, when the error occurred the first time.

Thanks and best regards, Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

Gerd Petermann
Hi Uli,

please try r3345. I think there is more to do, but the NPE should
no longer occur.

Gerd

UliBaer wrote
Hi Gerd,

this is good news! I'll use my single threaded workaround for the time being.
When a patch or a different approach is committed, i will run again multi threaded.

I was sure that the error also happened with Java 7, but now i'm not so sure anymore.
Maybe, i had already switched to Java 8, when the error occurred the first time.

Thanks and best regards, Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

UliBaer
Hi Gerd,

now it works multithreaded on JDK 8.
Thanks again!

Uli
Reply | Threaded
Open this post in threaded view
|

Re: NPE with mkgmap 3320

Steve Ratcliffe
In reply to this post by Gerd Petermann
On 06/11/14 09:42, Gerd Petermann wrote:
> @Steve:
> I am not sure what to do here. A method like City.getName() should
> probably used the getEncText() method if getText() returns null?
> But getEncText() returns a char[] while getText() returns a String,
> so that seems to be inefficient :-(

No, not without further thought anyway.  It is done this way so that it
fails if used in the wrong way.

The problem is that we have the real name in unicode and a file
representation that can be very different (due to character set and
transliteration) and it is important to sort on the correct one.

In this particular case of adding to a hash map where the ordering does
not matter to it, then it would probably be OK to compare using the
string name, but that is quite an unusual (and unexpected to me!) case.

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev