Please test performance branch

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Please test performance branch

WanMil
Hi all,

I want to merge back the performance branch with big improvements to the
time required by the LocationHook which assigns zip/city/region and
country information for address search.

Please test the performance branch now so that remaining problems can be
fixed before merging it back.

I have created bounds file covering the whole world with data from April
1st. You can download it at:
http://www.navmaps.eu/wanmil/bounds_20120401.zip

Big thanks to Gerd who developed the improvements!

WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

UliBaer-3
Hi WanMil,

tested the performance branch of mkgmap with your new bounds on an extract from europe, containing Germany + 50 km +
Iceland. Works as advertised without error!

UliBär

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

Felix Hartmann-2
In reply to this post by WanMil
I got the following error - and consequently (using keep-going) one empty tile. I don't know if the old version worked fine with boundaries. Never used world boundaries before:
java.lang.AssertionError: boundary bbox doesn't fit into quadtree java.awt.Rectangle[x=4950000,y=-300000,width=50000,height=50000] java.awt.geom.Rectangle2D$Double[x=4973847.840828372,y=-287167.508728
9282,w=0.0,h=5.820766091346741E-11]
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.add(BoundaryQuadTree.java:603)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.access$900(BoundaryQuadTree.java:392)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.readStreamQuadTreeFormat(BoundaryQuadTree.java:356)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.<init>(BoundaryQuadTree.java:109)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTreeFromStream(BoundaryUtil.java:573)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTree(BoundaryUtil.java:150)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.init(BoundaryGrid.java:93)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.<init>(BoundaryGrid.java:61)
        at uk.me.parabola.mkgmap.reader.osm.LocationHook.end(LocationHook.java:112)
        at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79)
        at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68)
        at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:144)
        at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:210)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:207)
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)



data: ccbysa 2.0 openstreetmap.org contributors

On 05.04.2012 21:59, WanMil wrote:
Hi all,

I want to merge back the performance branch with big improvements to the 
time required by the LocationHook which assigns zip/city/region and 
country information for address search.

Please test the performance branch now so that remaining problems can be 
fixed before merging it back.

I have created bounds file covering the whole world with data from April 
1st. You can download it at: 
http://www.navmaps.eu/wanmil/bounds_20120401.zip

Big thanks to Gerd who developed the improvements!

WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

Felix Hartmann-2
ups sorry, for replication: Country is Indonesia from Geofabrik 1.April. Without boundaries the mkgmap trunk compiled fine here (well with 1 week older data, but I think the error is strictly boundary related..).

On 09.04.2012 12:32, Felix Hartmann wrote:
I got the following error - and consequently (using keep-going) one empty tile. I don't know if the old version worked fine with boundaries. Never used world boundaries before:
java.lang.AssertionError: boundary bbox doesn't fit into quadtree java.awt.Rectangle[x=4950000,y=-300000,width=50000,height=50000] java.awt.geom.Rectangle2D$Double[x=4973847.840828372,y=-287167.508728
9282,w=0.0,h=5.820766091346741E-11]
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.add(BoundaryQuadTree.java:603)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.access$900(BoundaryQuadTree.java:392)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.readStreamQuadTreeFormat(BoundaryQuadTree.java:356)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.<init>(BoundaryQuadTree.java:109)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTreeFromStream(BoundaryUtil.java:573)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTree(BoundaryUtil.java:150)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.init(BoundaryGrid.java:93)
        at uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.<init>(BoundaryGrid.java:61)
        at uk.me.parabola.mkgmap.reader.osm.LocationHook.end(LocationHook.java:112)
        at uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79)
        at uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68)
        at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:144)
        at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:210)
        at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:207)
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)



data: ccbysa 2.0 openstreetmap.org contributors

On 05.04.2012 21:59, WanMil wrote:
Hi all,

I want to merge back the performance branch with big improvements to the 
time required by the LocationHook which assigns zip/city/region and 
country information for address search.

Please test the performance branch now so that remaining problems can be 
fixed before merging it back.

I have created bounds file covering the whole world with data from April 
1st. You can download it at: 
http://www.navmaps.eu/wanmil/bounds_20120401.zip

Big thanks to Gerd who developed the improvements!

WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

Gerd Petermann
Hi Felix,

I am able to reproduce the error on my machine and try to find a solution.
As a workaround you can try without -ea

Gerd



Felix Hartmann-2 wrote
ups sorry, for replication: Country is Indonesia from Geofabrik 1.April.
Without boundaries the mkgmap trunk compiled fine here (well with 1 week
older data, but I think the error is strictly boundary related..).

On 09.04.2012 12:32, Felix Hartmann wrote:
> I got the following error - and consequently (using keep-going) one
> empty tile. I don't know if the old version worked fine with
> boundaries. Never used world boundaries before:
> java.lang.AssertionError: boundary bbox doesn't fit into quadtree
> java.awt.Rectangle[x=4950000,y=-300000,width=50000,height=50000]
> java.awt.geom.Rectangle2D$Double[x=4973847.840828372,y=-287167.508728
> 9282,w=0.0,h=5.820766091346741E-11]
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.add(BoundaryQuadTree.java:603)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.access$900(BoundaryQuadTree.java:392)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.readStreamQuadTreeFormat(BoundaryQuadTree.java:356)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.<init>(BoundaryQuadTree.java:109)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTreeFromStream(BoundaryUtil.java:573)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTree(BoundaryUtil.java:150)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.init(BoundaryGrid.java:93)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.<init>(BoundaryGrid.java:61)
>         at
> uk.me.parabola.mkgmap.reader.osm.LocationHook.end(LocationHook.java:112)
>         at
> uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79)
>         at
> uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68)
>         at
> uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:144)
>         at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56)
>         at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:210)
>         at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:207)
>         at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
>
>
>
> data: ccbysa 2.0 openstreetmap.org contributors
>
> On 05.04.2012 21:59, WanMil wrote:
>> Hi all,
>>
>> I want to merge back the performance branch with big improvements to the
>> time required by the LocationHook which assigns zip/city/region and
>> country information for address search.
>>
>> Please test the performance branch now so that remaining problems can be
>> fixed before merging it back.
>>
>> I have created bounds file covering the whole world with data from April
>> 1st. You can download it at:
>> http://www.navmaps.eu/wanmil/bounds_20120401.zip
>>
>> Big thanks to Gerd who developed the improvements!
>>
>> WanMil
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

Gerd Petermann
In reply to this post by Felix Hartmann-2
Hi Felix,

the problem occurs because I've added a check which should detect totally wrong bnd files.
The test should detect that one tries to load data into the wrong part of the quadtree.
Unfortunately, the test fails for an area which describes a vertical or horizontal line.

The bnd file that caused the problem is
bounds_-300000_4950000.bnd
and it seems that someone has already fixed the OSM data that caused the strange area.

The attached patch changes the assertion to allow any kind of area with either zero height or
zero width. The test is quite complex, so it slows down LocationHook a little bit when you
run mkgmap with -ea

BoundaryQuadTree_assertion.patch

Gerd


Felix Hartmann-2 wrote
ups sorry, for replication: Country is Indonesia from Geofabrik 1.April.
Without boundaries the mkgmap trunk compiled fine here (well with 1 week
older data, but I think the error is strictly boundary related..).

On 09.04.2012 12:32, Felix Hartmann wrote:
> I got the following error - and consequently (using keep-going) one
> empty tile. I don't know if the old version worked fine with
> boundaries. Never used world boundaries before:
> java.lang.AssertionError: boundary bbox doesn't fit into quadtree
> java.awt.Rectangle[x=4950000,y=-300000,width=50000,height=50000]
> java.awt.geom.Rectangle2D$Double[x=4973847.840828372,y=-287167.508728
> 9282,w=0.0,h=5.820766091346741E-11]
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.add(BoundaryQuadTree.java:603)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree$Node.access$900(BoundaryQuadTree.java:392)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.readStreamQuadTreeFormat(BoundaryQuadTree.java:356)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryQuadTree.<init>(BoundaryQuadTree.java:109)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTreeFromStream(BoundaryUtil.java:573)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil.loadQuadTree(BoundaryUtil.java:150)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.init(BoundaryGrid.java:93)
>         at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid.<init>(BoundaryGrid.java:61)
>         at
> uk.me.parabola.mkgmap.reader.osm.LocationHook.end(LocationHook.java:112)
>         at
> uk.me.parabola.mkgmap.reader.osm.OsmReadingHooksChain.end(OsmReadingHooksChain.java:79)
>         at
> uk.me.parabola.mkgmap.reader.osm.bin.OsmBinMapDataSource.load(OsmBinMapDataSource.java:68)
>         at
> uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:144)
>         at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:56)
>         at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:210)
>         at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:207)
>         at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
>
>
>
> data: ccbysa 2.0 openstreetmap.org contributors
>
> On 05.04.2012 21:59, WanMil wrote:
>> Hi all,
>>
>> I want to merge back the performance branch with big improvements to the
>> time required by the LocationHook which assigns zip/city/region and
>> country information for address search.
>>
>> Please test the performance branch now so that remaining problems can be
>> fixed before merging it back.
>>
>> I have created bounds file covering the whole world with data from April
>> 1st. You can download it at:
>> http://www.navmaps.eu/wanmil/bounds_20120401.zip
>>
>> Big thanks to Gerd who developed the improvements!
>>
>> WanMil
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

Felix Hartmann-2
Okay, I'll give the patch a try. Without -ea it works fine.

Besides Indonesia most US extracts have several tiles with the same
error message. I gonna recheck if they work too.

On 09.04.2012 16:47, GerdP wrote:

> Hi Felix,
>
> the problem occurs because I've added a check which should detect totally
> wrong bnd files.
> The test should detect that one tries to load data into the wrong part of
> the quadtree.
> Unfortunately, the test fails for an area which describes a vertical or
> horizontal line.
>
> The bnd file that caused the problem is
> bounds_-300000_4950000.bnd
> and it seems that someone has already fixed the OSM data that caused the
> strange area.
>
> The attached patch changes the assertion to allow any kind of area with
> either zero height or
> zero width. The test is quite complex, so it slows down LocationHook a
> little bit when you
> run mkgmap with -ea
>
> http://gis.19327.n5.nabble.com/file/n5627485/BoundaryQuadTree_assertion.patch
> BoundaryQuadTree_assertion.patch
>
> Gerd
>
>
>
>
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

osm-8
In reply to this post by WanMil
Hi,
does it works now with the zip-file of bounds dir?

Henning

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Please test performance branch

WanMil
Hi Henning,

yes --bounds=bounds.zip works.

WanMil

> Hi,
> does it works now with the zip-file of bounds dir?
>
> Henning
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev