Performance with --bounds option

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Performance with --bounds option

Ticker Berkin
Hi Gerd

With the current version I have a tile that takes 3.5 hours to build.
With an old version (r4295) it took about 3 minutes. Without --bounds
it takes 1 minute 15 secs.

Relevant options are:
 --bounds=bounds.zip --location-autofill=is_in,nearest


From the mkgmap.log files:
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
...
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Starting with location hook
INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
 ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
-750000.bnd
... about 40 similar messages ...
INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
 ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
-250000.bnd
WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
 ../mapGB/74220001.osm.pbf: no precompiled boundary information
available for raster tile 2750000_-400000
... 16 similar messages ...
WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
 ../mapGB/74220001.osm.pbf: no precompiled boundary information
available for raster tile 2800000_-250000
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms

The old version had similar lines, but the stats at the end are:

INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms

Ticker

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Gerd Petermann
Hi Ticker,

please tell me the tile boundaries

Gerd


________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Mittwoch, 18. März 2020 11:43
An: mkgmap development
Betreff: [mkgmap-dev] Performance with --bounds option

Hi Gerd

With the current version I have a tile that takes 3.5 hours to build.
With an old version (r4295) it took about 3 minutes. Without --bounds
it takes 1 minute 15 secs.

Relevant options are:
 --bounds=bounds.zip --location-autofill=is_in,nearest


From the mkgmap.log files:
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
...
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Starting with location hook
INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
 ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
-750000.bnd
... about 40 similar messages ...
INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
 ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
-250000.bnd
WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
 ../mapGB/74220001.osm.pbf: no precompiled boundary information
available for raster tile 2750000_-400000
... 16 similar messages ...
WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
 ../mapGB/74220001.osm.pbf: no precompiled boundary information
available for raster tile 2800000_-250000
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms

The old version had similar lines, but the stats at the end are:

INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
 ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms

Ticker

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Ticker Berkin
Hi Gerd

It's

74220001: 2674688,-700416 to 2811904,-233472
#       : 57.392578,-15.029297 to 60.336914,-5.009766

Ticker

On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> please tell me the tile boundaries
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 11:43
> An: mkgmap development
> Betreff: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> With the current version I have a tile that takes 3.5 hours to build.
> With an old version (r4295) it took about 3 minutes. Without --bounds
> it takes 1 minute 15 secs.
>
> Relevant options are:
>  --bounds=bounds.zip --location-autofill=is_in,nearest
>
>
> From the mkgmap.log files:
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> ...
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Starting with location hook
> INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
>  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
> -750000.bnd
> ... about 40 similar messages ...
> INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
>  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
> -250000.bnd
> WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
>  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> available for raster tile 2750000_-400000
> ... 16 similar messages ...
> WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
>  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> available for raster tile 2800000_-250000
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
>
> The old version had similar lines, but the stats at the end are:
>
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
>
> Ticker
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Gerd Petermann
Hi Ticker,

will look at it tomorrow. Maybe you can upload the file to http://files.mkgmap.org.uk/

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Mittwoch, 18. März 2020 11:56
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Hi Gerd

It's

74220001: 2674688,-700416 to 2811904,-233472
#       : 57.392578,-15.029297 to 60.336914,-5.009766

Ticker

On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> please tell me the tile boundaries
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 11:43
> An: mkgmap development
> Betreff: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> With the current version I have a tile that takes 3.5 hours to build.
> With an old version (r4295) it took about 3 minutes. Without --bounds
> it takes 1 minute 15 secs.
>
> Relevant options are:
>  --bounds=bounds.zip --location-autofill=is_in,nearest
>
>
> From the mkgmap.log files:
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> ...
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Starting with location hook
> INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
>  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
> -750000.bnd
> ... about 40 similar messages ...
> INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
>  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
> -250000.bnd
> WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
>  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> available for raster tile 2750000_-400000
> ... 16 similar messages ...
> WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
>  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> available for raster tile 2800000_-250000
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
>
> The old version had similar lines, but the stats at the end are:
>
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
>  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
>
> Ticker
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Ticker Berkin
Have done:

http://files.mkgmap.org.uk/download/462/74220001.osm.pbf

Ticker

On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> will look at it tomorrow. Maybe you can upload the file to
> http://files.mkgmap.org.uk/
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 11:56
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> It's
>
> 74220001: 2674688,-700416 to 2811904,-233472
> #       : 57.392578,-15.029297 to 60.336914,-5.009766
>
> Ticker
>
> On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > please tell me the tile boundaries
> >
> > Gerd
> >
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 11:43
> > An: mkgmap development
> > Betreff: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Gerd
> >
> > With the current version I have a tile that takes 3.5 hours to
> > build.
> > With an old version (r4295) it took about 3 minutes. Without -
> > -bounds
> > it takes 1 minute 15 secs.
> >
> > Relevant options are:
> >  --bounds=bounds.zip --location-autofill=is_in,nearest
> >
> >
> > From the mkgmap.log files:
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > ...
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> >  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
> > -750000.bnd
> > ... about 40 similar messages ...
> > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> >  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
> > -250000.bnd
> > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > available for raster tile 2750000_-400000
> > ... 16 similar messages ...
> > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > available for raster tile 2800000_-250000
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
> >
> > The old version had similar lines, but the stats at the end are:
> >
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> >
> > Ticker
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Gerd Petermann
Hi Ticker,

I forgot to remove some experimental code for is-in branch in BoundaryQuadTree.
Fixed with r4469.
I'll probably also remove the methods pointInsideArea() and pointInsideSingularArea() in BoundaryUtil.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Mittwoch, 18. März 2020 18:16
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Have done:

http://files.mkgmap.org.uk/download/462/74220001.osm.pbf

Ticker

On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> will look at it tomorrow. Maybe you can upload the file to
> http://files.mkgmap.org.uk/
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 11:56
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> It's
>
> 74220001: 2674688,-700416 to 2811904,-233472
> #       : 57.392578,-15.029297 to 60.336914,-5.009766
>
> Ticker
>
> On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > please tell me the tile boundaries
> >
> > Gerd
> >
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 11:43
> > An: mkgmap development
> > Betreff: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Gerd
> >
> > With the current version I have a tile that takes 3.5 hours to
> > build.
> > With an old version (r4295) it took about 3 minutes. Without -
> > -bounds
> > it takes 1 minute 15 secs.
> >
> > Relevant options are:
> >  --bounds=bounds.zip --location-autofill=is_in,nearest
> >
> >
> > From the mkgmap.log files:
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > ...
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> >  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
> > -750000.bnd
> > ... about 40 similar messages ...
> > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> >  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
> > -250000.bnd
> > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > available for raster tile 2750000_-400000
> > ... 16 similar messages ...
> > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > available for raster tile 2800000_-250000
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
> >
> > The old version had similar lines, but the stats at the end are:
> >
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> >
> > Ticker
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Gerd Petermann
Hi Ticker,

some more details:
I didn't think of boundary relations like r1959008 which contains > 700 outer rings. For each node close to that boundary mkgmap called Java2DConverter.areaToSingularAreas() which calculated all the areas again and again.
In Germany, boundary relations typically have only one ring ;)
I think I should look into the code which compiles bounds.zip, somehow I expected that those complex areas are split to singular areas, might improve performance.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Gerd Petermann <[hidden email]>
Gesendet: Mittwoch, 18. März 2020 19:46
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Hi Ticker,

I forgot to remove some experimental code for is-in branch in BoundaryQuadTree.
Fixed with r4469.
I'll probably also remove the methods pointInsideArea() and pointInsideSingularArea() in BoundaryUtil.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Mittwoch, 18. März 2020 18:16
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Have done:

http://files.mkgmap.org.uk/download/462/74220001.osm.pbf

Ticker

On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> will look at it tomorrow. Maybe you can upload the file to
> http://files.mkgmap.org.uk/
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 11:56
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> It's
>
> 74220001: 2674688,-700416 to 2811904,-233472
> #       : 57.392578,-15.029297 to 60.336914,-5.009766
>
> Ticker
>
> On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > please tell me the tile boundaries
> >
> > Gerd
> >
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 11:43
> > An: mkgmap development
> > Betreff: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Gerd
> >
> > With the current version I have a tile that takes 3.5 hours to
> > build.
> > With an old version (r4295) it took about 3 minutes. Without -
> > -bounds
> > it takes 1 minute 15 secs.
> >
> > Relevant options are:
> >  --bounds=bounds.zip --location-autofill=is_in,nearest
> >
> >
> > From the mkgmap.log files:
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > ...
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> >  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2650000_
> > -750000.bnd
> > ... about 40 similar messages ...
> > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> >  ../mapGB/74220001.osm.pbf: loading boundary file: bounds_2800000_
> > -250000.bnd
> > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > available for raster tile 2750000_-400000
> > ... 16 similar messages ...
> > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > available for raster tile 2800000_-250000
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
> >
> > The old version had similar lines, but the stats at the end are:
> >
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> >
> > Ticker
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Ticker Berkin
Hi Gerd

With the reversion of the code in r4469, the performance is acceptable
again. It is only a few tiles that have this problem and, for GB, they
don't add significantly to the overall map generation time

Ticker


On Thu, 2020-03-19 at 06:52 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> some more details:
> I didn't think of boundary relations like r1959008 which contains >
> 700 outer rings. For each node close to that boundary mkgmap called
> Java2DConverter.areaToSingularAreas() which calculated all the areas
> again and again.
> In Germany, boundary relations typically have only one ring ;)
> I think I should look into the code which compiles bounds.zip,
> somehow I expected that those complex areas are split to singular
> areas, might improve performance.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Gerd Petermann <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 19:46
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Ticker,
>
> I forgot to remove some experimental code for is-in branch in
> BoundaryQuadTree.
> Fixed with r4469.
> I'll probably also remove the methods pointInsideArea() and
> pointInsideSingularArea() in BoundaryUtil.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 18:16
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Have done:
>
> http://files.mkgmap.org.uk/download/462/74220001.osm.pbf
>
> Ticker
>
> On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > will look at it tomorrow. Maybe you can upload the file to
> > http://files.mkgmap.org.uk/
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 11:56
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Gerd
> >
> > It's
> >
> > 74220001: 2674688,-700416 to 2811904,-233472
> > #       : 57.392578,-15.029297 to 60.336914,-5.009766
> >
> > Ticker
> >
> > On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > please tell me the tile boundaries
> > >
> > > Gerd
> > >
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Mittwoch, 18. März 2020 11:43
> > > An: mkgmap development
> > > Betreff: [mkgmap-dev] Performance with --bounds option
> > >
> > > Hi Gerd
> > >
> > > With the current version I have a tile that takes 3.5 hours to
> > > build.
> > > With an old version (r4295) it took about 3 minutes. Without -
> > > -bounds
> > > it takes 1 minute 15 secs.
> > >
> > > Relevant options are:
> > >  --bounds=bounds.zip --location-autofill=is_in,nearest
> > >
> > >
> > > From the mkgmap.log files:
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > > ...
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > bounds_2650000_
> > > -750000.bnd
> > > ... about 40 similar messages ...
> > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > bounds_2800000_
> > > -250000.bnd
> > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > available for raster tile 2750000_-400000
> > > ... 16 similar messages ...
> > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > available for raster tile 2800000_-250000
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
> > >
> > > The old version had similar lines, but the stats at the end are:
> > >
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> > >
> > > Ticker
> > >
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Gerd Petermann
Hi Ticker,

when you say acceptable does that mean that latest mkgmap is still much slower than e.g. r4290?

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Donnerstag, 19. März 2020 11:04
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Hi Gerd

With the reversion of the code in r4469, the performance is acceptable
again. It is only a few tiles that have this problem and, for GB, they
don't add significantly to the overall map generation time

Ticker


On Thu, 2020-03-19 at 06:52 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> some more details:
> I didn't think of boundary relations like r1959008 which contains >
> 700 outer rings. For each node close to that boundary mkgmap called
> Java2DConverter.areaToSingularAreas() which calculated all the areas
> again and again.
> In Germany, boundary relations typically have only one ring ;)
> I think I should look into the code which compiles bounds.zip,
> somehow I expected that those complex areas are split to singular
> areas, might improve performance.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Gerd Petermann <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 19:46
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Ticker,
>
> I forgot to remove some experimental code for is-in branch in
> BoundaryQuadTree.
> Fixed with r4469.
> I'll probably also remove the methods pointInsideArea() and
> pointInsideSingularArea() in BoundaryUtil.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 18. März 2020 18:16
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Have done:
>
> http://files.mkgmap.org.uk/download/462/74220001.osm.pbf
>
> Ticker
>
> On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > will look at it tomorrow. Maybe you can upload the file to
> > http://files.mkgmap.org.uk/
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 11:56
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Gerd
> >
> > It's
> >
> > 74220001: 2674688,-700416 to 2811904,-233472
> > #       : 57.392578,-15.029297 to 60.336914,-5.009766
> >
> > Ticker
> >
> > On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > please tell me the tile boundaries
> > >
> > > Gerd
> > >
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Mittwoch, 18. März 2020 11:43
> > > An: mkgmap development
> > > Betreff: [mkgmap-dev] Performance with --bounds option
> > >
> > > Hi Gerd
> > >
> > > With the current version I have a tile that takes 3.5 hours to
> > > build.
> > > With an old version (r4295) it took about 3 minutes. Without -
> > > -bounds
> > > it takes 1 minute 15 secs.
> > >
> > > Relevant options are:
> > >  --bounds=bounds.zip --location-autofill=is_in,nearest
> > >
> > >
> > > From the mkgmap.log files:
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > > ...
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > bounds_2650000_
> > > -750000.bnd
> > > ... about 40 similar messages ...
> > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > bounds_2800000_
> > > -250000.bnd
> > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > available for raster tile 2750000_-400000
> > > ... 16 similar messages ...
> > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > available for raster tile 2800000_-250000
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318 ms
> > >
> > > The old version had similar lines, but the stats at the end are:
> > >
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> > >
> > > Ticker
> > >
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Ticker Berkin
Hi Gerd

These are the timings I get for the single tile 74220001:

trunk with --bounds=bounds.zip
 Total time taken: 3 minutes 44 seconds

r4295 with --bounds=bounds.zip
 Total time taken: 3 minutes 51 seconds

trunk with --no-bounds
 Total time taken: 1 minute 11 seconds

r4295 with --no-bounds
 Total time taken: 1 minute 6 seconds

so the latest version is about the same speed as versions before the
is_in merge.

What I meant by 'acceptable' was that the extra time for bounds
processing, although bad for this tile, is not a concern.
Building britain-and-ireland with bounds takes 36 minutes, without it
takes 29 minutes.

Ticker

On Thu, 2020-03-19 at 12:46 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> when you say acceptable does that mean that latest mkgmap is still
> much slower than e.g. r4290?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Donnerstag, 19. März 2020 11:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> With the reversion of the code in r4469, the performance is
> acceptable
> again. It is only a few tiles that have this problem and, for GB,
> they
> don't add significantly to the overall map generation time
>
> Ticker
>
>
> On Thu, 2020-03-19 at 06:52 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > some more details:
> > I didn't think of boundary relations like r1959008 which contains >
> > 700 outer rings. For each node close to that boundary mkgmap called
> > Java2DConverter.areaToSingularAreas() which calculated all the
> > areas
> > again and again.
> > In Germany, boundary relations typically have only one ring ;)
> > I think I should look into the code which compiles bounds.zip,
> > somehow I expected that those complex areas are split to singular
> > areas, might improve performance.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Gerd Petermann <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 19:46
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Ticker,
> >
> > I forgot to remove some experimental code for is-in branch in
> > BoundaryQuadTree.
> > Fixed with r4469.
> > I'll probably also remove the methods pointInsideArea() and
> > pointInsideSingularArea() in BoundaryUtil.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 18:16
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Have done:
> >
> > http://files.mkgmap.org.uk/download/462/74220001.osm.pbf
> >
> > Ticker
> >
> > On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > will look at it tomorrow. Maybe you can upload the file to
> > > http://files.mkgmap.org.uk/
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Mittwoch, 18. März 2020 11:56
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> > >
> > > Hi Gerd
> > >
> > > It's
> > >
> > > 74220001: 2674688,-700416 to 2811904,-233472
> > > #       : 57.392578,-15.029297 to 60.336914,-5.009766
> > >
> > > Ticker
> > >
> > > On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > please tell me the tile boundaries
> > > >
> > > > Gerd
> > > >
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Mittwoch, 18. März 2020 11:43
> > > > An: mkgmap development
> > > > Betreff: [mkgmap-dev] Performance with --bounds option
> > > >
> > > > Hi Gerd
> > > >
> > > > With the current version I have a tile that takes 3.5 hours to
> > > > build.
> > > > With an old version (r4295) it took about 3 minutes. Without -
> > > > -bounds
> > > > it takes 1 minute 15 secs.
> > > >
> > > > Relevant options are:
> > > >  --bounds=bounds.zip --location-autofill=is_in,nearest
> > > >
> > > >
> > > > From the mkgmap.log files:
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > > > ...
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > > bounds_2650000_
> > > > -750000.bnd
> > > > ... about 40 similar messages ...
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > > bounds_2800000_
> > > > -250000.bnd
> > > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > > available for raster tile 2750000_-400000
> > > > ... 16 similar messages ...
> > > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > > available for raster tile 2800000_-250000
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318
> > > > ms
> > > >
> > > > The old version had similar lines, but the stats at the end
> > > > are:
> > > >
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> > > >
> > > > Ticker
> > > >
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Performance with --bounds option

Gerd Petermann
Hi Ticker,

OK, those numbers look plausible. I see 24 secs without --bounds and 1 minute 49 seconds with --bounds.
The latter value goes down to 1 minute 19 seconds when I use -Xmx2G
I've tried my idea reg. splitting of the complex boundary relations and found no effect on runtime, only a small increase in the bounds.zip.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Donnerstag, 19. März 2020 15:47
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Hi Gerd

These are the timings I get for the single tile 74220001:

trunk with --bounds=bounds.zip
 Total time taken: 3 minutes 44 seconds

r4295 with --bounds=bounds.zip
 Total time taken: 3 minutes 51 seconds

trunk with --no-bounds
 Total time taken: 1 minute 11 seconds

r4295 with --no-bounds
 Total time taken: 1 minute 6 seconds

so the latest version is about the same speed as versions before the
is_in merge.

What I meant by 'acceptable' was that the extra time for bounds
processing, although bad for this tile, is not a concern.
Building britain-and-ireland with bounds takes 36 minutes, without it
takes 29 minutes.

Ticker

On Thu, 2020-03-19 at 12:46 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> when you say acceptable does that mean that latest mkgmap is still
> much slower than e.g. r4290?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Donnerstag, 19. März 2020 11:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> With the reversion of the code in r4469, the performance is
> acceptable
> again. It is only a few tiles that have this problem and, for GB,
> they
> don't add significantly to the overall map generation time
>
> Ticker
>
>
> On Thu, 2020-03-19 at 06:52 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > some more details:
> > I didn't think of boundary relations like r1959008 which contains >
> > 700 outer rings. For each node close to that boundary mkgmap called
> > Java2DConverter.areaToSingularAreas() which calculated all the
> > areas
> > again and again.
> > In Germany, boundary relations typically have only one ring ;)
> > I think I should look into the code which compiles bounds.zip,
> > somehow I expected that those complex areas are split to singular
> > areas, might improve performance.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Gerd Petermann <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 19:46
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Ticker,
> >
> > I forgot to remove some experimental code for is-in branch in
> > BoundaryQuadTree.
> > Fixed with r4469.
> > I'll probably also remove the methods pointInsideArea() and
> > pointInsideSingularArea() in BoundaryUtil.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Mittwoch, 18. März 2020 18:16
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Have done:
> >
> > http://files.mkgmap.org.uk/download/462/74220001.osm.pbf
> >
> > Ticker
> >
> > On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > will look at it tomorrow. Maybe you can upload the file to
> > > http://files.mkgmap.org.uk/
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Mittwoch, 18. März 2020 11:56
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> > >
> > > Hi Gerd
> > >
> > > It's
> > >
> > > 74220001: 2674688,-700416 to 2811904,-233472
> > > #       : 57.392578,-15.029297 to 60.336914,-5.009766
> > >
> > > Ticker
> > >
> > > On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > please tell me the tile boundaries
> > > >
> > > > Gerd
> > > >
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Mittwoch, 18. März 2020 11:43
> > > > An: mkgmap development
> > > > Betreff: [mkgmap-dev] Performance with --bounds option
> > > >
> > > > Hi Gerd
> > > >
> > > > With the current version I have a tile that takes 3.5 hours to
> > > > build.
> > > > With an old version (r4295) it took about 3 minutes. Without -
> > > > -bounds
> > > > it takes 1 minute 15 secs.
> > > >
> > > > Relevant options are:
> > > >  --bounds=bounds.zip --location-autofill=is_in,nearest
> > > >
> > > >
> > > > From the mkgmap.log files:
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > > > ...
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > > bounds_2650000_
> > > > -750000.bnd
> > > > ... about 40 similar messages ...
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > > bounds_2800000_
> > > > -250000.bnd
> > > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > > available for raster tile 2750000_-400000
> > > > ... 16 similar messages ...
> > > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > > available for raster tile 2800000_-250000
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318
> > > > ms
> > > >
> > > > The old version had similar lines, but the stats at the end
> > > > are:
> > > >
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> > > >
> > > > Ticker
> > > >
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev