[PATCH V1] Idea for minor speed improvement for boundary preprocessing

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH V1] Idea for minor speed improvement for boundary preprocessing

Gerd Petermann
Hi WanMil,

this was a very good hint :-)

Time for african boundaries was 250 secs with /branch/performance/r2225, 267 secs with trunk,
and with this patch it is now 152 secs

- use simple quadtree in BoundarySaver.splitArea()
- add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert. Without that,
an assertion was handled by the threading routines and caused program stop without any
message.

Please double check this, I am still not so experienced
with try+catch and wrong placed

Ciao,
Gerd

splitArea_v1.patch

WanMil wrote
Hi Gerd,

boundary preprocessing takes quite a lot of time. That's not a big
problem because it's performed only by some people who publish their
results.

But I think some speed improvements would not be bad anyhow?!

While preprocessing asia and america I go an idea. The area of a
boundary is splitted into the raster. But each time the complete area is
intersected to the raster. In case an area is big and/or complex
(boundary of russia, USA, canada etc.) this takes a long time. It would
be possible first to cut the area into smaller pieces (columns or rows
of the raster) and then do the final cut to the raster. Maybe this saves
time because the column or row areas should be less complex.

What do you think?

WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH V1] Idea for minor speed improvement for boundary preprocessing

WanMil
Hi Gerd,

that's great!

> Hi WanMil,
>
> this was a very good hint :-)
>
> Time for african boundaries was 250 secs with /branch/performance/r2225, 267
> secs with trunk,
> and with this patch it is now 152 secs

wow, that's more improvement than I expected.

>
> - use simple quadtree in BoundarySaver.splitArea()

You are a quadtree guy.
In this case I thinks it's more irritating and complex to split the area
into four subareas. I've changed that to split into two subareas and the
code is now much easier to read (I think so..?!?) and performs in
similar speed.

WanMil

> - add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert.
> Without that,
> an assertion was handled by the threading routines and caused program stop
> without any
> message.
>
> Please double check this, I am still not so experienced
> with try+catch and wrong placed
>
> Ciao,
> Gerd
>
> http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch
> splitArea_v1.patch
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> boundary preprocessing takes quite a lot of time. That's not a big
>> problem because it's performed only by some people who publish their
>> results.
>>
>> But I think some speed improvements would not be bad anyhow?!
>>
>> While preprocessing asia and america I go an idea. The area of a
>> boundary is splitted into the raster. But each time the complete area is
>> intersected to the raster. In case an area is big and/or complex
>> (boundary of russia, USA, canada etc.) this takes a long time. It would
>> be possible first to cut the area into smaller pieces (columns or rows
>> of the raster) and then do the final cut to the raster. Maybe this saves
>> time because the column or row areas should be less complex.
>>
>> What do you think?
>>
>> WanMil
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for-boundary-preprocessing-tp5512906p5512906.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

splitArea_v2.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH V1] Idea for minor speed improvement for boundary preprocessing

Gerd Petermann
Hi WanMil,

yes, your solution is better. I wanted to do it in the same way, but failed to find the
correct tile sizes for areas that cover e.g. 7 split rectangles (the middle is not on the grid)

two small points:
1) the variable names midLon and midLat are a bit missleading, since you don't split in the middle, but near the middle (which perfectly solves my problem described above)

2) You removed my added try/catch from BoundaryPreparer.run()
I hope you found a better solution? I needed it there to make assert work in e.g. splitArea()

Gerd

WanMil wrote
Hi Gerd,

that's great!

> Hi WanMil,
>
> this was a very good hint :-)
>
> Time for african boundaries was 250 secs with /branch/performance/r2225, 267
> secs with trunk,
> and with this patch it is now 152 secs

wow, that's more improvement than I expected.

>
> - use simple quadtree in BoundarySaver.splitArea()

You are a quadtree guy.
In this case I thinks it's more irritating and complex to split the area
into four subareas. I've changed that to split into two subareas and the
code is now much easier to read (I think so..?!?) and performs in
similar speed.

WanMil

> - add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert.
> Without that,
> an assertion was handled by the threading routines and caused program stop
> without any
> message.
>
> Please double check this, I am still not so experienced
> with try+catch and wrong placed
>
> Ciao,
> Gerd
>
> http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch
> splitArea_v1.patch
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> boundary preprocessing takes quite a lot of time. That's not a big
>> problem because it's performed only by some people who publish their
>> results.
>>
>> But I think some speed improvements would not be bad anyhow?!
>>
>> While preprocessing asia and america I go an idea. The area of a
>> boundary is splitted into the raster. But each time the complete area is
>> intersected to the raster. In case an area is big and/or complex
>> (boundary of russia, USA, canada etc.) this takes a long time. It would
>> be possible first to cut the area into smaller pieces (columns or rows
>> of the raster) and then do the final cut to the raster. Maybe this saves
>> time because the column or row areas should be less complex.
>>
>> What do you think?
>>
>> WanMil
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for-boundary-preprocessing-tp5512906p5512906.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH V1] Idea for minor speed improvement for boundary preprocessing

WanMil
> Hi WanMil,
>
> yes, your solution is better. I wanted to do it in the same way, but failed
> to find the
> correct tile sizes for areas that cover e.g. 7 split rectangles (the middle
> is not on the grid)
>
> two small points:
> 1) the variable names midLon and midLat are a bit missleading, since you
> don't split in the middle, but near the middle (which perfectly solves my
> problem described above)
>
> 2) You removed my added try/catch from BoundaryPreparer.run()
> I hope you found a better solution? I needed it there to make assert work in
> e.g. splitArea()

An assertion is a check for something that should never be true. Only in
case there is a bug this might become true. By default assertions are
not checked only if you add the java paramerter -ea.
If you want to perform a runtime check you should throw an Exception.

WanMil

>
> Gerd
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> that's great!
>>
>>> Hi WanMil,
>>>
>>> this was a very good hint :-)
>>>
>>> Time for african boundaries was 250 secs with /branch/performance/r2225,
>>> 267
>>> secs with trunk,
>>> and with this patch it is now 152 secs
>>
>> wow, that's more improvement than I expected.
>>
>>>
>>> - use simple quadtree in BoundarySaver.splitArea()
>>
>> You are a quadtree guy.
>> In this case I thinks it's more irritating and complex to split the area
>> into four subareas. I've changed that to split into two subareas and the
>> code is now much easier to read (I think so..?!?) and performs in
>> similar speed.
>>
>> WanMil
>>
>>> - add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert.
>>> Without that,
>>> an assertion was handled by the threading routines and caused program
>>> stop
>>> without any
>>> message.
>>>
>>> Please double check this, I am still not so experienced
>>> with try+catch and wrong placed
>>>
>>> Ciao,
>>> Gerd
>>>
>>> http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch
>>> splitArea_v1.patch
>>>
>>>
>>> WanMil wrote
>>>>
>>>> Hi Gerd,
>>>>
>>>> boundary preprocessing takes quite a lot of time. That's not a big
>>>> problem because it's performed only by some people who publish their
>>>> results.
>>>>
>>>> But I think some speed improvements would not be bad anyhow?!
>>>>
>>>> While preprocessing asia and america I go an idea. The area of a
>>>> boundary is splitted into the raster. But each time the complete area is
>>>> intersected to the raster. In case an area is big and/or complex
>>>> (boundary of russia, USA, canada etc.) this takes a long time. It would
>>>> be possible first to cut the area into smaller pieces (columns or rows
>>>> of the raster) and then do the final cut to the raster. Maybe this saves
>>>> time because the column or row areas should be less complex.
>>>>
>>>> What do you think?
>>>>
>>>> WanMil
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev@.org
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for-boundary-preprocessing-tp5512906p5512906.html
>>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for-boundary-preprocessing-tp5512906p5531433.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH V1] Idea for minor speed improvement for boundary preprocessing

Gerd Petermann
Hi WanMil,

but if I code an assert statement and execute the program with -ea, I surely want to see the assertion that happened. Instead, the program stopped without any message.

Gerd

WanMil wrote
> Hi WanMil,
>
> yes, your solution is better. I wanted to do it in the same way, but failed
> to find the
> correct tile sizes for areas that cover e.g. 7 split rectangles (the middle
> is not on the grid)
>
> two small points:
> 1) the variable names midLon and midLat are a bit missleading, since you
> don't split in the middle, but near the middle (which perfectly solves my
> problem described above)
>
> 2) You removed my added try/catch from BoundaryPreparer.run()
> I hope you found a better solution? I needed it there to make assert work in
> e.g. splitArea()

An assertion is a check for something that should never be true. Only in
case there is a bug this might become true. By default assertions are
not checked only if you add the java paramerter -ea.
If you want to perform a runtime check you should throw an Exception.

WanMil

>
> Gerd
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> that's great!
>>
>>> Hi WanMil,
>>>
>>> this was a very good hint :-)
>>>
>>> Time for african boundaries was 250 secs with /branch/performance/r2225,
>>> 267
>>> secs with trunk,
>>> and with this patch it is now 152 secs
>>
>> wow, that's more improvement than I expected.
>>
>>>
>>> - use simple quadtree in BoundarySaver.splitArea()
>>
>> You are a quadtree guy.
>> In this case I thinks it's more irritating and complex to split the area
>> into four subareas. I've changed that to split into two subareas and the
>> code is now much easier to read (I think so..?!?) and performs in
>> similar speed.
>>
>> WanMil
>>
>>> - add try/catch in BoundaryPreparer.run() to allow e.g. usage of assert.
>>> Without that,
>>> an assertion was handled by the threading routines and caused program
>>> stop
>>> without any
>>> message.
>>>
>>> Please double check this, I am still not so experienced
>>> with try+catch and wrong placed
>>>
>>> Ciao,
>>> Gerd
>>>
>>> http://gis.19327.n5.nabble.com/file/n5512906/splitArea_v1.patch
>>> splitArea_v1.patch
>>>
>>>
>>> WanMil wrote
>>>>
>>>> Hi Gerd,
>>>>
>>>> boundary preprocessing takes quite a lot of time. That's not a big
>>>> problem because it's performed only by some people who publish their
>>>> results.
>>>>
>>>> But I think some speed improvements would not be bad anyhow?!
>>>>
>>>> While preprocessing asia and america I go an idea. The area of a
>>>> boundary is splitted into the raster. But each time the complete area is
>>>> intersected to the raster. In case an area is big and/or complex
>>>> (boundary of russia, USA, canada etc.) this takes a long time. It would
>>>> be possible first to cut the area into smaller pieces (columns or rows
>>>> of the raster) and then do the final cut to the raster. Maybe this saves
>>>> time because the column or row areas should be less complex.
>>>>
>>>> What do you think?
>>>>
>>>> WanMil
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev@.org
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for-boundary-preprocessing-tp5512906p5512906.html
>>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/PATCH-V1-Idea-for-minor-speed-improvement-for-boundary-preprocessing-tp5512906p5531433.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev