Small holes in boundary coverage

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Small holes in boundary coverage

WanMil
Hi,

I have compiled world bounds using the performance branch. They can be
downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.

I have checked the coverage of different admin_levels with the
BoundaryCoverageUtil. The result for admin_level=2  can be downloaded
from http://files.mkgmap.org.uk/detail/61.

Some countries are missing (USA, Canada, some african countries). I
assume that their boundaries were/are corrupt.

The more interesting thing is that there are small holes throughout the
world in admin_level=2. They seem to be created by the bounds
preparation and should not be there. Gerd, can you please have a look on it?

Thanks!
WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Thorsten Kukuk
On Thu, Mar 15, WanMil wrote:

> Some countries are missing (USA, Canada, some african countries). I
> assume that their boundaries were/are corrupt.

The boundary between USA and Canada was broken, but fixed yesterday.

  Thorsten

--
Thorsten Kukuk, Project Manager/Release Manager SLES
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imend├Ârffer, HRB 16746 (AG N├╝rnberg)
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
In reply to this post by WanMil
Hi WanMil,

I saw these holes, but did not find a good reason besides the rounding problems.
I look at it again tomorrow.

Gerd

WanMil wrote
Hi,

I have compiled world bounds using the performance branch. They can be
downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.

I have checked the coverage of different admin_levels with the
BoundaryCoverageUtil. The result for admin_level=2  can be downloaded
from http://files.mkgmap.org.uk/detail/61.

Some countries are missing (USA, Canada, some african countries). I
assume that their boundaries were/are corrupt.

The more interesting thing is that there are small holes throughout the
world in admin_level=2. They seem to be created by the bounds
preparation and should not be there. Gerd, can you please have a look on it?

Thanks!
WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi,

short status update
I fixed one problem in BoundaryQuadTree which removed most of the holes,
but I still find one in africa which is not shown in with trunk version, so I
will continue tomorrow ...

Gerd

GerdP wrote
Hi WanMil,

I saw these holes, but did not find a good reason besides the rounding problems.
I look at it again tomorrow.

Gerd

WanMil wrote
Hi,

I have compiled world bounds using the performance branch. They can be
downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.

I have checked the coverage of different admin_levels with the
BoundaryCoverageUtil. The result for admin_level=2  can be downloaded
from http://files.mkgmap.org.uk/detail/61.

Some countries are missing (USA, Canada, some african countries). I
assume that their boundaries were/are corrupt.

The more interesting thing is that there are small holes throughout the
world in admin_level=2. They seem to be created by the bounds
preparation and should not be there. Gerd, can you please have a look on it?

Thanks!
WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
In reply to this post by WanMil
Hi WanMil,

attached is a fix for this problem.

Please, can you review if the UnusedElementsRemoverHook is still useful?
With my test data, it is slowing down mkgmap a little bit and I also see
a different result for one tile in the UK when I disable it.

Gerd

> Date: Thu, 15 Mar 2012 21:11:02 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: [mkgmap-dev] Small holes in boundary coverage
>
> Hi,
>
> I have compiled world bounds using the performance branch. They can be
> downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>
> I have checked the coverage of different admin_levels with the
> BoundaryCoverageUtil. The result for admin_level=2 can be downloaded
> from http://files.mkgmap.org.uk/detail/61.
>
> Some countries are missing (USA, Canada, some african countries). I
> assume that their boundaries were/are corrupt.
>
> The more interesting thing is that there are small holes throughout the
> world in admin_level=2. They seem to be created by the bounds
> preparation and should not be there. Gerd, can you please have a look on it?
>
> Thanks!
> WanMil
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

holes_patch_v1.patch (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
Hi Gerd,

thanks for the patch. I have commited it although I still see the
problem. I am not sure if the holes are at the same place but there are
some of them which I cannot explain with wrong boundary data.

Can you please recheck that?

Thanks!
WanMil

> Hi WanMil,
>
> attached is a fix for this problem.
>
> Please, can you review if the UnusedElementsRemoverHook is still useful?
> With my test data, it is slowing down mkgmap a little bit and I also see
> a different result for one tile in the UK when I disable it.
>
> Gerd
>
>  > Date: Thu, 15 Mar 2012 21:11:02 +0100
>  > From: [hidden email]
>  > To: [hidden email]
>  > Subject: [mkgmap-dev] Small holes in boundary coverage
>  >
>  > Hi,
>  >
>  > I have compiled world bounds using the performance branch. They can be
>  > downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>  >
>  > I have checked the coverage of different admin_levels with the
>  > BoundaryCoverageUtil. The result for admin_level=2 can be downloaded
>  > from http://files.mkgmap.org.uk/detail/61.
>  >
>  > Some countries are missing (USA, Canada, some african countries). I
>  > assume that their boundaries were/are corrupt.
>  >
>  > The more interesting thing is that there are small holes throughout the
>  > world in admin_level=2. They seem to be created by the bounds
>  > preparation and should not be there. Gerd, can you please have a look
> on it?
>  >
>  > Thanks!
>  > WanMil
>  > _______________________________________________
>  > mkgmap-dev mailing list
>  > [hidden email]
>  > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

I found three reasons for these holes: errors in BoundaryQuadTree, "errors" in java.awt.geom.Area and rounding erros.

My last patches fixed the errors that I found in BoundaryQuadTree, I also tried to avoid the problem
with the rounding errors (we are doing the calculations with double precision, but we save with float
precision).

I stopped  searching when I saw fewer holes in the result of the performance branch
than in the result of trunk.

The errors in java.awt.geom.Area are complex. I think one has different options to calculate the intersection of two areas a1 and a2:
a) Area x = new Area(a1); x.intersect(a2)
b) Area x = new Area(a2); x.intersect(a1)
c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);

In an ideal world I would expect to get exactly the same result for these three calculations, but sometimes the real results are not equal. The same problem occurs when we add the areas again in the BoundaryCoverageUtil.

Maybe we can get rid of a few more of the holes if we manage to do all calculations with doubles, but probably even doubles will not solve all problems.

The question is if we should invest time for that. I think the holes are purely cosmetic, if you really manage to query BoundaryQuadTree for a point that lies within such a hole, it will return the result for a point that is next to it, I think this is good enough.

What do you mean?

Gerd


WanMil wrote
Hi Gerd,

thanks for the patch. I have commited it although I still see the
problem. I am not sure if the holes are at the same place but there are
some of them which I cannot explain with wrong boundary data.

Can you please recheck that?

Thanks!
WanMil

> Hi WanMil,
>
> attached is a fix for this problem.
>
> Please, can you review if the UnusedElementsRemoverHook is still useful?
> With my test data, it is slowing down mkgmap a little bit and I also see
> a different result for one tile in the UK when I disable it.
>
> Gerd
>
>  > Date: Thu, 15 Mar 2012 21:11:02 +0100
>  > From: [hidden email]
>  > To: [hidden email]
>  > Subject: [mkgmap-dev] Small holes in boundary coverage
>  >
>  > Hi,
>  >
>  > I have compiled world bounds using the performance branch. They can be
>  > downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>  >
>  > I have checked the coverage of different admin_levels with the
>  > BoundaryCoverageUtil. The result for admin_level=2 can be downloaded
>  > from http://files.mkgmap.org.uk/detail/61.
>  >
>  > Some countries are missing (USA, Canada, some african countries). I
>  > assume that their boundaries were/are corrupt.
>  >
>  > The more interesting thing is that there are small holes throughout the
>  > world in admin_level=2. They seem to be created by the bounds
>  > preparation and should not be there. Gerd, can you please have a look
> on it?
>  >
>  > Thanks!
>  > WanMil
>  > _______________________________________________
>  > mkgmap-dev mailing list
>  > [hidden email]
>  > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
Hi Gerd,

your conclusion sound very reasonable. The number of holes has been
reduced quite a lot after your last patch and if your example a) b) and
c) are not equal all the time there is no chance to avoid such holes
completely.

Just a quick idea: Would it improve (and reduce the number of holes) if
the intermediate bounds format which is written first uses double
instead of float precision? I think it doesn't really matter if the
intermediate format uses more space on disk?!

I will try to create updated bounds files during the weekend and then we
can start a one week test phase with more users before merging back to
trunk.

WanMil

> Hi WanMil,
>
> I found three reasons for these holes: errors in BoundaryQuadTree, "errors"
> in java.awt.geom.Area and rounding erros.
>
> My last patches fixed the errors that I found in BoundaryQuadTree, I also
> tried to avoid the problem
> with the rounding errors (we are doing the calculations with double
> precision, but we save with float
> precision).
>
> I stopped  searching when I saw fewer holes in the result of the performance
> branch
> than in the result of trunk.
>
> The errors in java.awt.geom.Area are complex. I think one has different
> options to calculate the intersection of two areas a1 and a2:
> a) Area x = new Area(a1); x.intersect(a2)
> b) Area x = new Area(a2); x.intersect(a1)
> c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);
>
> In an ideal world I would expect to get exactly the same result for these
> three calculations, but sometimes the real results are not equal. The same
> problem occurs when we add the areas again in the BoundaryCoverageUtil.
>
> Maybe we can get rid of a few more of the holes if we manage to do all
> calculations with doubles, but probably even doubles will not solve all
> problems.
>
> The question is if we should invest time for that. I think the holes are
> purely cosmetic, if you really manage to query BoundaryQuadTree for a point
> that lies within such a hole, it will return the result for a point that is
> next to it, I think this is good enough.
>
> What do you mean?
>
> Gerd
>
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> thanks for the patch. I have commited it although I still see the
>> problem. I am not sure if the holes are at the same place but there are
>> some of them which I cannot explain with wrong boundary data.
>>
>> Can you please recheck that?
>>
>> Thanks!
>> WanMil
>>
>>> Hi WanMil,
>>>
>>> attached is a fix for this problem.
>>>
>>> Please, can you review if the UnusedElementsRemoverHook is still useful?
>>> With my test data, it is slowing down mkgmap a little bit and I also see
>>> a different result for one tile in the UK when I disable it.
>>>
>>> Gerd
>>>
>>>   >  Date: Thu, 15 Mar 2012 21:11:02 +0100
>>>   >  From: wmgcnfg@
>>>   >  To: mkgmap-dev@.org
>>>   >  Subject: [mkgmap-dev] Small holes in boundary coverage
>>>   >
>>>   >  Hi,
>>>   >
>>>   >  I have compiled world bounds using the performance branch. They can be
>>>   >  downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>>>   >
>>>   >  I have checked the coverage of different admin_levels with the
>>>   >  BoundaryCoverageUtil. The result for admin_level=2 can be downloaded
>>>   >  from http://files.mkgmap.org.uk/detail/61.
>>>   >
>>>   >  Some countries are missing (USA, Canada, some african countries). I
>>>   >  assume that their boundaries were/are corrupt.
>>>   >
>>>   >  The more interesting thing is that there are small holes throughout
>>> the
>>>   >  world in admin_level=2. They seem to be created by the bounds
>>>   >  preparation and should not be there. Gerd, can you please have a look
>>> on it?
>>>   >
>>>   >  Thanks!
>>>   >  WanMil
>>>   >  _______________________________________________
>>>   >  mkgmap-dev mailing list
>>>   >  mkgmap-dev@.org
>>>   >  http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5597147.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

I can try writing the RAW format with double precision. Do you have a sample input that
creates a hole? I tried africa and asia and found none, the europe file is too large for my
machine.
We may also try to save the quadtree data with doubles without getting much larger files if we use a compressed format (e.g. the delta coding used in pbf or o5m).

Gerd

WanMil wrote
Hi Gerd,

your conclusion sound very reasonable. The number of holes has been
reduced quite a lot after your last patch and if your example a) b) and
c) are not equal all the time there is no chance to avoid such holes
completely.

Just a quick idea: Would it improve (and reduce the number of holes) if
the intermediate bounds format which is written first uses double
instead of float precision? I think it doesn't really matter if the
intermediate format uses more space on disk?!

I will try to create updated bounds files during the weekend and then we
can start a one week test phase with more users before merging back to
trunk.

WanMil

> Hi WanMil,
>
> I found three reasons for these holes: errors in BoundaryQuadTree, "errors"
> in java.awt.geom.Area and rounding erros.
>
> My last patches fixed the errors that I found in BoundaryQuadTree, I also
> tried to avoid the problem
> with the rounding errors (we are doing the calculations with double
> precision, but we save with float
> precision).
>
> I stopped  searching when I saw fewer holes in the result of the performance
> branch
> than in the result of trunk.
>
> The errors in java.awt.geom.Area are complex. I think one has different
> options to calculate the intersection of two areas a1 and a2:
> a) Area x = new Area(a1); x.intersect(a2)
> b) Area x = new Area(a2); x.intersect(a1)
> c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);
>
> In an ideal world I would expect to get exactly the same result for these
> three calculations, but sometimes the real results are not equal. The same
> problem occurs when we add the areas again in the BoundaryCoverageUtil.
>
> Maybe we can get rid of a few more of the holes if we manage to do all
> calculations with doubles, but probably even doubles will not solve all
> problems.
>
> The question is if we should invest time for that. I think the holes are
> purely cosmetic, if you really manage to query BoundaryQuadTree for a point
> that lies within such a hole, it will return the result for a point that is
> next to it, I think this is good enough.
>
> What do you mean?
>
> Gerd
>
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> thanks for the patch. I have commited it although I still see the
>> problem. I am not sure if the holes are at the same place but there are
>> some of them which I cannot explain with wrong boundary data.
>>
>> Can you please recheck that?
>>
>> Thanks!
>> WanMil
>>
>>> Hi WanMil,
>>>
>>> attached is a fix for this problem.
>>>
>>> Please, can you review if the UnusedElementsRemoverHook is still useful?
>>> With my test data, it is slowing down mkgmap a little bit and I also see
>>> a different result for one tile in the UK when I disable it.
>>>
>>> Gerd
>>>
>>>   >  Date: Thu, 15 Mar 2012 21:11:02 +0100
>>>   >  From: wmgcnfg@
>>>   >  To: mkgmap-dev@.org
>>>   >  Subject: [mkgmap-dev] Small holes in boundary coverage
>>>   >
>>>   >  Hi,
>>>   >
>>>   >  I have compiled world bounds using the performance branch. They can be
>>>   >  downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>>>   >
>>>   >  I have checked the coverage of different admin_levels with the
>>>   >  BoundaryCoverageUtil. The result for admin_level=2 can be downloaded
>>>   >  from http://files.mkgmap.org.uk/detail/61.
>>>   >
>>>   >  Some countries are missing (USA, Canada, some african countries). I
>>>   >  assume that their boundaries were/are corrupt.
>>>   >
>>>   >  The more interesting thing is that there are small holes throughout
>>> the
>>>   >  world in admin_level=2. They seem to be created by the bounds
>>>   >  preparation and should not be there. Gerd, can you please have a look
>>> on it?
>>>   >
>>>   >  Thanks!
>>>   >  WanMil
>>>   >  _______________________________________________
>>>   >  mkgmap-dev mailing list
>>>   >  mkgmap-dev@.org
>>>   >  http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5597147.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

attached is a patch that changes all boundary routines to use double precision.
The file format has changed again, so one has to create new boundaries.

boundary_double_v1.patch

Although it uses double precision, the new format produces smaller files because
it uses delta coding for the boundary coordinates.

I found a strange error in Area.subtract(). For two given areas a1 and a2 lying within
a bbox, a1.subtract(a2) may result in an area lying outside of the bbox.
I've added a test for that.
The performance is almost equal, the newer version  may be a bit faster.

Gerd


GerdP wrote
Hi WanMil,

I can try writing the RAW format with double precision. Do you have a sample input that
creates a hole? I tried africa and asia and found none, the europe file is too large for my
machine.
We may also try to save the quadtree data with doubles without getting much larger files if we use a compressed format (e.g. the delta coding used in pbf or o5m).

Gerd

WanMil wrote
Hi Gerd,

your conclusion sound very reasonable. The number of holes has been
reduced quite a lot after your last patch and if your example a) b) and
c) are not equal all the time there is no chance to avoid such holes
completely.

Just a quick idea: Would it improve (and reduce the number of holes) if
the intermediate bounds format which is written first uses double
instead of float precision? I think it doesn't really matter if the
intermediate format uses more space on disk?!

I will try to create updated bounds files during the weekend and then we
can start a one week test phase with more users before merging back to
trunk.

WanMil

> Hi WanMil,
>
> I found three reasons for these holes: errors in BoundaryQuadTree, "errors"
> in java.awt.geom.Area and rounding erros.
>
> My last patches fixed the errors that I found in BoundaryQuadTree, I also
> tried to avoid the problem
> with the rounding errors (we are doing the calculations with double
> precision, but we save with float
> precision).
>
> I stopped  searching when I saw fewer holes in the result of the performance
> branch
> than in the result of trunk.
>
> The errors in java.awt.geom.Area are complex. I think one has different
> options to calculate the intersection of two areas a1 and a2:
> a) Area x = new Area(a1); x.intersect(a2)
> b) Area x = new Area(a2); x.intersect(a1)
> c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);
>
> In an ideal world I would expect to get exactly the same result for these
> three calculations, but sometimes the real results are not equal. The same
> problem occurs when we add the areas again in the BoundaryCoverageUtil.
>
> Maybe we can get rid of a few more of the holes if we manage to do all
> calculations with doubles, but probably even doubles will not solve all
> problems.
>
> The question is if we should invest time for that. I think the holes are
> purely cosmetic, if you really manage to query BoundaryQuadTree for a point
> that lies within such a hole, it will return the result for a point that is
> next to it, I think this is good enough.
>
> What do you mean?
>
> Gerd
>
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> thanks for the patch. I have commited it although I still see the
>> problem. I am not sure if the holes are at the same place but there are
>> some of them which I cannot explain with wrong boundary data.
>>
>> Can you please recheck that?
>>
>> Thanks!
>> WanMil
>>
>>> Hi WanMil,
>>>
>>> attached is a fix for this problem.
>>>
>>> Please, can you review if the UnusedElementsRemoverHook is still useful?
>>> With my test data, it is slowing down mkgmap a little bit and I also see
>>> a different result for one tile in the UK when I disable it.
>>>
>>> Gerd
>>>
>>>   >  Date: Thu, 15 Mar 2012 21:11:02 +0100
>>>   >  From: wmgcnfg@
>>>   >  To: mkgmap-dev@.org
>>>   >  Subject: [mkgmap-dev] Small holes in boundary coverage
>>>   >
>>>   >  Hi,
>>>   >
>>>   >  I have compiled world bounds using the performance branch. They can be
>>>   >  downloaded from http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>>>   >
>>>   >  I have checked the coverage of different admin_levels with the
>>>   >  BoundaryCoverageUtil. The result for admin_level=2 can be downloaded
>>>   >  from http://files.mkgmap.org.uk/detail/61.
>>>   >
>>>   >  Some countries are missing (USA, Canada, some african countries). I
>>>   >  assume that their boundaries were/are corrupt.
>>>   >
>>>   >  The more interesting thing is that there are small holes throughout
>>> the
>>>   >  world in admin_level=2. They seem to be created by the bounds
>>>   >  preparation and should not be there. Gerd, can you please have a look
>>> on it?
>>>   >
>>>   >  Thanks!
>>>   >  WanMil
>>>   >  _______________________________________________
>>>   >  mkgmap-dev mailing list
>>>   >  mkgmap-dev@.org
>>>   >  http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5597147.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
Hi Gerd,

thanks a lot.
Unfortunately the new patched code creates more holes than the current
SVN code. The BoundaryCoverageUtil creates 1000 separate polygons for
admin_level=2 instead of 640.

I have uploaded the results of the BoundaryCoverageUtil to
http://www.navmaps.eu/wanmil/coverage_2.zip.

It seems that there are many holes around the Greenwich meridian. Can
you have a look on it and check if old or new code is better?

Anyhow I aggree that the holes are more a cosmetic problem. But maybe
it's worthy to invest the time to know exactly what happens there to be
sure that there are no other problems.

Thanks!
WanMil

> Hi WanMil,
>
> attached is a patch that changes all boundary routines to use double
> precision.
> The file format has changed again, so one has to create new boundaries.
>
> http://gis.19327.n5.nabble.com/file/n5604371/boundary_double_v1.patch
> boundary_double_v1.patch
>
> Although it uses double precision, the new format produces smaller files
> because
> it uses delta coding for the boundary coordinates.
>
> I found a strange error in Area.subtract(). For two given areas a1 and a2
> lying within
> a bbox, a1.subtract(a2) may result in an area lying outside of the bbox.
> I've added a test for that.
> The performance is almost equal, the newer version  may be a bit faster.
>
> Gerd
>
>
>
> GerdP wrote
>>
>> Hi WanMil,
>>
>> I can try writing the RAW format with double precision. Do you have a
>> sample input that
>> creates a hole? I tried africa and asia and found none, the europe file is
>> too large for my
>> machine.
>> We may also try to save the quadtree data with doubles without getting
>> much larger files if we use a compressed format (e.g. the delta coding
>> used in pbf or o5m).
>>
>> Gerd
>>
>>
>> WanMil wrote
>>>
>>> Hi Gerd,
>>>
>>> your conclusion sound very reasonable. The number of holes has been
>>> reduced quite a lot after your last patch and if your example a) b) and
>>> c) are not equal all the time there is no chance to avoid such holes
>>> completely.
>>>
>>> Just a quick idea: Would it improve (and reduce the number of holes) if
>>> the intermediate bounds format which is written first uses double
>>> instead of float precision? I think it doesn't really matter if the
>>> intermediate format uses more space on disk?!
>>>
>>> I will try to create updated bounds files during the weekend and then we
>>> can start a one week test phase with more users before merging back to
>>> trunk.
>>>
>>> WanMil
>>>
>>>> Hi WanMil,
>>>>
>>>> I found three reasons for these holes: errors in BoundaryQuadTree,
>>>> "errors"
>>>> in java.awt.geom.Area and rounding erros.
>>>>
>>>> My last patches fixed the errors that I found in BoundaryQuadTree, I
>>>> also
>>>> tried to avoid the problem
>>>> with the rounding errors (we are doing the calculations with double
>>>> precision, but we save with float
>>>> precision).
>>>>
>>>> I stopped  searching when I saw fewer holes in the result of the
>>>> performance
>>>> branch
>>>> than in the result of trunk.
>>>>
>>>> The errors in java.awt.geom.Area are complex. I think one has different
>>>> options to calculate the intersection of two areas a1 and a2:
>>>> a) Area x = new Area(a1); x.intersect(a2)
>>>> b) Area x = new Area(a2); x.intersect(a1)
>>>> c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);
>>>>
>>>> In an ideal world I would expect to get exactly the same result for
>>>> these
>>>> three calculations, but sometimes the real results are not equal. The
>>>> same
>>>> problem occurs when we add the areas again in the BoundaryCoverageUtil.
>>>>
>>>> Maybe we can get rid of a few more of the holes if we manage to do all
>>>> calculations with doubles, but probably even doubles will not solve all
>>>> problems.
>>>>
>>>> The question is if we should invest time for that. I think the holes are
>>>> purely cosmetic, if you really manage to query BoundaryQuadTree for a
>>>> point
>>>> that lies within such a hole, it will return the result for a point that
>>>> is
>>>> next to it, I think this is good enough.
>>>>
>>>> What do you mean?
>>>>
>>>> Gerd
>>>>
>>>>
>>>>
>>>> WanMil wrote
>>>>>
>>>>> Hi Gerd,
>>>>>
>>>>> thanks for the patch. I have commited it although I still see the
>>>>> problem. I am not sure if the holes are at the same place but there are
>>>>> some of them which I cannot explain with wrong boundary data.
>>>>>
>>>>> Can you please recheck that?
>>>>>
>>>>> Thanks!
>>>>> WanMil
>>>>>
>>>>>> Hi WanMil,
>>>>>>
>>>>>> attached is a fix for this problem.
>>>>>>
>>>>>> Please, can you review if the UnusedElementsRemoverHook is still
>>>>>> useful?
>>>>>> With my test data, it is slowing down mkgmap a little bit and I also
>>>>>> see
>>>>>> a different result for one tile in the UK when I disable it.
>>>>>>
>>>>>> Gerd
>>>>>>
>>>>>>    >   Date: Thu, 15 Mar 2012 21:11:02 +0100
>>>>>>    >   From: wmgcnfg@
>>>>>>    >   To: mkgmap-dev@.org
>>>>>>    >   Subject: [mkgmap-dev] Small holes in boundary coverage
>>>>>>    >
>>>>>>    >   Hi,
>>>>>>    >
>>>>>>    >   I have compiled world bounds using the performance branch. They
>>>>>> can be
>>>>>>    >   downloaded from
>>>>>> http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>>>>>>    >
>>>>>>    >   I have checked the coverage of different admin_levels with the
>>>>>>    >   BoundaryCoverageUtil. The result for admin_level=2 can be
>>>>>> downloaded
>>>>>>    >   from http://files.mkgmap.org.uk/detail/61.
>>>>>>    >
>>>>>>    >   Some countries are missing (USA, Canada, some african countries).
>>>>>> I
>>>>>>    >   assume that their boundaries were/are corrupt.
>>>>>>    >
>>>>>>    >   The more interesting thing is that there are small holes
>>>>>> throughout
>>>>>> the
>>>>>>    >   world in admin_level=2. They seem to be created by the bounds
>>>>>>    >   preparation and should not be there. Gerd, can you please have a
>>>>>> look
>>>>>> on it?
>>>>>>    >
>>>>>>    >   Thanks!
>>>>>>    >   WanMil
>>>>>>    >   _______________________________________________
>>>>>>    >   mkgmap-dev mailing list
>>>>>>    >   mkgmap-dev@.org
>>>>>>    >   http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> mkgmap-dev mailing list
>>>>>> mkgmap-dev@.org
>>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>>
>>>>> _______________________________________________
>>>>> mkgmap-dev mailing list
>>>>> mkgmap-dev@.org
>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5597147.html
>>>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev@.org
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5604371.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

I've analysed the problem and found two different reasons.
1) delta coding for doubles sometimes involves rounding errors:
delta = lat-oldLat;
if (oldLat + delta != lat) System.out.println("delta coding causes rounding");
This happens more often when the delta is near 0 (e.g. 60000-59999.99999999999999),
I thinks this also explains why you saw more holes around the Greenwich meridian.

This case is now signaled with a Double.POSITIVE_INFINITY and handled by
the read routine (it uses the next value without applying the delta).

2) Conversion between Area and Path2D.Double is not reversible. Sample:
                Path2D.Double path = new Path2D.Double(area);
                Area testArea = new Area(path);
if area contains only those holes (one or more), testArea might be empty.
I've used this knowledge to re-implement some tests in BoundaryQuadTree.makeDistinct()

With my test data these changes produce a result that has fewer holes than trunk
and no holes that I could not explain with OSM data.
(I've tested latest africa.osm.pbf)

I've also changed the routines that read and write the varying double format to avoid
the Long.reverse() operation.

One has to create boundary files again.

Gerd

boundary_double_v2.patch




WanMil wrote
Hi Gerd,

thanks a lot.
Unfortunately the new patched code creates more holes than the current
SVN code. The BoundaryCoverageUtil creates 1000 separate polygons for
admin_level=2 instead of 640.

I have uploaded the results of the BoundaryCoverageUtil to
http://www.navmaps.eu/wanmil/coverage_2.zip.

It seems that there are many holes around the Greenwich meridian. Can
you have a look on it and check if old or new code is better?

Anyhow I aggree that the holes are more a cosmetic problem. But maybe
it's worthy to invest the time to know exactly what happens there to be
sure that there are no other problems.

Thanks!
WanMil

> Hi WanMil,
>
> attached is a patch that changes all boundary routines to use double
> precision.
> The file format has changed again, so one has to create new boundaries.
>
> http://gis.19327.n5.nabble.com/file/n5604371/boundary_double_v1.patch
> boundary_double_v1.patch
>
> Although it uses double precision, the new format produces smaller files
> because
> it uses delta coding for the boundary coordinates.
>
> I found a strange error in Area.subtract(). For two given areas a1 and a2
> lying within
> a bbox, a1.subtract(a2) may result in an area lying outside of the bbox.
> I've added a test for that.
> The performance is almost equal, the newer version  may be a bit faster.
>
> Gerd
>
>
>
> GerdP wrote
>>
>> Hi WanMil,
>>
>> I can try writing the RAW format with double precision. Do you have a
>> sample input that
>> creates a hole? I tried africa and asia and found none, the europe file is
>> too large for my
>> machine.
>> We may also try to save the quadtree data with doubles without getting
>> much larger files if we use a compressed format (e.g. the delta coding
>> used in pbf or o5m).
>>
>> Gerd
>>
>>
>> WanMil wrote
>>>
>>> Hi Gerd,
>>>
>>> your conclusion sound very reasonable. The number of holes has been
>>> reduced quite a lot after your last patch and if your example a) b) and
>>> c) are not equal all the time there is no chance to avoid such holes
>>> completely.
>>>
>>> Just a quick idea: Would it improve (and reduce the number of holes) if
>>> the intermediate bounds format which is written first uses double
>>> instead of float precision? I think it doesn't really matter if the
>>> intermediate format uses more space on disk?!
>>>
>>> I will try to create updated bounds files during the weekend and then we
>>> can start a one week test phase with more users before merging back to
>>> trunk.
>>>
>>> WanMil
>>>
>>>> Hi WanMil,
>>>>
>>>> I found three reasons for these holes: errors in BoundaryQuadTree,
>>>> "errors"
>>>> in java.awt.geom.Area and rounding erros.
>>>>
>>>> My last patches fixed the errors that I found in BoundaryQuadTree, I
>>>> also
>>>> tried to avoid the problem
>>>> with the rounding errors (we are doing the calculations with double
>>>> precision, but we save with float
>>>> precision).
>>>>
>>>> I stopped  searching when I saw fewer holes in the result of the
>>>> performance
>>>> branch
>>>> than in the result of trunk.
>>>>
>>>> The errors in java.awt.geom.Area are complex. I think one has different
>>>> options to calculate the intersection of two areas a1 and a2:
>>>> a) Area x = new Area(a1); x.intersect(a2)
>>>> b) Area x = new Area(a2); x.intersect(a1)
>>>> c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);
>>>>
>>>> In an ideal world I would expect to get exactly the same result for
>>>> these
>>>> three calculations, but sometimes the real results are not equal. The
>>>> same
>>>> problem occurs when we add the areas again in the BoundaryCoverageUtil.
>>>>
>>>> Maybe we can get rid of a few more of the holes if we manage to do all
>>>> calculations with doubles, but probably even doubles will not solve all
>>>> problems.
>>>>
>>>> The question is if we should invest time for that. I think the holes are
>>>> purely cosmetic, if you really manage to query BoundaryQuadTree for a
>>>> point
>>>> that lies within such a hole, it will return the result for a point that
>>>> is
>>>> next to it, I think this is good enough.
>>>>
>>>> What do you mean?
>>>>
>>>> Gerd
>>>>
>>>>
>>>>
>>>> WanMil wrote
>>>>>
>>>>> Hi Gerd,
>>>>>
>>>>> thanks for the patch. I have commited it although I still see the
>>>>> problem. I am not sure if the holes are at the same place but there are
>>>>> some of them which I cannot explain with wrong boundary data.
>>>>>
>>>>> Can you please recheck that?
>>>>>
>>>>> Thanks!
>>>>> WanMil
>>>>>
>>>>>> Hi WanMil,
>>>>>>
>>>>>> attached is a fix for this problem.
>>>>>>
>>>>>> Please, can you review if the UnusedElementsRemoverHook is still
>>>>>> useful?
>>>>>> With my test data, it is slowing down mkgmap a little bit and I also
>>>>>> see
>>>>>> a different result for one tile in the UK when I disable it.
>>>>>>
>>>>>> Gerd
>>>>>>
>>>>>>    >   Date: Thu, 15 Mar 2012 21:11:02 +0100
>>>>>>    >   From: wmgcnfg@
>>>>>>    >   To: mkgmap-dev@.org
>>>>>>    >   Subject: [mkgmap-dev] Small holes in boundary coverage
>>>>>>    >
>>>>>>    >   Hi,
>>>>>>    >
>>>>>>    >   I have compiled world bounds using the performance branch. They
>>>>>> can be
>>>>>>    >   downloaded from
>>>>>> http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip.
>>>>>>    >
>>>>>>    >   I have checked the coverage of different admin_levels with the
>>>>>>    >   BoundaryCoverageUtil. The result for admin_level=2 can be
>>>>>> downloaded
>>>>>>    >   from http://files.mkgmap.org.uk/detail/61.
>>>>>>    >
>>>>>>    >   Some countries are missing (USA, Canada, some african countries).
>>>>>> I
>>>>>>    >   assume that their boundaries were/are corrupt.
>>>>>>    >
>>>>>>    >   The more interesting thing is that there are small holes
>>>>>> throughout
>>>>>> the
>>>>>>    >   world in admin_level=2. They seem to be created by the bounds
>>>>>>    >   preparation and should not be there. Gerd, can you please have a
>>>>>> look
>>>>>> on it?
>>>>>>    >
>>>>>>    >   Thanks!
>>>>>>    >   WanMil
>>>>>>    >   _______________________________________________
>>>>>>    >   mkgmap-dev mailing list
>>>>>>    >   mkgmap-dev@.org
>>>>>>    >   http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> mkgmap-dev mailing list
>>>>>> mkgmap-dev@.org
>>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>>
>>>>> _______________________________________________
>>>>> mkgmap-dev mailing list
>>>>> mkgmap-dev@.org
>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5597147.html
>>>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev@.org
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5604371.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
In reply to this post by Gerd Petermann
>
> Please, can you review if the UnusedElementsRemoverHook is still useful?
> With my test data, it is slowing down mkgmap a little bit and I also see
> a different result for one tile in the UK when I disable it.
>
> Gerd
>

Gerd,

I cannot reproduce that the UnusedElementsRemoverHook does not improve
the speed of mkgmap. Can you please try again?

If you see a different result it should be analysed. So please post your
tile and your mkgmap parameters and all the details so that I can check
that.

WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

I've tried again.
Reg. the performance improvements I see only small differences, I guess that's because I use
splitter with default overlap of 2000. I assume the greater the overlap the greater is the
improvement of the UnusedElementsRemoverHook ?

reg. different result:
I've uploaded the tile (sorry, it is very big):
http://files.mkgmap.org.uk/download/65/63240022.osm.pbf
I can reproduce the problem with trunk (r2263) (and my identical_output.patch) using the following parms:
java -Xmx1600m -Xms1600m -jar mkgmap.jar --remove-short-arcs --route --preserve-element-order 63240022.osm.pbf

If I comment the call of UnusedElementsRemoverHook I get a different output file.
I hope you can reproduce it?

Gerd


WanMil wrote
>
> Please, can you review if the UnusedElementsRemoverHook is still useful?
> With my test data, it is slowing down mkgmap a little bit and I also see
> a different result for one tile in the UK when I disable it.
>
> Gerd
>

Gerd,

I cannot reproduce that the UnusedElementsRemoverHook does not improve
the speed of mkgmap. Can you please try again?

If you see a different result it should be analysed. So please post your
tile and your mkgmap parameters and all the details so that I can check
that.

WanMil
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
Hi Gerd,

can you please post again the identical output patch? I cannot find it
on my computer and the patch referenced by nabble cannot be found
(http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q4/013223.html).

Thanks!
WanMil

> Hi WanMil,
>
> I've tried again.
> Reg. the performance improvements I see only small differences, I guess
> that's because I use
> splitter with default overlap of 2000. I assume the greater the overlap the
> greater is the
> improvement of the UnusedElementsRemoverHook ?
>
> reg. different result:
> I've uploaded the tile (sorry, it is very big):
> http://files.mkgmap.org.uk/download/65/63240022.osm.pbf
> I can reproduce the problem with trunk (r2263) (and my
> identical_output.patch) using the following parms:
> java -Xmx1600m -Xms1600m -jar mkgmap.jar --remove-short-arcs --route
> --preserve-element-order 63240022.osm.pbf
>
> If I comment the call of UnusedElementsRemoverHook I get a different output
> file.
> I hope you can reproduce it?
>
> Gerd
>
>
>
> WanMil wrote
>>
>>>
>>> Please, can you review if the UnusedElementsRemoverHook is still useful?
>>> With my test data, it is slowing down mkgmap a little bit and I also see
>>> a different result for one tile in the UK when I disable it.
>>>
>>> Gerd
>>>
>>
>> Gerd,
>>
>> I cannot reproduce that the UnusedElementsRemoverHook does not improve
>> the speed of mkgmap. Can you please try again?
>>
>> If you see a different result it should be analysed. So please post your
>> tile and your mkgmap parameters and all the details so that I can check
>> that.
>>
>> WanMil
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5622393.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

I've created a new one:
identical_output_v2.patch

Gerd


WanMil wrote
Hi Gerd,

can you please post again the identical output patch? I cannot find it
on my computer and the patch referenced by nabble cannot be found
(http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q4/013223.html).

Thanks!
WanMil

> Hi WanMil,
>
> I've tried again.
> Reg. the performance improvements I see only small differences, I guess
> that's because I use
> splitter with default overlap of 2000. I assume the greater the overlap the
> greater is the
> improvement of the UnusedElementsRemoverHook ?
>
> reg. different result:
> I've uploaded the tile (sorry, it is very big):
> http://files.mkgmap.org.uk/download/65/63240022.osm.pbf
> I can reproduce the problem with trunk (r2263) (and my
> identical_output.patch) using the following parms:
> java -Xmx1600m -Xms1600m -jar mkgmap.jar --remove-short-arcs --route
> --preserve-element-order 63240022.osm.pbf
>
> If I comment the call of UnusedElementsRemoverHook I get a different output
> file.
> I hope you can reproduce it?
>
> Gerd
>
>
>
> WanMil wrote
>>
>>>
>>> Please, can you review if the UnusedElementsRemoverHook is still useful?
>>> With my test data, it is slowing down mkgmap a little bit and I also see
>>> a different result for one tile in the UK when I disable it.
>>>
>>> Gerd
>>>
>>
>> Gerd,
>>
>> I cannot reproduce that the UnusedElementsRemoverHook does not improve
>> the speed of mkgmap. Can you please try again?
>>
>> If you see a different result it should be analysed. So please post your
>> tile and your mkgmap parameters and all the details so that I can check
>> that.
>>
>> WanMil
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5622393.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
Hi Gerd,

I see a difference in the turn restrictions. It seems that they are not
added in sorted order.

I've printed out all elements passing the style converter. When printing
out the restrictions in a sorted order there is no more difference (in
my printout). So I assume that the binary output is also the same.

I am not sure why the difference does not appear in other tiles.

WanMil

> Hi WanMil,
>
> I've created a new one:
> http://gis.19327.n5.nabble.com/file/n5624815/identical_output_v2.patch
> identical_output_v2.patch
>
> Gerd
>
>
>
> WanMil wrote
>>
>> Hi Gerd,
>>
>> can you please post again the identical output patch? I cannot find it
>> on my computer and the patch referenced by nabble cannot be found
>> (http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q4/013223.html).
>>
>> Thanks!
>> WanMil
>>
>>> Hi WanMil,
>>>
>>> I've tried again.
>>> Reg. the performance improvements I see only small differences, I guess
>>> that's because I use
>>> splitter with default overlap of 2000. I assume the greater the overlap
>>> the
>>> greater is the
>>> improvement of the UnusedElementsRemoverHook ?
>>>
>>> reg. different result:
>>> I've uploaded the tile (sorry, it is very big):
>>> http://files.mkgmap.org.uk/download/65/63240022.osm.pbf
>>> I can reproduce the problem with trunk (r2263) (and my
>>> identical_output.patch) using the following parms:
>>> java -Xmx1600m -Xms1600m -jar mkgmap.jar --remove-short-arcs --route
>>> --preserve-element-order 63240022.osm.pbf
>>>
>>> If I comment the call of UnusedElementsRemoverHook I get a different
>>> output
>>> file.
>>> I hope you can reproduce it?
>>>
>>> Gerd
>>>
>>>
>>>
>>> WanMil wrote
>>>>
>>>>>
>>>>> Please, can you review if the UnusedElementsRemoverHook is still
>>>>> useful?
>>>>> With my test data, it is slowing down mkgmap a little bit and I also
>>>>> see
>>>>> a different result for one tile in the UK when I disable it.
>>>>>
>>>>> Gerd
>>>>>
>>>>
>>>> Gerd,
>>>>
>>>> I cannot reproduce that the UnusedElementsRemoverHook does not improve
>>>> the speed of mkgmap. Can you please try again?
>>>>
>>>> If you see a different result it should be analysed. So please post your
>>>> tile and your mkgmap parameters and all the details so that I can check
>>>> that.
>>>>
>>>> WanMil
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev@.org
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5622393.html
>>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p5624815.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

Gerd Petermann
Hi WanMil,

okay, so let's assume the hook is working as it should.
Do you see a simple way to change the identical_output.patch so that the
turn restrictions are processed in a fixed order?

Gerd

> Date: Sun, 8 Apr 2012 21:51:45 +0200

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] Small holes in boundary coverage
>
> Hi Gerd,
>
> I see a difference in the turn restrictions. It seems that they are not
> added in sorted order.
>
> I've printed out all elements passing the style converter. When printing
> out the restrictions in a sorted order there is no more difference (in
> my printout). So I assume that the binary output is also the same.
>
> I am not sure why the difference does not appear in other tiles.
>
> WanMil
>


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Small holes in boundary coverage

WanMil
Hi Gerd,

yes I think that should be possible.
AFAIK in method end() of class StyledConverter the first lines

Collection<List<RestrictionRelation>> lists = restrictions.values();
for (List<RestrictionRelation> l : lists) {
        for (RestrictionRelation rr : l) {
                rr.addRestriction(collector);
        }
}

must be changed in such a way that it is processed in a definite order.

WanMil

> Hi WanMil,
>
> okay, so let's assume the hook is working as it should.
> Do you see a simple way to change the identical_output.patch so that the
> turn restrictions are processed in a fixed order?
>
> Gerd
>
>  > Date: Sun, 8 Apr 2012 21:51:45 +0200
>  > From: [hidden email]
>  > To: [hidden email]
>  > Subject: Re: [mkgmap-dev] Small holes in boundary coverage
>  >
>  > Hi Gerd,
>  >
>  > I see a difference in the turn restrictions. It seems that they are not
>  > added in sorted order.
>  >
>  > I've printed out all elements passing the style converter. When printing
>  > out the restrictions in a sorted order there is no more difference (in
>  > my printout). So I assume that the binary output is also the same.
>  >
>  > I am not sure why the difference does not appear in other tiles.
>  >
>  > WanMil
>  >
>
>
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev