Problem with splitter

classic Classic list List threaded Threaded
36 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Problem with splitter

Matteo Gottardi-2
Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
using the standard splitter-r200 options.

The area is at http://osm.org/go/0CmXpRz , and at
http://www.gomatteo.net/10.jpg you can see how it look in
QlandkarteGT.

The osm data seem ok, is a splitter problem?
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
Hi Matteo,

can you please describe the problem and what it is that
makes you believe that this is a splitter problem?

Gerd

Matteo Gottardi-2 wrote
Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
using the standard splitter-r200 options.

The area is at http://osm.org/go/0CmXpRz , and at
http://www.gomatteo.net/10.jpg you can see how it look in
QlandkarteGT.

The osm data seem ok, is a splitter problem?
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

railrun
Looks like a coastline-problem.
Which parameter you use for mkgmap?

//Martin

Am 11.03.2012 um 22:10 schrieb GerdP:

> Hi Matteo,
>
> can you please describe the problem and what it is that
> makes you believe that this is a splitter problem?
>
> Gerd
>
>
> Matteo Gottardi-2 wrote
>>
>> Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
>> using the standard splitter-r200 options.
>>
>> The area is at http://osm.org/go/0CmXpRz , and at
>> http://www.gomatteo.net/10.jpg you can see how it look in
>> QlandkarteGT.
>>
>> The osm data seem ok, is a splitter problem?
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5555911.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Matteo Gottardi
2012/3/11 Martin <[hidden email]>:
> Looks like a coastline-problem.
> Which parameter you use for mkgmap?

The options are:
----- BEGIN -----
code-page=1252
latin1
reduce-point-density=4
reduce-point-density-polygon=8
min-size-polygon=8
remove-short-arcs
merge-lines
add-pois-to-areas
pois-to-areas-placement
style-file=/home/matteo/opt/mkgmap_styles/teo_style
overview-mapname=teo
family-name=teo
mapname=63240001
family-id=6324
series-name=teo
country-name=Italia
country-abbr=IT
area-name=Italia
bounds=/home/matteo/opt/mkgmap_styles/bounds/europe
location-autofill=bounds
generate-sea=multipolygon
coastlinefile=/home/matteo/opt/mkgmap_styles/coastlines_europe-latest.osm.pbf
ignore-maxspeeds
ignore-turn-restrictions
tdbfile
----- END -----

The style file is the default one with some minor changes.

The lake is a multipoligon, tagged natural=water in the relation.

If it can help, you can download the bad tile at
http://www.gomatteo.net/63240018.osm.pbf , or download the italy file
from http://download.geofabrik.de/osm/europe/italy.osm.pbf and split
it.

--
* Matteo Gottardi | [hidden email]
* ICQ UIN 20381372
* Linux - the choice of a GNU generation
* GPG Fingerprint:
* B9EE 108F 52C8 D50C B667 B1F2 AB56 8A01 BA3D 36A1
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Wolfgang Hammel
In reply to this post by railrun
Hi,

yes you are right, this is a splitter problem which
is already known.

As you can see in your example, the lake is split in a way that requires it
to be split more than once along the bounding rectangle.

see : http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q3/012119.html
and http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q3/012120.html

This prob. may occur, when the polygon to be split has more than two
intersections with the bounding
rectangle and consists of some ways that have all their nodes outside
the clipping rectangle.
As lakes and riverbanks may have curved outlines this can occasionally
occur when the
bounding rectangle intersects the polygon in a way such that the clipped
polygon falls into more than one
(or probably two) parts.

I usually take the following manual procedure to circumvent the problem
when it occurs.

1. Run Tile splitter once and create a map
2. check the map for floodings along large riverbanks or lakes
3. edit the file areas.list (back up the original one createt by splitter)
     in order to move the border of the tiles away from the offending
riverbank or lake.
    But take care: you have to modify the corner coordinates of at least two
   areas in a consistent way in order to get a map without holes
   The modification of the coordinates should be done in multiples of 0x800.
3a. If needed you can visualize the splitting rectangles when you use
the --write-kml option in step 1
       paste the complete contents of the generated kml-file here:
http://display-kml.appspot.com/
4. rerun splitter using the option --split-file=<edited areas.list>
5. I use the same edited areas.list later again after I have downloaded
new OSM-data
    so this manual procedure has to be done only once for a certain area

Maybe someone has enough time to enhance TileSpltter, I unfortunately
don't have at the moment.

But anyhow: All those working on splitter and mkgmap are doing a great
job. Thank you!

Wolfgang


Am 11.03.2012 22:16, schrieb Martin:

> Looks like a coastline-problem.
> Which parameter you use for mkgmap?
>
> //Martin
>
> Am 11.03.2012 um 22:10 schrieb GerdP:
>
>> Hi Matteo,
>>
>> can you please describe the problem and what it is that
>> makes you believe that this is a splitter problem?
>>
>> Gerd
>>
>>
>> Matteo Gottardi-2 wrote
>>> Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
>>> using the standard splitter-r200 options.
>>>
>>> The area is at http://osm.org/go/0CmXpRz , and at
>>> http://www.gomatteo.net/10.jpg you can see how it look in
>>> QlandkarteGT.
>>>
>>> The osm data seem ok, is a splitter problem?
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev@.org
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>
>> --
>> View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5555911.html
>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Matteo Gottardi
In reply to this post by Gerd Petermann
2012/3/11 GerdP <[hidden email]>:
> Hi Matteo,
>
> can you please describe the problem and what it is that
> makes you believe that this is a splitter problem?

Hi, I don't speak english very well, so maybe a screenshot can help :)

If you look at http://www.gomatteo.net/out.jpeg you can see the
generated map. I marked with the red color the shape of the lake
(taken from the osm data).
As you can see there is some flooding, and the flooding end exactly at
the end of the splitted tile.

Thank you for your help,
  Teo
--
* Matteo Gottardi | [hidden email]
* ICQ UIN 20381372
* Linux - the choice of a GNU generation
* GPG Fingerprint:
* B9EE 108F 52C8 D50C B667 B1F2 AB56 8A01 BA3D 36A1
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
In reply to this post by Wolfgang Hammel
Hi Wolfgang,

I'll try to find a solution once the remaining problems in the performance branch are solved.

Gerd


> Date: Mon, 12 Mar 2012 00:14:42 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] Problem with splitter
>
> Hi,
>
> yes you are right, this is a splitter problem which
> is already known.
>
> As you can see in your example, the lake is split in a way that requires it
> to be split more than once along the bounding rectangle.
>
> see : http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q3/012119.html
> and http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q3/012120.html
>
> This prob. may occur, when the polygon to be split has more than two
> intersections with the bounding
> rectangle and consists of some ways that have all their nodes outside
> the clipping rectangle.
> As lakes and riverbanks may have curved outlines this can occasionally
> occur when the
> bounding rectangle intersects the polygon in a way such that the clipped
> polygon falls into more than one
> (or probably two) parts.
>
> I usually take the following manual procedure to circumvent the problem
> when it occurs.
>
> 1. Run Tile splitter once and create a map
> 2. check the map for floodings along large riverbanks or lakes
> 3. edit the file areas.list (back up the original one createt by splitter)
> in order to move the border of the tiles away from the offending
> riverbank or lake.
> But take care: you have to modify the corner coordinates of at least two
> areas in a consistent way in order to get a map without holes
> The modification of the coordinates should be done in multiples of 0x800.
> 3a. If needed you can visualize the splitting rectangles when you use
> the --write-kml option in step 1
> paste the complete contents of the generated kml-file here:
> http://display-kml.appspot.com/
> 4. rerun splitter using the option --split-file=<edited areas.list>
> 5. I use the same edited areas.list later again after I have downloaded
> new OSM-data
> so this manual procedure has to be done only once for a certain area
>
> Maybe someone has enough time to enhance TileSpltter, I unfortunately
> don't have at the moment.
>
> But anyhow: All those working on splitter and mkgmap are doing a great
> job. Thank you!
>
> Wolfgang
>
>
> Am 11.03.2012 22:16, schrieb Martin:
> > Looks like a coastline-problem.
> > Which parameter you use for mkgmap?
> >
> > //Martin
> >
> > Am 11.03.2012 um 22:10 schrieb GerdP:
> >
> >> Hi Matteo,
> >>
> >> can you please describe the problem and what it is that
> >> makes you believe that this is a splitter problem?
> >>
> >> Gerd
> >>
> >>
> >> Matteo Gottardi-2 wrote
> >>> Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
> >>> using the standard splitter-r200 options.
> >>>
> >>> The area is at http://osm.org/go/0CmXpRz , and at
> >>> http://www.gomatteo.net/10.jpg you can see how it look in
> >>> QlandkarteGT.
> >>>
> >>> The osm data seem ok, is a splitter problem?
> >>> _______________________________________________
> >>> mkgmap-dev mailing list
> >>> mkgmap-dev@.org
> >>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >>>
> >>
> >> --
> >> View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5555911.html
> >> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> >> _______________________________________________
> >> mkgmap-dev mailing list
> >> [hidden email]
> >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >>
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
In reply to this post by Matteo Gottardi-2
Hi Teo,

I tried to reproduce the problem with mkgmap trunk version r2248, but I get different results, esp. I don't see this flooding.
I am using coastlines_europe-120128.osm.pbf, maybe your file is older?

Gerd



Matteo Gottardi-2 wrote
Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
using the standard splitter-r200 options.

The area is at http://osm.org/go/0CmXpRz , and at
http://www.gomatteo.net/10.jpg you can see how it look in
QlandkarteGT.

The osm data seem ok, is a splitter problem?
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Thorsten Kukuk
On Mon, Mar 12, GerdP wrote:

> Hi Teo,
>
> I tried to reproduce the problem with mkgmap trunk version r2248, but I get
> different results, esp. I don't see this flooding.
> I am using coastlines_europe-120128.osm.pbf, maybe your file is older?

You need the exact same tiles, else you will not see it.
I think to reproduce you need the areas.list from Teo.

  Thorsten

> Matteo Gottardi-2 wrote
> >
> > Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
> > using the standard splitter-r200 options.
> >
> > The area is at http://osm.org/go/0CmXpRz , and at
> > http://www.gomatteo.net/10.jpg you can see how it look in
> > QlandkarteGT.
> >
> > The osm data seem ok, is a splitter problem?
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev@.org
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5556974.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Thorsten Kukuk, Project Manager/Release Manager SLES
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
Hi Thorsten,

I tried with his *.cfg and his 63240018.osm.pbf
With splitter r200 I produced an identical 63240018.osm.pbf

Why do you think that I should use his areas.list ?

Gerd

> Date: Mon, 12 Mar 2012 10:03:21 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] Problem with splitter
>
> On Mon, Mar 12, GerdP wrote:
>
> > Hi Teo,
> >
> > I tried to reproduce the problem with mkgmap trunk version r2248, but I get
> > different results, esp. I don't see this flooding.
> > I am using coastlines_europe-120128.osm.pbf, maybe your file is older?
>
> You need the exact same tiles, else you will not see it.
> I think to reproduce you need the areas.list from Teo.
>
> Thorsten
>
> > Matteo Gottardi-2 wrote
> > >
> > > Hi, I noticed a problem splitting the geofabrik pbf extract of Italy
> > > using the standard splitter-r200 options.
> > >
> > > The area is at http://osm.org/go/0CmXpRz , and at
> > > http://www.gomatteo.net/10.jpg you can see how it look in
> > > QlandkarteGT.
> > >
> > > The osm data seem ok, is a splitter problem?
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev@.org
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > >
> >
> >
> > --
> > View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5556974.html
> > Sent from the Mkgmap Development mailing list archive at Nabble.com.
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
> --
> Thorsten Kukuk, Project Manager/Release Manager SLES
> SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Matteo Gottardi
In reply to this post by Gerd Petermann
2012/3/12 GerdP <[hidden email]>:
> Hi Teo,
>
> I tried to reproduce the problem with mkgmap trunk version r2248, but I get
> different results, esp. I don't see this flooding.
> I am using coastlines_europe-120128.osm.pbf, maybe your file is older?

Hi Gerd,
my coastlines file was the same as yours, only with a different name :)

I did some tests. The results were a bit different because of my typ
and style files.
Using no typ file, the default style file and passing only
--generate-sea=multipolygon
--coastlinefile=coastlines_europe-120128.osm.pbf the result look like
this: http://www.gomatteo.net/17.png

PS: I would like to thank all the developers who spends their time
working on this great project, without mkgmap my gpsmap60c would be
useless :)

--
* Matteo Gottardi | [hidden email]
* ICQ UIN 20381372
* Linux - the choice of a GNU generation
* GPG Fingerprint:
* B9EE 108F 52C8 D50C B667 B1F2 AB56 8A01 BA3D 36A1
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
Hi Matteo,

okay, I am able to reproduce the problem (also without the coastfile parameter).
The log shows some warnings for  relation 541757 (the Lago di Como) , so I should be
able to understand what's happening and why it fails.

Gerd


Matteo Gottardi wrote
2012/3/12 GerdP <[hidden email]>:
> Hi Teo,
>
> I tried to reproduce the problem with mkgmap trunk version r2248, but I get
> different results, esp. I don't see this flooding.
> I am using coastlines_europe-120128.osm.pbf, maybe your file is older?

Hi Gerd,
my coastlines file was the same as yours, only with a different name :)

I did some tests. The results were a bit different because of my typ
and style files.
Using no typ file, the default style file and passing only
--generate-sea=multipolygon
--coastlinefile=coastlines_europe-120128.osm.pbf the result look like
this: http://www.gomatteo.net/17.png

PS: I would like to thank all the developers who spends their time
working on this great project, without mkgmap my gpsmap60c would be
useless :)

--
* Matteo Gottardi | [hidden email]
* ICQ UIN 20381372
* Linux - the choice of a GNU generation
* GPG Fingerprint:
* B9EE 108F 52C8 D50C B667 B1F2 AB56 8A01 BA3D 36A1
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Wolfgang Hammel
Hi Gerd,

when I had the problem some time ago, I did some rough checking on
splitters output.
What I know so far is, that splitter removes all the ways from a certain
tile that have no
node inside this tile.
The problem arises when a tile boundary divides a multipolyon that
consist of normally
a lot of different ways. Tile splitter includes the complete relation
for that multipolygon
in the output including all the references to the ways that
mulitipolygon originally consisted of.
But as some of the ways are removed from the output, the multipolygon is
corrupted and
mkgmap is later no more able to correctly reconstruct the part (or
parts) of the multipolygon
that fall inside the tile.

Wolfgang

Am 12.03.2012 16:06, schrieb GerdP:

> Hi Matteo,
>
> okay, I am able to reproduce the problem (also without the coastfile
> parameter).
> The log shows some warnings for  relation 541757 (the Lago di Como) , so I
> should be
> able to understand what's happening and why it fails.
>
> Gerd
>
>
>
> Matteo Gottardi wrote
>> 2012/3/12 GerdP&lt;gpetermann_muenchen@&gt;:
>>> Hi Teo,
>>>
>>> I tried to reproduce the problem with mkgmap trunk version r2248, but I
>>> get
>>> different results, esp. I don't see this flooding.
>>> I am using coastlines_europe-120128.osm.pbf, maybe your file is older?
>> Hi Gerd,
>> my coastlines file was the same as yours, only with a different name :)
>>
>> I did some tests. The results were a bit different because of my typ
>> and style files.
>> Using no typ file, the default style file and passing only
>> --generate-sea=multipolygon
>> --coastlinefile=coastlines_europe-120128.osm.pbf the result look like
>> this: http://www.gomatteo.net/17.png
>>
>> PS: I would like to thank all the developers who spends their time
>> working on this great project, without mkgmap my gpsmap60c would be
>> useless :)
>>
>> --
>> * Matteo Gottardi | matgott@
>> * ICQ UIN 20381372
>> * Linux - the choice of a GNU generation
>> * GPG Fingerprint:
>> * B9EE 108F 52C8 D50C B667 B1F2 AB56 8A01 BA3D 36A1
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5558068.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
Hi Wolfgang,

yes, that' s exactly what happens. I see three ways to solve this problem:
1) Enhance the logic in mkgmap that guesses how the missing ways completed the multipolygon, e.g. by adding a backtracking algorithmn (this is already suggested in the code).
2) Enhance splitter so that it writes all points and all ways of multipolygon to each tile.
3) Enhance splitter to write one extra output file that contains only the 1st and last point of each way that is part of a multipolygon, and create a method in mkgmap that looks for this data when
it doesn't find the way in the normal input. We need only the end points because we use the data only in cases where we know that they are outside of the bounding box. Maybe that can be done with osmfilter as well ?

I did not start coding, but I think option 3) should be easy to do and I hope it solves most
of the problems. Option 2) looks more difficult and will blow up tile sizes and CPU cost both in splitter and mkgmap. Option 1) can be done as well.

Does that sound reasonable?

Gerd

Wolfgang Hammel wrote
Hi Gerd,

when I had the problem some time ago, I did some rough checking on
splitters output.
What I know so far is, that splitter removes all the ways from a certain
tile that have no
node inside this tile.
The problem arises when a tile boundary divides a multipolyon that
consist of normally
a lot of different ways. Tile splitter includes the complete relation
for that multipolygon
in the output including all the references to the ways that
mulitipolygon originally consisted of.
But as some of the ways are removed from the output, the multipolygon is
corrupted and
mkgmap is later no more able to correctly reconstruct the part (or
parts) of the multipolygon
that fall inside the tile.

Wolfgang

Am 12.03.2012 16:06, schrieb GerdP:
> Hi Matteo,
>
> okay, I am able to reproduce the problem (also without the coastfile
> parameter).
> The log shows some warnings for  relation 541757 (the Lago di Como) , so I
> should be
> able to understand what's happening and why it fails.
>
> Gerd
>
>
>
> Matteo Gottardi wrote
>> 2012/3/12 GerdP<gpetermann_muenchen@>:
>>> Hi Teo,
>>>
>>> I tried to reproduce the problem with mkgmap trunk version r2248, but I
>>> get
>>> different results, esp. I don't see this flooding.
>>> I am using coastlines_europe-120128.osm.pbf, maybe your file is older?
>> Hi Gerd,
>> my coastlines file was the same as yours, only with a different name :)
>>
>> I did some tests. The results were a bit different because of my typ
>> and style files.
>> Using no typ file, the default style file and passing only
>> --generate-sea=multipolygon
>> --coastlinefile=coastlines_europe-120128.osm.pbf the result look like
>> this: http://www.gomatteo.net/17.png
>>
>> PS: I would like to thank all the developers who spends their time
>> working on this great project, without mkgmap my gpsmap60c would be
>> useless :)
>>
>> --
>> * Matteo Gottardi | matgott@
>> * ICQ UIN 20381372
>> * Linux - the choice of a GNU generation
>> * GPG Fingerprint:
>> * B9EE 108F 52C8 D50C B667 B1F2 AB56 8A01 BA3D 36A1
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/Problem-with-splitter-tp5555886p5558068.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Thorsten Kukuk
On Mon, Mar 12, GerdP wrote:

> 1) Enhance the logic in mkgmap that guesses how the missing ways completed
> the multipolygon, e.g. by adding a backtracking algorithmn (this is already
> suggested in the code).
> 2) Enhance splitter so that it writes all points and all ways of
> multipolygon to each tile.
> 3) Enhance splitter to write one extra output file that contains only the
> 1st and last point of each way that is part of a multipolygon, and create a
> method in mkgmap that looks for this data when
> it doesn't find the way in the normal input. We need only the end points
> because we use the data only in cases where we know that they are outside of
> the bounding box. Maybe that can be done with osmfilter as well ?
>
> I did not start coding, but I think option 3) should be easy to do and I
> hope it solves most
> of the problems. Option 2) looks more difficult and will blow up tile sizes
> and CPU cost both in splitter and mkgmap. Option 1) can be done as well.

I don't like Option 1) at all, guessing what the data was will
only create new problems.

I would prefer Option 2), even if it is more difficult, for several
reasons. You have the correct data. You cannot run into problems if
the extra output file from 3) get's out of sync to the tiles.
And every tile is self-contained, means you can mix them with others,
use only a subset, or give it away to somebody else without the need
to fiddle with other extra files, which have always to fit.

I'm not that sure option 2 will really blow up the tile size and
CPU cost. Today, most people creating a map use a very huge number
for --overlap to avoid this kind of problems. And how many multipolygons
are really that big that they go over several tiles?

  Thorsten

--
Thorsten Kukuk, Project Manager/Release Manager SLES
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

osm-8
In reply to this post by Gerd Petermann
Am 13.03.2012 06:59, schrieb GerdP:
> 3) Enhance splitter to write one extra output file that contains only the
> 1st and last point of each way that is part of a multipolygon, and create a
> method in mkgmap that looks for this data when
> it doesn't find the way in the normal input. We need only the end points
> because we use the data only in cases where we know that they are outside of
> the bounding box. Maybe that can be done with osmfilter as well ?
Would it be possible, that splitter closes the multipolygons after
cutting them?

Henning

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
Hi Henning,

well, I think this can be done, but I see two problems:
1) Splitter would have to write the data in OSM format, means, it has to invent some ways, nodes and the ids for them. I think we cannot simply invent nodes for existing OSM ways.
2) I think the data structures in splitter don't allow to detect unclosed multipolygons without massive changes.

Gerd

aighes-2 wrote
Am 13.03.2012 06:59, schrieb GerdP:
> 3) Enhance splitter to write one extra output file that contains only the
> 1st and last point of each way that is part of a multipolygon, and create a
> method in mkgmap that looks for this data when
> it doesn't find the way in the normal input. We need only the end points
> because we use the data only in cases where we know that they are outside of
> the bounding box. Maybe that can be done with osmfilter as well ?
Would it be possible, that splitter closes the multipolygons after
cutting them?

Henning

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Gerd Petermann
In reply to this post by Thorsten Kukuk
Hi Thorsten,

thanks for the qucick feedback.

> Date: Tue, 13 Mar 2012 09:07:18 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] Problem with splitter
>
> On Mon, Mar 12, GerdP wrote:
>
> > 1) Enhance the logic in mkgmap that guesses how the missing ways completed
> > the multipolygon, e.g. by adding a backtracking algorithmn (this is already
> > suggested in the code).
> > 2) Enhance splitter so that it writes all points and all ways of
> > multipolygon to each tile.
> > 3) Enhance splitter to write one extra output file that contains only the
> > 1st and last point of each way that is part of a multipolygon, and create a
> > method in mkgmap that looks for this data when
> > it doesn't find the way in the normal input. We need only the end points
> > because we use the data only in cases where we know that they are outside of
> > the bounding box. Maybe that can be done with osmfilter as well ?
> >
> > I did not start coding, but I think option 3) should be easy to do and I
> > hope it solves most
> > of the problems. Option 2) looks more difficult and will blow up tile sizes
> > and CPU cost both in splitter and mkgmap. Option 1) can be done as well.
>
> I don't like Option 1) at all, guessing what the data was will
> only create new problems.

okay. Anyway, as long as you don't use planet.osm as input to splitter, you'll
always have the risk that something is missing.

>
> I would prefer Option 2), even if it is more difficult, for several
> reasons. You have the correct data. You cannot run into problems if
> the extra output file from 3) get's out of sync to the tiles.
> And every tile is self-contained, means you can mix them with others,
> use only a subset, or give it away to somebody else without the need
> to fiddle with other extra files, which have always to fit.

okay, these are good points

>
> I'm not that sure option 2 will really blow up the tile size and
> CPU cost. Today, most people creating a map use a very huge number
> for --overlap to avoid this kind of problems. And how many multipolygons
> are really that big that they go over several tiles?

I fear the administrative boundaries and coastlines.

Gerd


>
> Thorsten
>
> --
> Thorsten Kukuk, Project Manager/Release Manager SLES
> SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

Thorsten Kukuk

Hi Gerd,

On Tue, Mar 13, Gerd Petermann wrote:

> > I'm not that sure option 2 will really blow up the tile size and
> > CPU cost. Today, most people creating a map use a very huge number
> > for --overlap to avoid this kind of problems. And how many multipolygons
> > are really that big that they go over several tiles?
>
> I fear the administrative boundaries and coastlines.

You are right, this could become a real problem :(

I think the coastlines are already incomplete for europe if you
don't work on the planet file. Since they are no longer closed,
would they really end in every tile?

Of course if you have an extract for great britain, or even
more worse, for complete north america. Especially in the last
case, you will have about 1500 tiles all containing the boundaries
and the coastline.

Maybe the solution would be to merge mkgmap and tilesplitter
and do the split on the processed mkgmap data. But I don't
know enough about how this works to know if this is doable
at all.

  Thorsten

--
Thorsten Kukuk, Project Manager/Release Manager SLES
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with splitter

osm-8
Maybe splitter could detect, if a tile is completely inside such a
"polygon" and then only add a rectangle with a negative ID (or parse for
highest used node/way-ID).

At all I think, that needed diskspace isn't a huge problem, if there is
no more guessing.

Henning

Am 13.03.2012 16:06, schrieb Thorsten Kukuk:

> Hi Gerd,
>
> On Tue, Mar 13, Gerd Petermann wrote:
>
>>> I'm not that sure option 2 will really blow up the tile size and
>>> CPU cost. Today, most people creating a map use a very huge number
>>> for --overlap to avoid this kind of problems. And how many multipolygons
>>> are really that big that they go over several tiles?
>> I fear the administrative boundaries and coastlines.
> You are right, this could become a real problem :(
>
> I think the coastlines are already incomplete for europe if you
> don't work on the planet file. Since they are no longer closed,
> would they really end in every tile?
>
> Of course if you have an extract for great britain, or even
> more worse, for complete north america. Especially in the last
> case, you will have about 1500 tiles all containing the boundaries
> and the coastline.
>
> Maybe the solution would be to merge mkgmap and tilesplitter
> and do the split on the processed mkgmap data. But I don't
> know enough about how this works to know if this is doable
> at all.
>
>    Thorsten
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
12