Possible problem with global search index since r3875

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible problem with global search index since r3875

Gerd Petermann
Hi all,

I've just noticed that I may have introduced a problem with r3875.
Road names containing special characters 0x1e and 0x1f are treated different now. I am not
sure where those characters appear in names. I think they are used to store so called double byte
characters within a single byte character string, at least that's what I remember from working
time with customers from Japan.
In the mkgmap code it is called a prefix, so maybe something completely different.
I did not yet find an input file where this occurs. Can anybody point me to one?

Gerd
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

popej
Hi Gerd,

cGPSmapper uses codes 0x1d - 0x1f as a special separators. 0x1f is used
for elevation, other should hide part of label depending on zoom.
Actually cGPSmapper's manual describe codes in *.mp source, but I guess
these values go directly to labels in compiled img.

--
Best regards,
Andrzej
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Gerd Petermann
Hi Andrzej,

thanks, that helps. I found another hint in imgformat-1.0.pdf which mentions these codes for 6 bit encoding.
I'll try to find out if that can be used with osm data as well.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Andrzej Popowski <[hidden email]>
Gesendet: Samstag, 8. April 2017 12:05:28
An: [hidden email]
Betreff: Re: [mkgmap-dev] Possible problem with global search index since r3875

Hi Gerd,

cGPSmapper uses codes 0x1d - 0x1f as a special separators. 0x1f is used
for elevation, other should hide part of label depending on zoom.
Actually cGPSmapper's manual describe codes in *.mp source, but I guess
these values go directly to labels in compiled img.

--
Best regards,
Andrzej
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Steve Ratcliffe
In reply to this post by Gerd Petermann

Hi

> I've just noticed that I may have introduced a problem with r3875.
> Road names containing special characters 0x1e and 0x1f are treated different now. I am not
> sure where those characters appear in names. I think they are used to store so called double byte

0x1e is a prefix separator and 0x1f a suffix separator.

They are used instead of a space in street names to separate the
main part of the name from Chemin, Rue, Road, Avenue etc.

If I use ']' for 0x1e and '[' for 0x1f, then streets can be written
as:

   Rue]Nicolo
   Chemin de]Pierre Froide
   Fernrodder[Strasse
   Street Lane[Road

This also affects searching in mapsource and the prefix/suffix parts
seem to be ignored.

For example if I want to search for 'Street Lane Road' (its a real
name!)
I type the letters 'street lane' and the full name is shown in the dropdown.
But if I add the 'r' as in 'street lane r', then it no longer
matches and the next matching name is shown in the dropdown, which
happens to be 'Street Lane Roundabout'.

There is also the sections in the index mdr30/31 and mdr32/33 which
are simple lists of prefixes and suffixes.  I do not know how they
are used.

In mapsource you have to pick from the dropdown list.  So to select
our favourite French road 'Chemain de Pierre Froide' you have to type
'pierre fr...' and select.  Starting from 'Chemain ...' does not
get you anywhere even if you type the whole name.  In BaseCamp,
you can type the whole name, and although it also will not show
in the dropdown, it will allow you to enter it and it will find it.

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Gerd Petermann
Hi Steve,

so I guess that I should revert some of the changes made in Mdr7 to make this work again (presuming that
the order produced by r3449 was okay for both variants (--x-split-name-index  on/off).
At the moment we have different versions in trunk and in the branch, and both might not work when this
feature is used.

Another point is the question how to sort/de-duplicate the index. I noticed that with option --x-split-name-index
the results shown in MapSource are sorted a bit unexpected.
Example: In Switzerland I found road names like
'14 Hauptstrasse' and nearby one named '13/14 Hauptstrasse'
When I type "14 Hau"  I want to find both entries, but I prefer to see '14 Hauptstrasse' first (topmost).
This depends on how we sort, maybe also on the order in which the names appear in the *.img files.

I continue experimenting with all this new knowledge later.

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Steve Ratcliffe <[hidden email]>
Gesendet: Samstag, 8. April 2017 14:10:07
An: [hidden email]
Betreff: Re: [mkgmap-dev] Possible problem with global search index since r3875

Hi

> I've just noticed that I may have introduced a problem with r3875.
> Road names containing special characters 0x1e and 0x1f are treated different now. I am not
> sure where those characters appear in names. I think they are used to store so called double byte

0x1e is a prefix separator and 0x1f a suffix separator.

They are used instead of a space in street names to separate the
main part of the name from Chemin, Rue, Road, Avenue etc.

If I use ']' for 0x1e and '[' for 0x1f, then streets can be written
as:

   Rue]Nicolo
   Chemin de]Pierre Froide
   Fernrodder[Strasse
   Street Lane[Road

This also affects searching in mapsource and the prefix/suffix parts
seem to be ignored.

For example if I want to search for 'Street Lane Road' (its a real
name!)
I type the letters 'street lane' and the full name is shown in the dropdown.
But if I add the 'r' as in 'street lane r', then it no longer
matches and the next matching name is shown in the dropdown, which
happens to be 'Street Lane Roundabout'.

There is also the sections in the index mdr30/31 and mdr32/33 which
are simple lists of prefixes and suffixes.  I do not know how they
are used.

In mapsource you have to pick from the dropdown list.  So to select
our favourite French road 'Chemain de Pierre Froide' you have to type
'pierre fr...' and select.  Starting from 'Chemain ...' does not
get you anywhere even if you type the whole name.  In BaseCamp,
you can type the whole name, and although it also will not show
in the dropdown, it will allow you to enter it and it will find it.

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Steve Ratcliffe

Hi

> '14 Hauptstrasse' and nearby one named '13/14 Hauptstrasse' When I
> type "14 Hau"  I want to find both entries, but I prefer to see '14
> Hauptstrasse' first (topmost).  This depends on how we sort, maybe
> also on the order in which the names appear in the *.img files.

Is '13/14 Hauptstrasse' the actual name of the street?

If is is and I use '|' to mark the name offset in the index, then
if you add an index entry for '13/|14 Hauptstrasse'
and '|14 Hauptstrasse' then I expect them to be ordered as:

   |14 Hauptstrasse
   13/|14 Hauptstrasse

So they should be in the order you expect in the dropdown on typing
'14 haup...'

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Steve Ratcliffe
Hi

> So they should be in the order you expect in the dropdown on typing
> '14 haup...'

Just tried it, it is correct on the current trunk, but not on the branch
in my quick test.

..Steve

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Gerd Petermann
Hi Steve,

yes, that's why I'd like to use getInitialPart() again in the branch. I want to do a lot more tests
with the different combinations of split-name, highway shield codes and the other special chars
to make up my mind.
This will take some time.

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Steve Ratcliffe <[hidden email]>
Gesendet: Samstag, 8. April 2017 21:58:08
An: [hidden email]
Betreff: Re: [mkgmap-dev] Possible problem with global search index since r3875

Hi

> So they should be in the order you expect in the dropdown on typing
> '14 haup...'

Just tried it, it is correct on the current trunk, but not on the branch
in my quick test.

..Steve

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Gerd Petermann
In reply to this post by Steve Ratcliffe
Hi Steve,

the names are produced by the default style when processing an area around this way:
http://www.openstreetmap.org/way/365980659
with options --index --housenumbers --bounds=bounds.zip --route --x-split-name-index
The rule that produces the label "13/14 Hauptstrasse"  (start is highway shield code) is this:
highway=primary {name '${ref|highway-symbol:box} ${name}' | '${ref|highway-symbol:box}' | '${name}'; addlabel '${name} (${ref})' }

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Steve Ratcliffe <[hidden email]>
Gesendet: Samstag, 8. April 2017 21:41:05
An: [hidden email]
Betreff: Re: [mkgmap-dev] Possible problem with global search index since r3875

Hi

> '14 Hauptstrasse' and nearby one named '13/14 Hauptstrasse' When I
> type "14 Hau"  I want to find both entries, but I prefer to see '14
> Hauptstrasse' first (topmost).  This depends on how we sort, maybe
> also on the order in which the names appear in the *.img files.

Is '13/14 Hauptstrasse' the actual name of the street?

If is is and I use '|' to mark the name offset in the index, then
if you add an index entry for '13/|14 Hauptstrasse'
and '|14 Hauptstrasse' then I expect them to be ordered as:

   |14 Hauptstrasse
   13/|14 Hauptstrasse

So they should be in the order you expect in the dropdown on typing
'14 haup...'

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Steve Ratcliffe

Hi

> The rule that produces the label "13/14 Hauptstrasse"  (start is highway shield code) is this:
> highway=primary {name '${ref|highway-symbol:box} ${name}' | '${ref|highway-symbol:box}' | '${name}'; addlabel '${name} (${ref})' }

This may be unpopular but we should stop doing this.  This was needed
before we could have multiple labels, but that was a long time ago.

The Garmin way to do this is that the name is the first label, then
any refs or alternate names.

So this case would ideally be represented as:

   Label 1: Hauptstrasse
   Label 2: 13
   Label 3: 14

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible problem with global search index since r3875

Gerd Petermann
Hi all,

I also don't like the current results for roads with a name and a ref. For the mentioned rule we get two labels
Label 1 :"13/14 Hauptstrasse" (with highway shield box)
Label 2 : "Hauptstrasse (13;14)"
If --housenumbers is used, we may see a third label if there are numbers:
Label 3 : "Hauptstrasse"

If I got that right the highway shield code only makes sense on label 1 and we have Java code in Subdivision.createLine()
which seems to make sure that we don't add duplicate ref labels if they only differ by the highway shield code.
So I came up with the attached patch for the default style.

Please comment!
Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Steve Ratcliffe <[hidden email]>
Gesendet: Sonntag, 9. April 2017 11:23:38
An: [hidden email]
Betreff: Re: [mkgmap-dev] Possible problem with global search index since r3875

Hi

> The rule that produces the label "13/14 Hauptstrasse"  (start is highway shield code) is this:
> highway=primary {name '${ref|highway-symbol:box} ${name}' | '${ref|highway-symbol:box}' | '${name}'; addlabel '${name} (${ref})' }

This may be unpopular but we should stop doing this.  This was needed
before we could have multiple labels, but that was a long time ago.

The Garmin way to do this is that the name is the first label, then
any refs or alternate names.

So this case would ideally be represented as:

   Label 1: Hauptstrasse
   Label 2: 13
   Label 3: 14

..Steve
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

refs.patch (2K) Download Attachment