Find Address - No cities

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Find Address - No cities

Ticker Berkin
I have a strange problem.

With a particular split of 'british-isles', one map from the split
causes the list of cities in the Find>Address function to be empty.

Find>Address does still find streets, but not this is not very useful
when cannot limit by City first.

Excluding this particular map but leaving everything else unchanged,
the city list is there and works as expected.

The list of cities in Find>City is correct.

The same problem occurs on an old build of mkgmap (r3694).

Splitting with a reduced --max_nodes, the problem goes away.
Also I think some minor changes to my style has caused the problem to
come and go.

I', using Etrex Legend, --gmapsupp --index --route and various other
options.

Any ideas about how to debug this?

Thanks
Ticker

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Ticker Berkin wrote
With a particular split of 'british-isles', one map from the split
causes the list of cities in the Find>Address function to be empty.
Maybe you have the same problem here which I described for road
name search when road names contain special characters. In your case
this would mean special characters in the citiy names.
I suggest to add debug code in
uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter writer)
to print all city names which are written to the global index.

Gerd
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

I've done a bit of investigation and debugging. It seems to be building
the indexes with enough cities and streets. MDR20 is the suspect index.

I'm now experimenting with mkgmap / Display

With just the suspect map built in gmapsupp:

$ ... test.ExtractFile gmapsupp.img
filter[]
Copying .               65024
Copying MAKEGMAP.MPS    78
Copying 74440044.RGN    4154927
Copying 74440044.TRE    18772
Copying 74440044.LBL    2174485
Copying 74440044.NET    1626350
Copying 74440044.NOD    3566829
Copying 00007444.MDR    2979295
Copying 00007444.SRT    912

$ ... test.display.MdrSummary 00007444.MDR
initial values    7    2    e
MDR 1  NR=1     (000001) RS=4  magic=0x0
MDR 4  NR=101   (000065) RS=3  magic=0x0
MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
MDR 7  NR=17674 (00450a) RS=6  magic=0x66
MDR 9  NR=11    (00000b) RS=4  magic=0x0
MDR 10 DataSize=427616   (00068660) magic=0x0
MDR 11 NR=106904(01a198) RS=10 magic=0x95
MDR 12 NR=11    (00000b) RS=7  magic=0x40a
MDR 17 DataSize=111036   (0001b1bc) magic=0x0
MDR 18 NR=102   (000066) RS=5  magic=0x6
MDR 19 NR=106904(01a198) RS=3  magic=0x2
MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
MDR 24 NR=2     (000002) RS=6  magic=0x0
MDR 25 NR=70016 (011180) RS=3  magic=0x0
MDR 29 NR=2     (000002) RS=8  magic=0x26

seems fine but

$ ... test.display.MdrCheck 00007444.MDR
---------- 00007444.MDR --------------------
EXTRA
remaining {}
# reading 74440044
mdr5 check
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index:
308736, Size: 70018
        at java.util.ArrayList.rangeCheck(ArrayList.java:653)
        at java.util.ArrayList.get(ArrayList.java:429)
        at test.display.check.MapDetails.getCity(MapDetails.java:188)
        at test.display.MdrCheck.check5(MdrCheck.java:282)
        at test.display.MdrCheck.print(MdrCheck.java:87)
        at test.display.CommonDisplay.display(CommonDisplay.java:171)
        at test.display.CommonDisplay.display(CommonDisplay.java:196)
        at test.check.CommonCheck.runMain(CommonCheck.java:145)
        at test.display.MdrCheck.main(MdrCheck.java:1789)
+ exit

On another map from the split this runs OK, producing masses of output:

$ ... test.display.MdrCheck 00007429.MDR
---------- 00007429.MDR --------------------
EXTRA
remaining {}
# reading 74290047
ERROR: map 1: city 1: no name; index 1
mdr5 check
1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
KINGDOM^]GBR) ind20=0 rep=false
2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED KINGDOM^]GBR)
ind20=0 rep=false
... 87000 more lines ...


Do you think mkgmap:Display is finding a real problem and I should
pursue this line, or could it be unreliable and this exception is due
to it rather than the format of the index?  

Thanks
Ticker

On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:

> Ticker Berkin wrote
> > With a particular split of 'british-isles', one map from the split
> > causes the list of cities in the Find>Address function to be empty.
>
> Maybe you have the same problem here which I described for road
> name search when road names contain special characters. In your case
> this would mean special characters in the citiy names.
> I suggest to add debug code in
> uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> writer)
> to print all city names which are written to the global index.
>
> Gerd
>
>
>
> --
> View this message in context: http://gis.19327.n8.nabble.com/Find-Add
> ress-No-cities-tp5899775p5899880.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin-2
Hi Gerd

I think I've found the problem:

/src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227

        private void putLocalCityIndex(ImgFileWriter writer, int
cityIndex) {
                if (localCitySize == 2)
                // 3 probably not possible in actual maps.
                        writer.putChar((char) cityIndex);
                else
                        writer.put((byte) cityIndex);
        }

There were more than 63335 cities in the failing map.

Ticker


On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:

> Hi Gerd
>
> I've done a bit of investigation and debugging. It seems to be
> building
> the indexes with enough cities and streets. MDR20 is the suspect
> index.
>
> I'm now experimenting with mkgmap / Display
>
> With just the suspect map built in gmapsupp:
>
> $ ... test.ExtractFile gmapsupp.img
> filter[]
> Copying .               65024
> Copying MAKEGMAP.MPS    78
> Copying 74440044.RGN    4154927
> Copying 74440044.TRE    18772
> Copying 74440044.LBL    2174485
> Copying 74440044.NET    1626350
> Copying 74440044.NOD    3566829
> Copying 00007444.MDR    2979295
> Copying 00007444.SRT    912
>
> $ ... test.display.MdrSummary 00007444.MDR
> initial values    7    2    e
> MDR 1  NR=1     (000001) RS=4  magic=0x0
> MDR 4  NR=101   (000065) RS=3  magic=0x0
> MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> MDR 9  NR=11    (00000b) RS=4  magic=0x0
> MDR 10 DataSize=427616   (00068660) magic=0x0
> MDR 11 NR=106904(01a198) RS=10 magic=0x95
> MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> MDR 18 NR=102   (000066) RS=5  magic=0x6
> MDR 19 NR=106904(01a198) RS=3  magic=0x2
> MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> MDR 24 NR=2     (000002) RS=6  magic=0x0
> MDR 25 NR=70016 (011180) RS=3  magic=0x0
> MDR 29 NR=2     (000002) RS=8  magic=0x26
>
> seems fine but
>
> $ ... test.display.MdrCheck 00007444.MDR
> ---------- 00007444.MDR --------------------
> EXTRA
> remaining {}
> # reading 74440044
> mdr5 check
> Exception in thread "main" java.lang.IndexOutOfBoundsException:
> Index:
> 308736, Size: 70018
>         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>         at java.util.ArrayList.get(ArrayList.java:429)
>         at test.display.check.MapDetails.getCity(MapDetails.java:188)
>         at test.display.MdrCheck.check5(MdrCheck.java:282)
>         at test.display.MdrCheck.print(MdrCheck.java:87)
>         at test.display.CommonDisplay.display(CommonDisplay.java:171)
>         at test.display.CommonDisplay.display(CommonDisplay.java:196)
>         at test.check.CommonCheck.runMain(CommonCheck.java:145)
>         at test.display.MdrCheck.main(MdrCheck.java:1789)
> + exit
>
> On another map from the split this runs OK, producing masses of
> output:
>
> $ ... test.display.MdrCheck 00007429.MDR
> ---------- 00007429.MDR --------------------
> EXTRA
> remaining {}
> # reading 74290047
> ERROR: map 1: city 1: no name; index 1
> mdr5 check
> 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> KINGDOM^]GBR) ind20=0 rep=false
> 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> KINGDOM^]GBR)
> ind20=0 rep=false
> ... 87000 more lines ...
>
>
> Do you think mkgmap:Display is finding a real problem and I should
> pursue this line, or could it be unreliable and this exception is due
> to it rather than the format of the index?  
>
> Thanks
> Ticker
>
> On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > Ticker Berkin wrote
> > > With a particular split of 'british-isles', one map from the
> > > split
> > > causes the list of cities in the Find>Address function to be
> > > empty.
> >
> > Maybe you have the same problem here which I described for road
> > name search when road names contain special characters. In your
> > case
> > this would mean special characters in the citiy names.
> > I suggest to add debug code in
> > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> > writer)
> > to print all city names which are written to the global index.
> >
> > Gerd
> >
> >
> >
> > --
> > View this message in context:
> > http://gis.19327.n8.nabble.com/Find-Add
> > ress-No-cities-tp5899775p5899880.html
> > Sent from the Mkgmap Development mailing list archive at
> > Nabble.com.
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi Ticker,

yes, sounds plausible. The comment seems to say that we don't know how to handle so many cities.
Why do you have so many cities. Do you "abuse" the corresponding POI for something?

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Samstag, 29. Juli 2017 18:13:20
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

I think I've found the problem:

/src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227

        private void putLocalCityIndex(ImgFileWriter writer, int
cityIndex) {
                if (localCitySize == 2)
                // 3 probably not possible in actual maps.
                        writer.putChar((char) cityIndex);
                else
                        writer.put((byte) cityIndex);
        }

There were more than 63335 cities in the failing map.

Ticker


On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:

> Hi Gerd
>
> I've done a bit of investigation and debugging. It seems to be
> building
> the indexes with enough cities and streets. MDR20 is the suspect
> index.
>
> I'm now experimenting with mkgmap / Display
>
> With just the suspect map built in gmapsupp:
>
> $ ... test.ExtractFile gmapsupp.img
> filter[]
> Copying .               65024
> Copying MAKEGMAP.MPS    78
> Copying 74440044.RGN    4154927
> Copying 74440044.TRE    18772
> Copying 74440044.LBL    2174485
> Copying 74440044.NET    1626350
> Copying 74440044.NOD    3566829
> Copying 00007444.MDR    2979295
> Copying 00007444.SRT    912
>
> $ ... test.display.MdrSummary 00007444.MDR
> initial values    7    2    e
> MDR 1  NR=1     (000001) RS=4  magic=0x0
> MDR 4  NR=101   (000065) RS=3  magic=0x0
> MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> MDR 9  NR=11    (00000b) RS=4  magic=0x0
> MDR 10 DataSize=427616   (00068660) magic=0x0
> MDR 11 NR=106904(01a198) RS=10 magic=0x95
> MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> MDR 18 NR=102   (000066) RS=5  magic=0x6
> MDR 19 NR=106904(01a198) RS=3  magic=0x2
> MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> MDR 24 NR=2     (000002) RS=6  magic=0x0
> MDR 25 NR=70016 (011180) RS=3  magic=0x0
> MDR 29 NR=2     (000002) RS=8  magic=0x26
>
> seems fine but
>
> $ ... test.display.MdrCheck 00007444.MDR
> ---------- 00007444.MDR --------------------
> EXTRA
> remaining {}
> # reading 74440044
> mdr5 check
> Exception in thread "main" java.lang.IndexOutOfBoundsException:
> Index:
> 308736, Size: 70018
>         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>         at java.util.ArrayList.get(ArrayList.java:429)
>         at test.display.check.MapDetails.getCity(MapDetails.java:188)
>         at test.display.MdrCheck.check5(MdrCheck.java:282)
>         at test.display.MdrCheck.print(MdrCheck.java:87)
>         at test.display.CommonDisplay.display(CommonDisplay.java:171)
>         at test.display.CommonDisplay.display(CommonDisplay.java:196)
>         at test.check.CommonCheck.runMain(CommonCheck.java:145)
>         at test.display.MdrCheck.main(MdrCheck.java:1789)
> + exit
>
> On another map from the split this runs OK, producing masses of
> output:
>
> $ ... test.display.MdrCheck 00007429.MDR
> ---------- 00007429.MDR --------------------
> EXTRA
> remaining {}
> # reading 74290047
> ERROR: map 1: city 1: no name; index 1
> mdr5 check
> 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> KINGDOM^]GBR) ind20=0 rep=false
> 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> KINGDOM^]GBR)
> ind20=0 rep=false
> ... 87000 more lines ...
>
>
> Do you think mkgmap:Display is finding a real problem and I should
> pursue this line, or could it be unreliable and this exception is due
> to it rather than the format of the index?
>
> Thanks
> Ticker
>
> On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > Ticker Berkin wrote
> > > With a particular split of 'british-isles', one map from the
> > > split
> > > causes the list of cities in the Find>Address function to be
> > > empty.
> >
> > Maybe you have the same problem here which I described for road
> > name search when road names contain special characters. In your
> > case
> > this would mean special characters in the citiy names.
> > I suggest to add debug code in
> > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> > writer)
> > to print all city names which are written to the global index.
> >
> > Gerd
> >
> >
> >
> > --
> > View this message in context:
> > http://gis.19327.n8.nabble.com/Find-Add
> > ress-No-cities-tp5899775p5899880.html
> > Sent from the Mkgmap Development mailing list archive at
> > Nabble.com.
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

It looks as if can simply write 1/2/3 bytes as have indicated in the
magic field in the MDR header and I've changed it to do this. All seems
OK.

Yes - I do misuse a City POI. But now, after having worked out how the
MDR20 index works, I might be able to do what I want in a better way.

Attached is patch to write the correct number of bytes.

Regards
Ticker

On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> yes, sounds plausible. The comment seems to say that we don't know
> how to handle so many cities.
> Why do you have so many cities. Do you "abuse" the corresponding POI
> for something?
>
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Samstag, 29. Juli 2017 18:13:20
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> I think I've found the problem:
>
> /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
>
>         private void putLocalCityIndex(ImgFileWriter writer, int
> cityIndex) {
>                 if (localCitySize == 2)
>                 // 3 probably not possible in actual maps.
>                         writer.putChar((char) cityIndex);
>                 else
>                         writer.put((byte) cityIndex);
>         }
>
> There were more than 63335 cities in the failing map.
>
> Ticker
>
>
> On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > Hi Gerd
> >
> > I've done a bit of investigation and debugging. It seems to be
> > building
> > the indexes with enough cities and streets. MDR20 is the suspect
> > index.
> >
> > I'm now experimenting with mkgmap / Display
> >
> > With just the suspect map built in gmapsupp:
> >
> > $ ... test.ExtractFile gmapsupp.img
> > filter[]
> > Copying .               65024
> > Copying MAKEGMAP.MPS    78
> > Copying 74440044.RGN    4154927
> > Copying 74440044.TRE    18772
> > Copying 74440044.LBL    2174485
> > Copying 74440044.NET    1626350
> > Copying 74440044.NOD    3566829
> > Copying 00007444.MDR    2979295
> > Copying 00007444.SRT    912
> >
> > $ ... test.display.MdrSummary 00007444.MDR
> > initial values    7    2    e
> > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > MDR 10 DataSize=427616   (00068660) magic=0x0
> > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > MDR 29 NR=2     (000002) RS=8  magic=0x26
> >
> > seems fine but
> >
> > $ ... test.display.MdrCheck 00007444.MDR
> > ---------- 00007444.MDR --------------------
> > EXTRA
> > remaining {}
> > # reading 74440044
> > mdr5 check
> > Exception in thread "main" java.lang.IndexOutOfBoundsException:
> > Index:
> > 308736, Size: 70018
> >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> >         at java.util.ArrayList.get(ArrayList.java:429)
> >         at
> > test.display.check.MapDetails.getCity(MapDetails.java:188)
> >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> >         at test.display.MdrCheck.print(MdrCheck.java:87)
> >         at
> > test.display.CommonDisplay.display(CommonDisplay.java:171)
> >         at
> > test.display.CommonDisplay.display(CommonDisplay.java:196)
> >         at test.check.CommonCheck.runMain(CommonCheck.java:145)
> >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > + exit
> >
> > On another map from the split this runs OK, producing masses of
> > output:
> >
> > $ ... test.display.MdrCheck 00007429.MDR
> > ---------- 00007429.MDR --------------------
> > EXTRA
> > remaining {}
> > # reading 74290047
> > ERROR: map 1: city 1: no name; index 1
> > mdr5 check
> > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > KINGDOM^]GBR) ind20=0 rep=false
> > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > KINGDOM^]GBR)
> > ind20=0 rep=false
> > ... 87000 more lines ...
> >
> >
> > Do you think mkgmap:Display is finding a real problem and I should
> > pursue this line, or could it be unreliable and this exception is
> > due
> > to it rather than the format of the index?
> >
> > Thanks
> > Ticker
> >
> > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > Ticker Berkin wrote
> > > > With a particular split of 'british-isles', one map from the
> > > > split
> > > > causes the list of cities in the Find>Address function to be
> > > > empty.
> > >
> > > Maybe you have the same problem here which I described for road
> > > name search when road names contain special characters. In your
> > > case
> > > this would mean special characters in the citiy names.
> > > I suggest to add debug code in
> > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> > > writer)
> > > to print all city names which are written to the global index.
> > >
> > > Gerd
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://gis.19327.n8.nabble.com/Find-Add
> > > ress-No-cities-tp5899775p5899880.html
> > > Sent from the Mkgmap Development mailing list archive at
> > > Nabble.com.
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

cityPtr3.patch (984 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi Tcker,

thanks for the patch. I seem to remember that the number of entries in mdr5 is also used
elsewhere. Will have a closer look tomorrow.

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Samstag, 29. Juli 2017 19:14:02
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

It looks as if can simply write 1/2/3 bytes as have indicated in the
magic field in the MDR header and I've changed it to do this. All seems
OK.

Yes - I do misuse a City POI. But now, after having worked out how the
MDR20 index works, I might be able to do what I want in a better way.

Attached is patch to write the correct number of bytes.

Regards
Ticker

On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> yes, sounds plausible. The comment seems to say that we don't know
> how to handle so many cities.
> Why do you have so many cities. Do you "abuse" the corresponding POI
> for something?
>
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Samstag, 29. Juli 2017 18:13:20
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> I think I've found the problem:
>
> /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
>
>         private void putLocalCityIndex(ImgFileWriter writer, int
> cityIndex) {
>                 if (localCitySize == 2)
>                 // 3 probably not possible in actual maps.
>                         writer.putChar((char) cityIndex);
>                 else
>                         writer.put((byte) cityIndex);
>         }
>
> There were more than 63335 cities in the failing map.
>
> Ticker
>
>
> On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > Hi Gerd
> >
> > I've done a bit of investigation and debugging. It seems to be
> > building
> > the indexes with enough cities and streets. MDR20 is the suspect
> > index.
> >
> > I'm now experimenting with mkgmap / Display
> >
> > With just the suspect map built in gmapsupp:
> >
> > $ ... test.ExtractFile gmapsupp.img
> > filter[]
> > Copying .               65024
> > Copying MAKEGMAP.MPS    78
> > Copying 74440044.RGN    4154927
> > Copying 74440044.TRE    18772
> > Copying 74440044.LBL    2174485
> > Copying 74440044.NET    1626350
> > Copying 74440044.NOD    3566829
> > Copying 00007444.MDR    2979295
> > Copying 00007444.SRT    912
> >
> > $ ... test.display.MdrSummary 00007444.MDR
> > initial values    7    2    e
> > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > MDR 10 DataSize=427616   (00068660) magic=0x0
> > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > MDR 29 NR=2     (000002) RS=8  magic=0x26
> >
> > seems fine but
> >
> > $ ... test.display.MdrCheck 00007444.MDR
> > ---------- 00007444.MDR --------------------
> > EXTRA
> > remaining {}
> > # reading 74440044
> > mdr5 check
> > Exception in thread "main" java.lang.IndexOutOfBoundsException:
> > Index:
> > 308736, Size: 70018
> >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> >         at java.util.ArrayList.get(ArrayList.java:429)
> >         at
> > test.display.check.MapDetails.getCity(MapDetails.java:188)
> >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> >         at test.display.MdrCheck.print(MdrCheck.java:87)
> >         at
> > test.display.CommonDisplay.display(CommonDisplay.java:171)
> >         at
> > test.display.CommonDisplay.display(CommonDisplay.java:196)
> >         at test.check.CommonCheck.runMain(CommonCheck.java:145)
> >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > + exit
> >
> > On another map from the split this runs OK, producing masses of
> > output:
> >
> > $ ... test.display.MdrCheck 00007429.MDR
> > ---------- 00007429.MDR --------------------
> > EXTRA
> > remaining {}
> > # reading 74290047
> > ERROR: map 1: city 1: no name; index 1
> > mdr5 check
> > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > KINGDOM^]GBR) ind20=0 rep=false
> > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > KINGDOM^]GBR)
> > ind20=0 rep=false
> > ... 87000 more lines ...
> >
> >
> > Do you think mkgmap:Display is finding a real problem and I should
> > pursue this line, or could it be unreliable and this exception is
> > due
> > to it rather than the format of the index?
> >
> > Thanks
> > Ticker
> >
> > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > Ticker Berkin wrote
> > > > With a particular split of 'british-isles', one map from the
> > > > split
> > > > causes the list of cities in the Find>Address function to be
> > > > empty.
> > >
> > > Maybe you have the same problem here which I described for road
> > > name search when road names contain special characters. In your
> > > case
> > > this would mean special characters in the citiy names.
> > > I suggest to add debug code in
> > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> > > writer)
> > > to print all city names which are written to the global index.
> > >
> > > Gerd
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://gis.19327.n8.nabble.com/Find-Add
> > > ress-No-cities-tp5899775p5899880.html
> > > Sent from the Mkgmap Development mailing list archive at
> > > Nabble.com.
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

Quite possibly, or maybe there are limits in the device.

More testing reveals that although my lists of cities and streets
within them appear correctly in Find>Address, it doesn't find anything.

Ticker

On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:

> Hi Tcker,
>
> thanks for the patch. I seem to remember that the number of entries
> in mdr5 is also used
> elsewhere. Will have a closer look tomorrow.
>
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Samstag, 29. Juli 2017 19:14:02
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> It looks as if can simply write 1/2/3 bytes as have indicated in the
> magic field in the MDR header and I've changed it to do this. All
> seems
> OK.
>
> Yes - I do misuse a City POI. But now, after having worked out how
> the
> MDR20 index works, I might be able to do what I want in a better way.
>
> Attached is patch to write the correct number of bytes.
>
> Regards
> Ticker
>
> On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > yes, sounds plausible. The comment seems to say that we don't know
> > how to handle so many cities.
> > Why do you have so many cities. Do you "abuse" the corresponding
> > POI
> > for something?
> >
> > Gerd
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > I think I've found the problem:
> >
> > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> >
> >         private void putLocalCityIndex(ImgFileWriter writer, int
> > cityIndex) {
> >                 if (localCitySize == 2)
> >                 // 3 probably not possible in actual maps.
> >                         writer.putChar((char) cityIndex);
> >                 else
> >                         writer.put((byte) cityIndex);
> >         }
> >
> > There were more than 63335 cities in the failing map.
> >
> > Ticker
> >
> >
> > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > >
> > > I've done a bit of investigation and debugging. It seems to be
> > > building
> > > the indexes with enough cities and streets. MDR20 is the suspect
> > > index.
> > >
> > > I'm now experimenting with mkgmap / Display
> > >
> > > With just the suspect map built in gmapsupp:
> > >
> > > $ ... test.ExtractFile gmapsupp.img
> > > filter[]
> > > Copying .               65024
> > > Copying MAKEGMAP.MPS    78
> > > Copying 74440044.RGN    4154927
> > > Copying 74440044.TRE    18772
> > > Copying 74440044.LBL    2174485
> > > Copying 74440044.NET    1626350
> > > Copying 74440044.NOD    3566829
> > > Copying 00007444.MDR    2979295
> > > Copying 00007444.SRT    912
> > >
> > > $ ... test.display.MdrSummary 00007444.MDR
> > > initial values    7    2    e
> > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > >
> > > seems fine but
> > >
> > > $ ... test.display.MdrCheck 00007444.MDR
> > > ---------- 00007444.MDR --------------------
> > > EXTRA
> > > remaining {}
> > > # reading 74440044
> > > mdr5 check
> > > Exception in thread "main" java.lang.IndexOutOfBoundsException:
> > > Index:
> > > 308736, Size: 70018
> > >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > >         at java.util.ArrayList.get(ArrayList.java:429)
> > >         at
> > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > >         at
> > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > >         at
> > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > >         at test.check.CommonCheck.runMain(CommonCheck.java:145)
> > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > + exit
> > >
> > > On another map from the split this runs OK, producing masses of
> > > output:
> > >
> > > $ ... test.display.MdrCheck 00007429.MDR
> > > ---------- 00007429.MDR --------------------
> > > EXTRA
> > > remaining {}
> > > # reading 74290047
> > > ERROR: map 1: city 1: no name; index 1
> > > mdr5 check
> > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > > KINGDOM^]GBR) ind20=0 rep=false
> > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > KINGDOM^]GBR)
> > > ind20=0 rep=false
> > > ... 87000 more lines ...
> > >
> > >
> > > Do you think mkgmap:Display is finding a real problem and I
> > > should
> > > pursue this line, or could it be unreliable and this exception is
> > > due
> > > to it rather than the format of the index?
> > >
> > > Thanks
> > > Ticker
> > >
> > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > Ticker Berkin wrote
> > > > > With a particular split of 'british-isles', one map from the
> > > > > split
> > > > > causes the list of cities in the Find>Address function to be
> > > > > empty.
> > > >
> > > > Maybe you have the same problem here which I described for road
> > > > name search when road names contain special characters. In your
> > > > case
> > > > this would mean special characters in the citiy names.
> > > > I suggest to add debug code in
> > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> > > > writer)
> > > > to print all city names which are written to the global index.
> > > >
> > > > Gerd
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > ress-No-cities-tp5899775p5899880.html
> > > > Sent from the Mkgmap Development mailing list archive at
> > > > Nabble.com.
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi Ticker,

hmm, I think the patch looks good and I have no idea why the search doesn't work
for you. Yesterday I thought the patch handles the overall number of cities in the gmapsupp,
but it is about the number of cities in a single map tile.
Maybe there is another problem caused by the high number of cities. Maybe you can post a link to the tile with
that high number?

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Samstag, 29. Juli 2017 21:12:27
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

Quite possibly, or maybe there are limits in the device.

More testing reveals that although my lists of cities and streets
within them appear correctly in Find>Address, it doesn't find anything.

Ticker

On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:

> Hi Tcker,
>
> thanks for the patch. I seem to remember that the number of entries
> in mdr5 is also used
> elsewhere. Will have a closer look tomorrow.
>
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Samstag, 29. Juli 2017 19:14:02
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> It looks as if can simply write 1/2/3 bytes as have indicated in the
> magic field in the MDR header and I've changed it to do this. All
> seems
> OK.
>
> Yes - I do misuse a City POI. But now, after having worked out how
> the
> MDR20 index works, I might be able to do what I want in a better way.
>
> Attached is patch to write the correct number of bytes.
>
> Regards
> Ticker
>
> On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > yes, sounds plausible. The comment seems to say that we don't know
> > how to handle so many cities.
> > Why do you have so many cities. Do you "abuse" the corresponding
> > POI
> > for something?
> >
> > Gerd
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > I think I've found the problem:
> >
> > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> >
> >         private void putLocalCityIndex(ImgFileWriter writer, int
> > cityIndex) {
> >                 if (localCitySize == 2)
> >                 // 3 probably not possible in actual maps.
> >                         writer.putChar((char) cityIndex);
> >                 else
> >                         writer.put((byte) cityIndex);
> >         }
> >
> > There were more than 63335 cities in the failing map.
> >
> > Ticker
> >
> >
> > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > >
> > > I've done a bit of investigation and debugging. It seems to be
> > > building
> > > the indexes with enough cities and streets. MDR20 is the suspect
> > > index.
> > >
> > > I'm now experimenting with mkgmap / Display
> > >
> > > With just the suspect map built in gmapsupp:
> > >
> > > $ ... test.ExtractFile gmapsupp.img
> > > filter[]
> > > Copying .               65024
> > > Copying MAKEGMAP.MPS    78
> > > Copying 74440044.RGN    4154927
> > > Copying 74440044.TRE    18772
> > > Copying 74440044.LBL    2174485
> > > Copying 74440044.NET    1626350
> > > Copying 74440044.NOD    3566829
> > > Copying 00007444.MDR    2979295
> > > Copying 00007444.SRT    912
> > >
> > > $ ... test.display.MdrSummary 00007444.MDR
> > > initial values    7    2    e
> > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > >
> > > seems fine but
> > >
> > > $ ... test.display.MdrCheck 00007444.MDR
> > > ---------- 00007444.MDR --------------------
> > > EXTRA
> > > remaining {}
> > > # reading 74440044
> > > mdr5 check
> > > Exception in thread "main" java.lang.IndexOutOfBoundsException:
> > > Index:
> > > 308736, Size: 70018
> > >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > >         at java.util.ArrayList.get(ArrayList.java:429)
> > >         at
> > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > >         at
> > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > >         at
> > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > >         at test.check.CommonCheck.runMain(CommonCheck.java:145)
> > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > + exit
> > >
> > > On another map from the split this runs OK, producing masses of
> > > output:
> > >
> > > $ ... test.display.MdrCheck 00007429.MDR
> > > ---------- 00007429.MDR --------------------
> > > EXTRA
> > > remaining {}
> > > # reading 74290047
> > > ERROR: map 1: city 1: no name; index 1
> > > mdr5 check
> > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > > KINGDOM^]GBR) ind20=0 rep=false
> > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > KINGDOM^]GBR)
> > > ind20=0 rep=false
> > > ... 87000 more lines ...
> > >
> > >
> > > Do you think mkgmap:Display is finding a real problem and I
> > > should
> > > pursue this line, or could it be unreliable and this exception is
> > > due
> > > to it rather than the format of the index?
> > >
> > > Thanks
> > > Ticker
> > >
> > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > Ticker Berkin wrote
> > > > > With a particular split of 'british-isles', one map from the
> > > > > split
> > > > > causes the list of cities in the Find>Address function to be
> > > > > empty.
> > > >
> > > > Maybe you have the same problem here which I described for road
> > > > name search when road names contain special characters. In your
> > > > case
> > > > this would mean special characters in the citiy names.
> > > > I suggest to add debug code in
> > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWriter
> > > > writer)
> > > > to print all city names which are written to the global index.
> > > >
> > > > Gerd
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > ress-No-cities-tp5899775p5899880.html
> > > > Sent from the Mkgmap Development mailing list archive at
> > > > Nabble.com.
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

I'll just done a bit more investigation and see what seems to be the
the same assumption in imgfmt/app/net/RoadDef.java around line 296
                                        if(numCities > 255)
                                               
 writer.putChar(cityIndex);
                                        else
                                               
 writer.put((byte)cityIndex);

I'll fix and test tomorrow and if there are still have problems I'll
assemble the problem map, style etc for you

Thanks
Ticker

On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> hmm, I think the patch looks good and I have no idea why the search
> doesn't work
> for you. Yesterday I thought the patch handles the overall number of
> cities in the gmapsupp,
> but it is about the number of cities in a single map tile.
> Maybe there is another problem caused by the high number of cities.
> Maybe you can post a link to the tile with
> that high number?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Samstag, 29. Juli 2017 21:12:27
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> Quite possibly, or maybe there are limits in the device.
>
> More testing reveals that although my lists of cities and streets
> within them appear correctly in Find>Address, it doesn't find
> anything.
>
> Ticker
>
> On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > Hi Tcker,
> >
> > thanks for the patch. I seem to remember that the number of entries
> > in mdr5 is also used
> > elsewhere. Will have a closer look tomorrow.
> >
> > Gerd
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > It looks as if can simply write 1/2/3 bytes as have indicated in
> > the
> > magic field in the MDR header and I've changed it to do this. All
> > seems
> > OK.
> >
> > Yes - I do misuse a City POI. But now, after having worked out how
> > the
> > MDR20 index works, I might be able to do what I want in a better
> > way.
> >
> > Attached is patch to write the correct number of bytes.
> >
> > Regards
> > Ticker
> >
> > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > yes, sounds plausible. The comment seems to say that we don't
> > > know
> > > how to handle so many cities.
> > > Why do you have so many cities. Do you "abuse" the corresponding
> > > POI
> > > for something?
> > >
> > > Gerd
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > I think I've found the problem:
> > >
> > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > >
> > >         private void putLocalCityIndex(ImgFileWriter writer, int
> > > cityIndex) {
> > >                 if (localCitySize == 2)
> > >                 // 3 probably not possible in actual maps.
> > >                         writer.putChar((char) cityIndex);
> > >                 else
> > >                         writer.put((byte) cityIndex);
> > >         }
> > >
> > > There were more than 63335 cities in the failing map.
> > >
> > > Ticker
> > >
> > >
> > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > Hi Gerd
> > > >
> > > > I've done a bit of investigation and debugging. It seems to be
> > > > building
> > > > the indexes with enough cities and streets. MDR20 is the
> > > > suspect
> > > > index.
> > > >
> > > > I'm now experimenting with mkgmap / Display
> > > >
> > > > With just the suspect map built in gmapsupp:
> > > >
> > > > $ ... test.ExtractFile gmapsupp.img
> > > > filter[]
> > > > Copying .               65024
> > > > Copying MAKEGMAP.MPS    78
> > > > Copying 74440044.RGN    4154927
> > > > Copying 74440044.TRE    18772
> > > > Copying 74440044.LBL    2174485
> > > > Copying 74440044.NET    1626350
> > > > Copying 74440044.NOD    3566829
> > > > Copying 00007444.MDR    2979295
> > > > Copying 00007444.SRT    912
> > > >
> > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > initial values    7    2    e
> > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > >
> > > > seems fine but
> > > >
> > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > ---------- 00007444.MDR --------------------
> > > > EXTRA
> > > > remaining {}
> > > > # reading 74440044
> > > > mdr5 check
> > > > Exception in thread "main" java.lang.IndexOutOfBoundsException:
> > > > Index:
> > > > 308736, Size: 70018
> > > >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > >         at
> > > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > >         at
> > > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > > >         at
> > > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > > >         at test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > + exit
> > > >
> > > > On another map from the split this runs OK, producing masses of
> > > > output:
> > > >
> > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > ---------- 00007429.MDR --------------------
> > > > EXTRA
> > > > remaining {}
> > > > # reading 74290047
> > > > ERROR: map 1: city 1: no name; index 1
> > > > mdr5 check
> > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > KINGDOM^]GBR)
> > > > ind20=0 rep=false
> > > > ... 87000 more lines ...
> > > >
> > > >
> > > > Do you think mkgmap:Display is finding a real problem and I
> > > > should
> > > > pursue this line, or could it be unreliable and this exception
> > > > is
> > > > due
> > > > to it rather than the format of the index?
> > > >
> > > > Thanks
> > > > Ticker
> > > >
> > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > Ticker Berkin wrote
> > > > > > With a particular split of 'british-isles', one map from
> > > > > > the
> > > > > > split
> > > > > > causes the list of cities in the Find>Address function to
> > > > > > be
> > > > > > empty.
> > > > >
> > > > > Maybe you have the same problem here which I described for
> > > > > road
> > > > > name search when road names contain special characters. In
> > > > > your
> > > > > case
> > > > > this would mean special characters in the citiy names.
> > > > > I suggest to add debug code in
> > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWrite
> > > > > r
> > > > > writer)
> > > > > to print all city names which are written to the global
> > > > > index.
> > > > >
> > > > > Gerd
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > View this message in context:
> > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > ress-No-cities-tp5899775p5899880.html
> > > > > Sent from the Mkgmap Development mailing list archive at
> > > > > Nabble.com.
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

I started fixing all the places where there is an assumption about the
number of cities (and zips) being < 65535 and I'm only part way through
my list of suspect files and have make a lot of changes so far.

I'm thinking it is not worth while because:

The overall MDR size jumps because all city entries change to 3 bytes.
I get a smaller GMAPSUPP by increasing the splitting so that it stays
in the 2 byte limit.

I might miss some subtle coding of the header flags and/or the
input/output.

It might not work anyway after I've made all the changes.

The changes are a bit messy because I've just hacked each problem area
with a fix. Really there should be some utility functions, enhancements
to the readers/writers and pointerSize variables in the relevant
classes.

I propose just to signal an error in individual maps when the exceed
the limit

What do you think?

Regards
Ticker

On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:

> Hi Gerd
>
> I'll just done a bit more investigation and see what seems to be the
> the same assumption in imgfmt/app/net/RoadDef.java around line 296
>                                         if(numCities > 255)
>                                                
>  writer.putChar(cityIndex);
>                                         else
>                                                
>  writer.put((byte)cityIndex);
>
> I'll fix and test tomorrow and if there are still have problems I'll
> assemble the problem map, style etc for you
>
> Thanks
> Ticker
>
> On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > hmm, I think the patch looks good and I have no idea why the search
> > doesn't work
> > for you. Yesterday I thought the patch handles the overall number
> > of
> > cities in the gmapsupp,
> > but it is about the number of cities in a single map tile.
> > Maybe there is another problem caused by the high number of cities.
> > Maybe you can post a link to the tile with
> > that high number?
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > Quite possibly, or maybe there are limits in the device.
> >
> > More testing reveals that although my lists of cities and streets
> > within them appear correctly in Find>Address, it doesn't find
> > anything.
> >
> > Ticker
> >
> > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > Hi Tcker,
> > >
> > > thanks for the patch. I seem to remember that the number of
> > > entries
> > > in mdr5 is also used
> > > elsewhere. Will have a closer look tomorrow.
> > >
> > > Gerd
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > It looks as if can simply write 1/2/3 bytes as have indicated in
> > > the
> > > magic field in the MDR header and I've changed it to do this. All
> > > seems
> > > OK.
> > >
> > > Yes - I do misuse a City POI. But now, after having worked out
> > > how
> > > the
> > > MDR20 index works, I might be able to do what I want in a better
> > > way.
> > >
> > > Attached is patch to write the correct number of bytes.
> > >
> > > Regards
> > > Ticker
> > >
> > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > yes, sounds plausible. The comment seems to say that we don't
> > > > know
> > > > how to handle so many cities.
> > > > Why do you have so many cities. Do you "abuse" the
> > > > corresponding
> > > > POI
> > > > for something?
> > > >
> > > > Gerd
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > An: [hidden email]
> > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > >
> > > > Hi Gerd
> > > >
> > > > I think I've found the problem:
> > > >
> > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > >
> > > >         private void putLocalCityIndex(ImgFileWriter writer,
> > > > int
> > > > cityIndex) {
> > > >                 if (localCitySize == 2)
> > > >                 // 3 probably not possible in actual maps.
> > > >                         writer.putChar((char) cityIndex);
> > > >                 else
> > > >                         writer.put((byte) cityIndex);
> > > >         }
> > > >
> > > > There were more than 63335 cities in the failing map.
> > > >
> > > > Ticker
> > > >
> > > >
> > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > Hi Gerd
> > > > >
> > > > > I've done a bit of investigation and debugging. It seems to
> > > > > be
> > > > > building
> > > > > the indexes with enough cities and streets. MDR20 is the
> > > > > suspect
> > > > > index.
> > > > >
> > > > > I'm now experimenting with mkgmap / Display
> > > > >
> > > > > With just the suspect map built in gmapsupp:
> > > > >
> > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > filter[]
> > > > > Copying .               65024
> > > > > Copying MAKEGMAP.MPS    78
> > > > > Copying 74440044.RGN    4154927
> > > > > Copying 74440044.TRE    18772
> > > > > Copying 74440044.LBL    2174485
> > > > > Copying 74440044.NET    1626350
> > > > > Copying 74440044.NOD    3566829
> > > > > Copying 00007444.MDR    2979295
> > > > > Copying 00007444.SRT    912
> > > > >
> > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > initial values    7    2    e
> > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > >
> > > > > seems fine but
> > > > >
> > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > ---------- 00007444.MDR --------------------
> > > > > EXTRA
> > > > > remaining {}
> > > > > # reading 74440044
> > > > > mdr5 check
> > > > > Exception in thread "main"
> > > > > java.lang.IndexOutOfBoundsException:
> > > > > Index:
> > > > > 308736, Size: 70018
> > > > >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > >         at
> > > > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > > > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > > >         at
> > > > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > > > >         at
> > > > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > > > >         at
> > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > + exit
> > > > >
> > > > > On another map from the split this runs OK, producing masses
> > > > > of
> > > > > output:
> > > > >
> > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > ---------- 00007429.MDR --------------------
> > > > > EXTRA
> > > > > remaining {}
> > > > > # reading 74290047
> > > > > ERROR: map 1: city 1: no name; index 1
> > > > > mdr5 check
> > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > > KINGDOM^]GBR)
> > > > > ind20=0 rep=false
> > > > > ... 87000 more lines ...
> > > > >
> > > > >
> > > > > Do you think mkgmap:Display is finding a real problem and I
> > > > > should
> > > > > pursue this line, or could it be unreliable and this
> > > > > exception
> > > > > is
> > > > > due
> > > > > to it rather than the format of the index?
> > > > >
> > > > > Thanks
> > > > > Ticker
> > > > >
> > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > > Ticker Berkin wrote
> > > > > > > With a particular split of 'british-isles', one map from
> > > > > > > the
> > > > > > > split
> > > > > > > causes the list of cities in the Find>Address function to
> > > > > > > be
> > > > > > > empty.
> > > > > >
> > > > > > Maybe you have the same problem here which I described for
> > > > > > road
> > > > > > name search when road names contain special characters. In
> > > > > > your
> > > > > > case
> > > > > > this would mean special characters in the citiy names.
> > > > > > I suggest to add debug code in
> > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWri
> > > > > > te
> > > > > > r
> > > > > > writer)
> > > > > > to print all city names which are written to the global
> > > > > > index.
> > > > > >
> > > > > > Gerd
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > View this message in context:
> > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > Sent from the Mkgmap Development mailing list archive at
> > > > > > Nabble.com.
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi Tcker,

yes, good enough. I also thought about this alternative.

Gerd


________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Montag, 31. Juli 2017 13:09:49
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

I started fixing all the places where there is an assumption about the
number of cities (and zips) being < 65535 and I'm only part way through
my list of suspect files and have make a lot of changes so far.

I'm thinking it is not worth while because:

The overall MDR size jumps because all city entries change to 3 bytes.
I get a smaller GMAPSUPP by increasing the splitting so that it stays
in the 2 byte limit.

I might miss some subtle coding of the header flags and/or the
input/output.

It might not work anyway after I've made all the changes.

The changes are a bit messy because I've just hacked each problem area
with a fix. Really there should be some utility functions, enhancements
to the readers/writers and pointerSize variables in the relevant
classes.

I propose just to signal an error in individual maps when the exceed
the limit

What do you think?

Regards
Ticker

On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:

> Hi Gerd
>
> I'll just done a bit more investigation and see what seems to be the
> the same assumption in imgfmt/app/net/RoadDef.java around line 296
>                                         if(numCities > 255)
>
>  writer.putChar(cityIndex);
>                                         else
>
>  writer.put((byte)cityIndex);
>
> I'll fix and test tomorrow and if there are still have problems I'll
> assemble the problem map, style etc for you
>
> Thanks
> Ticker
>
> On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > hmm, I think the patch looks good and I have no idea why the search
> > doesn't work
> > for you. Yesterday I thought the patch handles the overall number
> > of
> > cities in the gmapsupp,
> > but it is about the number of cities in a single map tile.
> > Maybe there is another problem caused by the high number of cities.
> > Maybe you can post a link to the tile with
> > that high number?
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > Quite possibly, or maybe there are limits in the device.
> >
> > More testing reveals that although my lists of cities and streets
> > within them appear correctly in Find>Address, it doesn't find
> > anything.
> >
> > Ticker
> >
> > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > Hi Tcker,
> > >
> > > thanks for the patch. I seem to remember that the number of
> > > entries
> > > in mdr5 is also used
> > > elsewhere. Will have a closer look tomorrow.
> > >
> > > Gerd
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > It looks as if can simply write 1/2/3 bytes as have indicated in
> > > the
> > > magic field in the MDR header and I've changed it to do this. All
> > > seems
> > > OK.
> > >
> > > Yes - I do misuse a City POI. But now, after having worked out
> > > how
> > > the
> > > MDR20 index works, I might be able to do what I want in a better
> > > way.
> > >
> > > Attached is patch to write the correct number of bytes.
> > >
> > > Regards
> > > Ticker
> > >
> > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > yes, sounds plausible. The comment seems to say that we don't
> > > > know
> > > > how to handle so many cities.
> > > > Why do you have so many cities. Do you "abuse" the
> > > > corresponding
> > > > POI
> > > > for something?
> > > >
> > > > Gerd
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > An: [hidden email]
> > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > >
> > > > Hi Gerd
> > > >
> > > > I think I've found the problem:
> > > >
> > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > >
> > > >         private void putLocalCityIndex(ImgFileWriter writer,
> > > > int
> > > > cityIndex) {
> > > >                 if (localCitySize == 2)
> > > >                 // 3 probably not possible in actual maps.
> > > >                         writer.putChar((char) cityIndex);
> > > >                 else
> > > >                         writer.put((byte) cityIndex);
> > > >         }
> > > >
> > > > There were more than 63335 cities in the failing map.
> > > >
> > > > Ticker
> > > >
> > > >
> > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > Hi Gerd
> > > > >
> > > > > I've done a bit of investigation and debugging. It seems to
> > > > > be
> > > > > building
> > > > > the indexes with enough cities and streets. MDR20 is the
> > > > > suspect
> > > > > index.
> > > > >
> > > > > I'm now experimenting with mkgmap / Display
> > > > >
> > > > > With just the suspect map built in gmapsupp:
> > > > >
> > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > filter[]
> > > > > Copying .               65024
> > > > > Copying MAKEGMAP.MPS    78
> > > > > Copying 74440044.RGN    4154927
> > > > > Copying 74440044.TRE    18772
> > > > > Copying 74440044.LBL    2174485
> > > > > Copying 74440044.NET    1626350
> > > > > Copying 74440044.NOD    3566829
> > > > > Copying 00007444.MDR    2979295
> > > > > Copying 00007444.SRT    912
> > > > >
> > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > initial values    7    2    e
> > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > >
> > > > > seems fine but
> > > > >
> > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > ---------- 00007444.MDR --------------------
> > > > > EXTRA
> > > > > remaining {}
> > > > > # reading 74440044
> > > > > mdr5 check
> > > > > Exception in thread "main"
> > > > > java.lang.IndexOutOfBoundsException:
> > > > > Index:
> > > > > 308736, Size: 70018
> > > > >         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > >         at
> > > > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > > > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > > >         at
> > > > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > > > >         at
> > > > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > > > >         at
> > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > + exit
> > > > >
> > > > > On another map from the split this runs OK, producing masses
> > > > > of
> > > > > output:
> > > > >
> > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > ---------- 00007429.MDR --------------------
> > > > > EXTRA
> > > > > remaining {}
> > > > > # reading 74290047
> > > > > ERROR: map 1: city 1: no name; index 1
> > > > > mdr5 check
> > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED
> > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > > KINGDOM^]GBR)
> > > > > ind20=0 rep=false
> > > > > ... 87000 more lines ...
> > > > >
> > > > >
> > > > > Do you think mkgmap:Display is finding a real problem and I
> > > > > should
> > > > > pursue this line, or could it be unreliable and this
> > > > > exception
> > > > > is
> > > > > due
> > > > > to it rather than the format of the index?
> > > > >
> > > > > Thanks
> > > > > Ticker
> > > > >
> > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > > Ticker Berkin wrote
> > > > > > > With a particular split of 'british-isles', one map from
> > > > > > > the
> > > > > > > split
> > > > > > > causes the list of cities in the Find>Address function to
> > > > > > > be
> > > > > > > empty.
> > > > > >
> > > > > > Maybe you have the same problem here which I described for
> > > > > > road
> > > > > > name search when road names contain special characters. In
> > > > > > your
> > > > > > case
> > > > > > this would mean special characters in the citiy names.
> > > > > > I suggest to add debug code in
> > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileWri
> > > > > > te
> > > > > > r
> > > > > > writer)
> > > > > > to print all city names which are written to the global
> > > > > > index.
> > > > > >
> > > > > > Gerd
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > View this message in context:
> > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > Sent from the Mkgmap Development mailing list archive at
> > > > > > Nabble.com.
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

I decided to both fix the code and put out a warning message if there
are structures that switch to 3 byte pointers (zips, highways and exit
facilities as well as cities)

Address search now works when there are > 65535 cities.

The code is a lot tidier - I've done it properly and it gets rid of
lots of messy bits of in-line testing against sizes > 255 etc.

Ticker


On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:

> Hi Tcker,
>
> yes, good enough. I also thought about this alternative.
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Montag, 31. Juli 2017 13:09:49
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> I started fixing all the places where there is an assumption about
> the
> number of cities (and zips) being < 65535 and I'm only part way
> through
> my list of suspect files and have make a lot of changes so far.
>
> I'm thinking it is not worth while because:
>
> The overall MDR size jumps because all city entries change to 3
> bytes.
> I get a smaller GMAPSUPP by increasing the splitting so that it stays
> in the 2 byte limit.
>
> I might miss some subtle coding of the header flags and/or the
> input/output.
>
> It might not work anyway after I've made all the changes.
>
> The changes are a bit messy because I've just hacked each problem
> area
> with a fix. Really there should be some utility functions,
> enhancements
> to the readers/writers and pointerSize variables in the relevant
> classes.
>
> I propose just to signal an error in individual maps when the exceed
> the limit
>
> What do you think?
>
> Regards
> Ticker
>
> On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
> > Hi Gerd
> >
> > I'll just done a bit more investigation and see what seems to be
> > the
> > the same assumption in imgfmt/app/net/RoadDef.java around line 296
> >                                         if(numCities > 255)
> >
> >  writer.putChar(cityIndex);
> >                                         else
> >
> >  writer.put((byte)cityIndex);
> >
> > I'll fix and test tomorrow and if there are still have problems
> > I'll
> > assemble the problem map, style etc for you
> >
> > Thanks
> > Ticker
> >
> > On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > hmm, I think the patch looks good and I have no idea why the
> > > search
> > > doesn't work
> > > for you. Yesterday I thought the patch handles the overall number
> > > of
> > > cities in the gmapsupp,
> > > but it is about the number of cities in a single map tile.
> > > Maybe there is another problem caused by the high number of
> > > cities.
> > > Maybe you can post a link to the tile with
> > > that high number?
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > Quite possibly, or maybe there are limits in the device.
> > >
> > > More testing reveals that although my lists of cities and streets
> > > within them appear correctly in Find>Address, it doesn't find
> > > anything.
> > >
> > > Ticker
> > >
> > > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > > Hi Tcker,
> > > >
> > > > thanks for the patch. I seem to remember that the number of
> > > > entries
> > > > in mdr5 is also used
> > > > elsewhere. Will have a closer look tomorrow.
> > > >
> > > > Gerd
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > > An: [hidden email]
> > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > >
> > > > Hi Gerd
> > > >
> > > > It looks as if can simply write 1/2/3 bytes as have indicated
> > > > in
> > > > the
> > > > magic field in the MDR header and I've changed it to do this.
> > > > All
> > > > seems
> > > > OK.
> > > >
> > > > Yes - I do misuse a City POI. But now, after having worked out
> > > > how
> > > > the
> > > > MDR20 index works, I might be able to do what I want in a
> > > > better
> > > > way.
> > > >
> > > > Attached is patch to write the correct number of bytes.
> > > >
> > > > Regards
> > > > Ticker
> > > >
> > > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > >
> > > > > yes, sounds plausible. The comment seems to say that we don't
> > > > > know
> > > > > how to handle so many cities.
> > > > > Why do you have so many cities. Do you "abuse" the
> > > > > corresponding
> > > > > POI
> > > > > for something?
> > > > >
> > > > > Gerd
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > Auftrag
> > > > > von Ticker Berkin <[hidden email]>
> > > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > > An: [hidden email]
> > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > I think I've found the problem:
> > > > >
> > > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > > >
> > > > >         private void putLocalCityIndex(ImgFileWriter writer,
> > > > > int
> > > > > cityIndex) {
> > > > >                 if (localCitySize == 2)
> > > > >                 // 3 probably not possible in actual maps.
> > > > >                         writer.putChar((char) cityIndex);
> > > > >                 else
> > > > >                         writer.put((byte) cityIndex);
> > > > >         }
> > > > >
> > > > > There were more than 63335 cities in the failing map.
> > > > >
> > > > > Ticker
> > > > >
> > > > >
> > > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > > Hi Gerd
> > > > > >
> > > > > > I've done a bit of investigation and debugging. It seems to
> > > > > > be
> > > > > > building
> > > > > > the indexes with enough cities and streets. MDR20 is the
> > > > > > suspect
> > > > > > index.
> > > > > >
> > > > > > I'm now experimenting with mkgmap / Display
> > > > > >
> > > > > > With just the suspect map built in gmapsupp:
> > > > > >
> > > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > > filter[]
> > > > > > Copying .               65024
> > > > > > Copying MAKEGMAP.MPS    78
> > > > > > Copying 74440044.RGN    4154927
> > > > > > Copying 74440044.TRE    18772
> > > > > > Copying 74440044.LBL    2174485
> > > > > > Copying 74440044.NET    1626350
> > > > > > Copying 74440044.NOD    3566829
> > > > > > Copying 00007444.MDR    2979295
> > > > > > Copying 00007444.SRT    912
> > > > > >
> > > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > > initial values    7    2    e
> > > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > > >
> > > > > > seems fine but
> > > > > >
> > > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > > ---------- 00007444.MDR --------------------
> > > > > > EXTRA
> > > > > > remaining {}
> > > > > > # reading 74440044
> > > > > > mdr5 check
> > > > > > Exception in thread "main"
> > > > > > java.lang.IndexOutOfBoundsException:
> > > > > > Index:
> > > > > > 308736, Size: 70018
> > > > > >         at
> > > > > > java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > > >         at
> > > > > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > > > > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > > > >         at
> > > > > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > > > > >         at
> > > > > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > > > > >         at
> > > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > > + exit
> > > > > >
> > > > > > On another map from the split this runs OK, producing
> > > > > > masses
> > > > > > of
> > > > > > output:
> > > > > >
> > > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > > ---------- 00007429.MDR --------------------
> > > > > > EXTRA
> > > > > > remaining {}
> > > > > > # reading 74290047
> > > > > > ERROR: map 1: city 1: no name; index 1
> > > > > > mdr5 check
> > > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0
> > > > > > (UNITED
> > > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > > > KINGDOM^]GBR)
> > > > > > ind20=0 rep=false
> > > > > > ... 87000 more lines ...
> > > > > >
> > > > > >
> > > > > > Do you think mkgmap:Display is finding a real problem and I
> > > > > > should
> > > > > > pursue this line, or could it be unreliable and this
> > > > > > exception
> > > > > > is
> > > > > > due
> > > > > > to it rather than the format of the index?
> > > > > >
> > > > > > Thanks
> > > > > > Ticker
> > > > > >
> > > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > > > Ticker Berkin wrote
> > > > > > > > With a particular split of 'british-isles', one map
> > > > > > > > from
> > > > > > > > the
> > > > > > > > split
> > > > > > > > causes the list of cities in the Find>Address function
> > > > > > > > to
> > > > > > > > be
> > > > > > > > empty.
> > > > > > >
> > > > > > > Maybe you have the same problem here which I described
> > > > > > > for
> > > > > > > road
> > > > > > > name search when road names contain special characters.
> > > > > > > In
> > > > > > > your
> > > > > > > case
> > > > > > > this would mean special characters in the citiy names.
> > > > > > > I suggest to add debug code in
> > > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileW
> > > > > > > ri
> > > > > > > te
> > > > > > > r
> > > > > > > writer)
> > > > > > > to print all city names which are written to the global
> > > > > > > index.
> > > > > > >
> > > > > > > Gerd
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > View this message in context:
> > > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > > Sent from the Mkgmap Development mailing list archive at
> > > > > > > Nabble.com.
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

img3BytePtr.patch (48K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi Ticker,

sounds good, but I cannot compile:
ant clean test
gives
build-test:
    [mkdir] Created dir: D:\mkgmap\build\test
    [javac] Compiling 73 source files to D:\mkgmap\build\test
    [javac] D:\mkgmap\test\func\lib\ArrayImgWriter.java:26: error: ArrayImgWriter is not abstract and does not override abstract method putN(int,int) in ImgFileWriter
    [javac] public class ArrayImgWriter implements ImgFileWriter {
    [javac]        ^
    [javac] 1 error

ciao,
Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Dienstag, 1. August 2017 14:31:57
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

I decided to both fix the code and put out a warning message if there
are structures that switch to 3 byte pointers (zips, highways and exit
facilities as well as cities)

Address search now works when there are > 65535 cities.

The code is a lot tidier - I've done it properly and it gets rid of
lots of messy bits of in-line testing against sizes > 255 etc.

Ticker


On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:

> Hi Tcker,
>
> yes, good enough. I also thought about this alternative.
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Montag, 31. Juli 2017 13:09:49
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> I started fixing all the places where there is an assumption about
> the
> number of cities (and zips) being < 65535 and I'm only part way
> through
> my list of suspect files and have make a lot of changes so far.
>
> I'm thinking it is not worth while because:
>
> The overall MDR size jumps because all city entries change to 3
> bytes.
> I get a smaller GMAPSUPP by increasing the splitting so that it stays
> in the 2 byte limit.
>
> I might miss some subtle coding of the header flags and/or the
> input/output.
>
> It might not work anyway after I've made all the changes.
>
> The changes are a bit messy because I've just hacked each problem
> area
> with a fix. Really there should be some utility functions,
> enhancements
> to the readers/writers and pointerSize variables in the relevant
> classes.
>
> I propose just to signal an error in individual maps when the exceed
> the limit
>
> What do you think?
>
> Regards
> Ticker
>
> On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
> > Hi Gerd
> >
> > I'll just done a bit more investigation and see what seems to be
> > the
> > the same assumption in imgfmt/app/net/RoadDef.java around line 296
> >                                         if(numCities > 255)
> >
> >  writer.putChar(cityIndex);
> >                                         else
> >
> >  writer.put((byte)cityIndex);
> >
> > I'll fix and test tomorrow and if there are still have problems
> > I'll
> > assemble the problem map, style etc for you
> >
> > Thanks
> > Ticker
> >
> > On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > hmm, I think the patch looks good and I have no idea why the
> > > search
> > > doesn't work
> > > for you. Yesterday I thought the patch handles the overall number
> > > of
> > > cities in the gmapsupp,
> > > but it is about the number of cities in a single map tile.
> > > Maybe there is another problem caused by the high number of
> > > cities.
> > > Maybe you can post a link to the tile with
> > > that high number?
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > Quite possibly, or maybe there are limits in the device.
> > >
> > > More testing reveals that although my lists of cities and streets
> > > within them appear correctly in Find>Address, it doesn't find
> > > anything.
> > >
> > > Ticker
> > >
> > > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > > Hi Tcker,
> > > >
> > > > thanks for the patch. I seem to remember that the number of
> > > > entries
> > > > in mdr5 is also used
> > > > elsewhere. Will have a closer look tomorrow.
> > > >
> > > > Gerd
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > > An: [hidden email]
> > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > >
> > > > Hi Gerd
> > > >
> > > > It looks as if can simply write 1/2/3 bytes as have indicated
> > > > in
> > > > the
> > > > magic field in the MDR header and I've changed it to do this.
> > > > All
> > > > seems
> > > > OK.
> > > >
> > > > Yes - I do misuse a City POI. But now, after having worked out
> > > > how
> > > > the
> > > > MDR20 index works, I might be able to do what I want in a
> > > > better
> > > > way.
> > > >
> > > > Attached is patch to write the correct number of bytes.
> > > >
> > > > Regards
> > > > Ticker
> > > >
> > > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > >
> > > > > yes, sounds plausible. The comment seems to say that we don't
> > > > > know
> > > > > how to handle so many cities.
> > > > > Why do you have so many cities. Do you "abuse" the
> > > > > corresponding
> > > > > POI
> > > > > for something?
> > > > >
> > > > > Gerd
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > Auftrag
> > > > > von Ticker Berkin <[hidden email]>
> > > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > > An: [hidden email]
> > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > I think I've found the problem:
> > > > >
> > > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > > >
> > > > >         private void putLocalCityIndex(ImgFileWriter writer,
> > > > > int
> > > > > cityIndex) {
> > > > >                 if (localCitySize == 2)
> > > > >                 // 3 probably not possible in actual maps.
> > > > >                         writer.putChar((char) cityIndex);
> > > > >                 else
> > > > >                         writer.put((byte) cityIndex);
> > > > >         }
> > > > >
> > > > > There were more than 63335 cities in the failing map.
> > > > >
> > > > > Ticker
> > > > >
> > > > >
> > > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > > Hi Gerd
> > > > > >
> > > > > > I've done a bit of investigation and debugging. It seems to
> > > > > > be
> > > > > > building
> > > > > > the indexes with enough cities and streets. MDR20 is the
> > > > > > suspect
> > > > > > index.
> > > > > >
> > > > > > I'm now experimenting with mkgmap / Display
> > > > > >
> > > > > > With just the suspect map built in gmapsupp:
> > > > > >
> > > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > > filter[]
> > > > > > Copying .               65024
> > > > > > Copying MAKEGMAP.MPS    78
> > > > > > Copying 74440044.RGN    4154927
> > > > > > Copying 74440044.TRE    18772
> > > > > > Copying 74440044.LBL    2174485
> > > > > > Copying 74440044.NET    1626350
> > > > > > Copying 74440044.NOD    3566829
> > > > > > Copying 00007444.MDR    2979295
> > > > > > Copying 00007444.SRT    912
> > > > > >
> > > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > > initial values    7    2    e
> > > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > > >
> > > > > > seems fine but
> > > > > >
> > > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > > ---------- 00007444.MDR --------------------
> > > > > > EXTRA
> > > > > > remaining {}
> > > > > > # reading 74440044
> > > > > > mdr5 check
> > > > > > Exception in thread "main"
> > > > > > java.lang.IndexOutOfBoundsException:
> > > > > > Index:
> > > > > > 308736, Size: 70018
> > > > > >         at
> > > > > > java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > > >         at
> > > > > > test.display.check.MapDetails.getCity(MapDetails.java:188)
> > > > > >         at test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > > > >         at
> > > > > > test.display.CommonDisplay.display(CommonDisplay.java:171)
> > > > > >         at
> > > > > > test.display.CommonDisplay.display(CommonDisplay.java:196)
> > > > > >         at
> > > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > > + exit
> > > > > >
> > > > > > On another map from the split this runs OK, producing
> > > > > > masses
> > > > > > of
> > > > > > output:
> > > > > >
> > > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > > ---------- 00007429.MDR --------------------
> > > > > > EXTRA
> > > > > > remaining {}
> > > > > > # reading 74290047
> > > > > > ERROR: map 1: city 1: no name; index 1
> > > > > > mdr5 check
> > > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0
> > > > > > (UNITED
> > > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > > > KINGDOM^]GBR)
> > > > > > ind20=0 rep=false
> > > > > > ... 87000 more lines ...
> > > > > >
> > > > > >
> > > > > > Do you think mkgmap:Display is finding a real problem and I
> > > > > > should
> > > > > > pursue this line, or could it be unreliable and this
> > > > > > exception
> > > > > > is
> > > > > > due
> > > > > > to it rather than the format of the index?
> > > > > >
> > > > > > Thanks
> > > > > > Ticker
> > > > > >
> > > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > > > Ticker Berkin wrote
> > > > > > > > With a particular split of 'british-isles', one map
> > > > > > > > from
> > > > > > > > the
> > > > > > > > split
> > > > > > > > causes the list of cities in the Find>Address function
> > > > > > > > to
> > > > > > > > be
> > > > > > > > empty.
> > > > > > >
> > > > > > > Maybe you have the same problem here which I described
> > > > > > > for
> > > > > > > road
> > > > > > > name search when road names contain special characters.
> > > > > > > In
> > > > > > > your
> > > > > > > case
> > > > > > > this would mean special characters in the citiy names.
> > > > > > > I suggest to add debug code in
> > > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileW
> > > > > > > ri
> > > > > > > te
> > > > > > > r
> > > > > > > writer)
> > > > > > > to print all city names which are written to the global
> > > > > > > index.
> > > > > > >
> > > > > > > Gerd
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > View this message in context:
> > > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > > Sent from the Mkgmap Development mailing list archive at
> > > > > > > Nabble.com.
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin-2
Hi Gerd

Sorry - I did do "ant test", but this doesn't detect the missing
methods unless attempt made to call then.

New patch attached - I noticed that some of the new method comments
where not quite correct so I've improved these.

Regards
Ticker

On Thu, 2017-08-03 at 05:15 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> sounds good, but I cannot compile:
> ant clean test
> gives
> build-test:
>     [mkdir] Created dir: D:\mkgmap\build\test
>     [javac] Compiling 73 source files to D:\mkgmap\build\test
>     [javac] D:\mkgmap\test\func\lib\ArrayImgWriter.java:26: error:
> ArrayImgWriter is not abstract and does not override abstract method
> putN(int,int) in ImgFileWriter
>     [javac] public class ArrayImgWriter implements ImgFileWriter {
>     [javac]        ^
>     [javac] 1 error
>
> ciao,
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Dienstag, 1. August 2017 14:31:57
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> I decided to both fix the code and put out a warning message if there
> are structures that switch to 3 byte pointers (zips, highways and
> exit
> facilities as well as cities)
>
> Address search now works when there are > 65535 cities.
>
> The code is a lot tidier - I've done it properly and it gets rid of
> lots of messy bits of in-line testing against sizes > 255 etc.
>
> Ticker
>
>
> On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:
> > Hi Tcker,
> >
> > yes, good enough. I also thought about this alternative.
> >
> > Gerd
> >
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Montag, 31. Juli 2017 13:09:49
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > I started fixing all the places where there is an assumption about
> > the
> > number of cities (and zips) being < 65535 and I'm only part way
> > through
> > my list of suspect files and have make a lot of changes so far.
> >
> > I'm thinking it is not worth while because:
> >
> > The overall MDR size jumps because all city entries change to 3
> > bytes.
> > I get a smaller GMAPSUPP by increasing the splitting so that it
> > stays
> > in the 2 byte limit.
> >
> > I might miss some subtle coding of the header flags and/or the
> > input/output.
> >
> > It might not work anyway after I've made all the changes.
> >
> > The changes are a bit messy because I've just hacked each problem
> > area
> > with a fix. Really there should be some utility functions,
> > enhancements
> > to the readers/writers and pointerSize variables in the relevant
> > classes.
> >
> > I propose just to signal an error in individual maps when the
> > exceed
> > the limit
> >
> > What do you think?
> >
> > Regards
> > Ticker
> >
> > On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > >
> > > I'll just done a bit more investigation and see what seems to be
> > > the
> > > the same assumption in imgfmt/app/net/RoadDef.java around line
> > > 296
> > >                                         if(numCities > 255)
> > >
> > >  writer.putChar(cityIndex);
> > >                                         else
> > >
> > >  writer.put((byte)cityIndex);
> > >
> > > I'll fix and test tomorrow and if there are still have problems
> > > I'll
> > > assemble the problem map, style etc for you
> > >
> > > Thanks
> > > Ticker
> > >
> > > On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > hmm, I think the patch looks good and I have no idea why the
> > > > search
> > > > doesn't work
> > > > for you. Yesterday I thought the patch handles the overall
> > > > number
> > > > of
> > > > cities in the gmapsupp,
> > > > but it is about the number of cities in a single map tile.
> > > > Maybe there is another problem caused by the high number of
> > > > cities.
> > > > Maybe you can post a link to the tile with
> > > > that high number?
> > > >
> > > > Gerd
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > > > An: [hidden email]
> > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > >
> > > > Hi Gerd
> > > >
> > > > Quite possibly, or maybe there are limits in the device.
> > > >
> > > > More testing reveals that although my lists of cities and
> > > > streets
> > > > within them appear correctly in Find>Address, it doesn't find
> > > > anything.
> > > >
> > > > Ticker
> > > >
> > > > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > > > Hi Tcker,
> > > > >
> > > > > thanks for the patch. I seem to remember that the number of
> > > > > entries
> > > > > in mdr5 is also used
> > > > > elsewhere. Will have a closer look tomorrow.
> > > > >
> > > > > Gerd
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > Auftrag
> > > > > von Ticker Berkin <[hidden email]>
> > > > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > > > An: [hidden email]
> > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > It looks as if can simply write 1/2/3 bytes as have indicated
> > > > > in
> > > > > the
> > > > > magic field in the MDR header and I've changed it to do this.
> > > > > All
> > > > > seems
> > > > > OK.
> > > > >
> > > > > Yes - I do misuse a City POI. But now, after having worked
> > > > > out
> > > > > how
> > > > > the
> > > > > MDR20 index works, I might be able to do what I want in a
> > > > > better
> > > > > way.
> > > > >
> > > > > Attached is patch to write the correct number of bytes.
> > > > >
> > > > > Regards
> > > > > Ticker
> > > > >
> > > > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > > > Hi Ticker,
> > > > > >
> > > > > > yes, sounds plausible. The comment seems to say that we
> > > > > > don't
> > > > > > know
> > > > > > how to handle so many cities.
> > > > > > Why do you have so many cities. Do you "abuse" the
> > > > > > corresponding
> > > > > > POI
> > > > > > for something?
> > > > > >
> > > > > > Gerd
> > > > > > ________________________________________
> > > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > > Auftrag
> > > > > > von Ticker Berkin <[hidden email]>
> > > > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > > > An: [hidden email]
> > > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > > >
> > > > > > Hi Gerd
> > > > > >
> > > > > > I think I've found the problem:
> > > > > >
> > > > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > > > >
> > > > > >         private void putLocalCityIndex(ImgFileWriter
> > > > > > writer,
> > > > > > int
> > > > > > cityIndex) {
> > > > > >                 if (localCitySize == 2)
> > > > > >                 // 3 probably not possible in actual maps.
> > > > > >                         writer.putChar((char) cityIndex);
> > > > > >                 else
> > > > > >                         writer.put((byte) cityIndex);
> > > > > >         }
> > > > > >
> > > > > > There were more than 63335 cities in the failing map.
> > > > > >
> > > > > > Ticker
> > > > > >
> > > > > >
> > > > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > > > Hi Gerd
> > > > > > >
> > > > > > > I've done a bit of investigation and debugging. It seems
> > > > > > > to
> > > > > > > be
> > > > > > > building
> > > > > > > the indexes with enough cities and streets. MDR20 is the
> > > > > > > suspect
> > > > > > > index.
> > > > > > >
> > > > > > > I'm now experimenting with mkgmap / Display
> > > > > > >
> > > > > > > With just the suspect map built in gmapsupp:
> > > > > > >
> > > > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > > > filter[]
> > > > > > > Copying .               65024
> > > > > > > Copying MAKEGMAP.MPS    78
> > > > > > > Copying 74440044.RGN    4154927
> > > > > > > Copying 74440044.TRE    18772
> > > > > > > Copying 74440044.LBL    2174485
> > > > > > > Copying 74440044.NET    1626350
> > > > > > > Copying 74440044.NOD    3566829
> > > > > > > Copying 00007444.MDR    2979295
> > > > > > > Copying 00007444.SRT    912
> > > > > > >
> > > > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > > > initial values    7    2    e
> > > > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > > > >
> > > > > > > seems fine but
> > > > > > >
> > > > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > > > ---------- 00007444.MDR --------------------
> > > > > > > EXTRA
> > > > > > > remaining {}
> > > > > > > # reading 74440044
> > > > > > > mdr5 check
> > > > > > > Exception in thread "main"
> > > > > > > java.lang.IndexOutOfBoundsException:
> > > > > > > Index:
> > > > > > > 308736, Size: 70018
> > > > > > >         at
> > > > > > > java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > > > >         at
> > > > > > > test.display.check.MapDetails.getCity(MapDetails.java:188
> > > > > > > )
> > > > > > >         at
> > > > > > > test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > > > > >         at
> > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:171
> > > > > > > )
> > > > > > >         at
> > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:196
> > > > > > > )
> > > > > > >         at
> > > > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > > > + exit
> > > > > > >
> > > > > > > On another map from the split this runs OK, producing
> > > > > > > masses
> > > > > > > of
> > > > > > > output:
> > > > > > >
> > > > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > > > ---------- 00007429.MDR --------------------
> > > > > > > EXTRA
> > > > > > > remaining {}
> > > > > > > # reading 74290047
> > > > > > > ERROR: map 1: city 1: no name; index 1
> > > > > > > mdr5 check
> > > > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0
> > > > > > > (UNITED
> > > > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > > > > KINGDOM^]GBR)
> > > > > > > ind20=0 rep=false
> > > > > > > ... 87000 more lines ...
> > > > > > >
> > > > > > >
> > > > > > > Do you think mkgmap:Display is finding a real problem and
> > > > > > > I
> > > > > > > should
> > > > > > > pursue this line, or could it be unreliable and this
> > > > > > > exception
> > > > > > > is
> > > > > > > due
> > > > > > > to it rather than the format of the index?
> > > > > > >
> > > > > > > Thanks
> > > > > > > Ticker
> > > > > > >
> > > > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > > > > Ticker Berkin wrote
> > > > > > > > > With a particular split of 'british-isles', one map
> > > > > > > > > from
> > > > > > > > > the
> > > > > > > > > split
> > > > > > > > > causes the list of cities in the Find>Address
> > > > > > > > > function
> > > > > > > > > to
> > > > > > > > > be
> > > > > > > > > empty.
> > > > > > > >
> > > > > > > > Maybe you have the same problem here which I described
> > > > > > > > for
> > > > > > > > road
> > > > > > > > name search when road names contain special characters.
> > > > > > > > In
> > > > > > > > your
> > > > > > > > case
> > > > > > > > this would mean special characters in the citiy names.
> > > > > > > > I suggest to add debug code in
> > > > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFil
> > > > > > > > eW
> > > > > > > > ri
> > > > > > > > te
> > > > > > > > r
> > > > > > > > writer)
> > > > > > > > to print all city names which are written to the global
> > > > > > > > index.
> > > > > > > >
> > > > > > > > Gerd
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > View this message in context:
> > > > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > > > Sent from the Mkgmap Development mailing list archive
> > > > > > > > at
> > > > > > > > Nabble.com.
> > > > > > > > _______________________________________________
> > > > > > > > mkgmap-dev mailing list
> > > > > > > > [hidden email]
> > > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

img3BytePtr_v2.patch (50K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi  Ticker,

thanks. Please review:
You introduced new methods put1 and put2 in Interface ImgFileWriter and SectionWriter implements them but I don't find where they are used.

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Donnerstag, 3. August 2017 10:07:30
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

Sorry - I did do "ant test", but this doesn't detect the missing
methods unless attempt made to call then.

New patch attached - I noticed that some of the new method comments
where not quite correct so I've improved these.

Regards
Ticker

On Thu, 2017-08-03 at 05:15 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> sounds good, but I cannot compile:
> ant clean test
> gives
> build-test:
>     [mkdir] Created dir: D:\mkgmap\build\test
>     [javac] Compiling 73 source files to D:\mkgmap\build\test
>     [javac] D:\mkgmap\test\func\lib\ArrayImgWriter.java:26: error:
> ArrayImgWriter is not abstract and does not override abstract method
> putN(int,int) in ImgFileWriter
>     [javac] public class ArrayImgWriter implements ImgFileWriter {
>     [javac]        ^
>     [javac] 1 error
>
> ciao,
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Dienstag, 1. August 2017 14:31:57
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> I decided to both fix the code and put out a warning message if there
> are structures that switch to 3 byte pointers (zips, highways and
> exit
> facilities as well as cities)
>
> Address search now works when there are > 65535 cities.
>
> The code is a lot tidier - I've done it properly and it gets rid of
> lots of messy bits of in-line testing against sizes > 255 etc.
>
> Ticker
>
>
> On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:
> > Hi Tcker,
> >
> > yes, good enough. I also thought about this alternative.
> >
> > Gerd
> >
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Montag, 31. Juli 2017 13:09:49
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > I started fixing all the places where there is an assumption about
> > the
> > number of cities (and zips) being < 65535 and I'm only part way
> > through
> > my list of suspect files and have make a lot of changes so far.
> >
> > I'm thinking it is not worth while because:
> >
> > The overall MDR size jumps because all city entries change to 3
> > bytes.
> > I get a smaller GMAPSUPP by increasing the splitting so that it
> > stays
> > in the 2 byte limit.
> >
> > I might miss some subtle coding of the header flags and/or the
> > input/output.
> >
> > It might not work anyway after I've made all the changes.
> >
> > The changes are a bit messy because I've just hacked each problem
> > area
> > with a fix. Really there should be some utility functions,
> > enhancements
> > to the readers/writers and pointerSize variables in the relevant
> > classes.
> >
> > I propose just to signal an error in individual maps when the
> > exceed
> > the limit
> >
> > What do you think?
> >
> > Regards
> > Ticker
> >
> > On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > >
> > > I'll just done a bit more investigation and see what seems to be
> > > the
> > > the same assumption in imgfmt/app/net/RoadDef.java around line
> > > 296
> > >                                         if(numCities > 255)
> > >
> > >  writer.putChar(cityIndex);
> > >                                         else
> > >
> > >  writer.put((byte)cityIndex);
> > >
> > > I'll fix and test tomorrow and if there are still have problems
> > > I'll
> > > assemble the problem map, style etc for you
> > >
> > > Thanks
> > > Ticker
> > >
> > > On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > hmm, I think the patch looks good and I have no idea why the
> > > > search
> > > > doesn't work
> > > > for you. Yesterday I thought the patch handles the overall
> > > > number
> > > > of
> > > > cities in the gmapsupp,
> > > > but it is about the number of cities in a single map tile.
> > > > Maybe there is another problem caused by the high number of
> > > > cities.
> > > > Maybe you can post a link to the tile with
> > > > that high number?
> > > >
> > > > Gerd
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <[hidden email]> im
> > > > Auftrag
> > > > von Ticker Berkin <[hidden email]>
> > > > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > > > An: [hidden email]
> > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > >
> > > > Hi Gerd
> > > >
> > > > Quite possibly, or maybe there are limits in the device.
> > > >
> > > > More testing reveals that although my lists of cities and
> > > > streets
> > > > within them appear correctly in Find>Address, it doesn't find
> > > > anything.
> > > >
> > > > Ticker
> > > >
> > > > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > > > Hi Tcker,
> > > > >
> > > > > thanks for the patch. I seem to remember that the number of
> > > > > entries
> > > > > in mdr5 is also used
> > > > > elsewhere. Will have a closer look tomorrow.
> > > > >
> > > > > Gerd
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > Auftrag
> > > > > von Ticker Berkin <[hidden email]>
> > > > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > > > An: [hidden email]
> > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > It looks as if can simply write 1/2/3 bytes as have indicated
> > > > > in
> > > > > the
> > > > > magic field in the MDR header and I've changed it to do this.
> > > > > All
> > > > > seems
> > > > > OK.
> > > > >
> > > > > Yes - I do misuse a City POI. But now, after having worked
> > > > > out
> > > > > how
> > > > > the
> > > > > MDR20 index works, I might be able to do what I want in a
> > > > > better
> > > > > way.
> > > > >
> > > > > Attached is patch to write the correct number of bytes.
> > > > >
> > > > > Regards
> > > > > Ticker
> > > > >
> > > > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > > > Hi Ticker,
> > > > > >
> > > > > > yes, sounds plausible. The comment seems to say that we
> > > > > > don't
> > > > > > know
> > > > > > how to handle so many cities.
> > > > > > Why do you have so many cities. Do you "abuse" the
> > > > > > corresponding
> > > > > > POI
> > > > > > for something?
> > > > > >
> > > > > > Gerd
> > > > > > ________________________________________
> > > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > > Auftrag
> > > > > > von Ticker Berkin <[hidden email]>
> > > > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > > > An: [hidden email]
> > > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > > >
> > > > > > Hi Gerd
> > > > > >
> > > > > > I think I've found the problem:
> > > > > >
> > > > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > > > >
> > > > > >         private void putLocalCityIndex(ImgFileWriter
> > > > > > writer,
> > > > > > int
> > > > > > cityIndex) {
> > > > > >                 if (localCitySize == 2)
> > > > > >                 // 3 probably not possible in actual maps.
> > > > > >                         writer.putChar((char) cityIndex);
> > > > > >                 else
> > > > > >                         writer.put((byte) cityIndex);
> > > > > >         }
> > > > > >
> > > > > > There were more than 63335 cities in the failing map.
> > > > > >
> > > > > > Ticker
> > > > > >
> > > > > >
> > > > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > > > Hi Gerd
> > > > > > >
> > > > > > > I've done a bit of investigation and debugging. It seems
> > > > > > > to
> > > > > > > be
> > > > > > > building
> > > > > > > the indexes with enough cities and streets. MDR20 is the
> > > > > > > suspect
> > > > > > > index.
> > > > > > >
> > > > > > > I'm now experimenting with mkgmap / Display
> > > > > > >
> > > > > > > With just the suspect map built in gmapsupp:
> > > > > > >
> > > > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > > > filter[]
> > > > > > > Copying .               65024
> > > > > > > Copying MAKEGMAP.MPS    78
> > > > > > > Copying 74440044.RGN    4154927
> > > > > > > Copying 74440044.TRE    18772
> > > > > > > Copying 74440044.LBL    2174485
> > > > > > > Copying 74440044.NET    1626350
> > > > > > > Copying 74440044.NOD    3566829
> > > > > > > Copying 00007444.MDR    2979295
> > > > > > > Copying 00007444.SRT    912
> > > > > > >
> > > > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > > > initial values    7    2    e
> > > > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > > > >
> > > > > > > seems fine but
> > > > > > >
> > > > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > > > ---------- 00007444.MDR --------------------
> > > > > > > EXTRA
> > > > > > > remaining {}
> > > > > > > # reading 74440044
> > > > > > > mdr5 check
> > > > > > > Exception in thread "main"
> > > > > > > java.lang.IndexOutOfBoundsException:
> > > > > > > Index:
> > > > > > > 308736, Size: 70018
> > > > > > >         at
> > > > > > > java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > > > >         at
> > > > > > > test.display.check.MapDetails.getCity(MapDetails.java:188
> > > > > > > )
> > > > > > >         at
> > > > > > > test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > > > >         at test.display.MdrCheck.print(MdrCheck.java:87)
> > > > > > >         at
> > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:171
> > > > > > > )
> > > > > > >         at
> > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:196
> > > > > > > )
> > > > > > >         at
> > > > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > > > >         at test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > > > + exit
> > > > > > >
> > > > > > > On another map from the split this runs OK, producing
> > > > > > > masses
> > > > > > > of
> > > > > > > output:
> > > > > > >
> > > > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > > > ---------- 00007429.MDR --------------------
> > > > > > > EXTRA
> > > > > > > remaining {}
> > > > > > > # reading 74290047
> > > > > > > ERROR: map 1: city 1: no name; index 1
> > > > > > > mdr5 check
> > > > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0
> > > > > > > (UNITED
> > > > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED
> > > > > > > KINGDOM^]GBR)
> > > > > > > ind20=0 rep=false
> > > > > > > ... 87000 more lines ...
> > > > > > >
> > > > > > >
> > > > > > > Do you think mkgmap:Display is finding a real problem and
> > > > > > > I
> > > > > > > should
> > > > > > > pursue this line, or could it be unreliable and this
> > > > > > > exception
> > > > > > > is
> > > > > > > due
> > > > > > > to it rather than the format of the index?
> > > > > > >
> > > > > > > Thanks
> > > > > > > Ticker
> > > > > > >
> > > > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote:
> > > > > > > > Ticker Berkin wrote
> > > > > > > > > With a particular split of 'british-isles', one map
> > > > > > > > > from
> > > > > > > > > the
> > > > > > > > > split
> > > > > > > > > causes the list of cities in the Find>Address
> > > > > > > > > function
> > > > > > > > > to
> > > > > > > > > be
> > > > > > > > > empty.
> > > > > > > >
> > > > > > > > Maybe you have the same problem here which I described
> > > > > > > > for
> > > > > > > > road
> > > > > > > > name search when road names contain special characters.
> > > > > > > > In
> > > > > > > > your
> > > > > > > > case
> > > > > > > > this would mean special characters in the citiy names.
> > > > > > > > I suggest to add debug code in
> > > > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFil
> > > > > > > > eW
> > > > > > > > ri
> > > > > > > > te
> > > > > > > > r
> > > > > > > > writer)
> > > > > > > > to print all city names which are written to the global
> > > > > > > > index.
> > > > > > > >
> > > > > > > > Gerd
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > View this message in context:
> > > > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > > > Sent from the Mkgmap Development mailing list archive
> > > > > > > > at
> > > > > > > > Nabble.com.
> > > > > > > > _______________________________________________
> > > > > > > > mkgmap-dev mailing list
> > > > > > > > [hidden email]
> > > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Ticker Berkin
Hi Gerd

I introduced them as methods to be used where unsigned bytes/words need
to be written and the range verified with an assertion, rather than
(byte)/(char) casts in the main code that simply discard the unwanted
bytes.

Almost all put()/putChar() in imgfmt/* should use these new methods,
but I didn't want to make excess changes while solving the city...
problem.

Maybe the names of these, which I chose to match existing put3() and
the names in ImageFileReader - where different methods are really
required for signed and unsigned - should be made consistent.

Regards
Ticker  

On Thu, 2017-08-03 at 08:38 +0000, Gerd Petermann wrote:

> Hi  Ticker,
>
> thanks. Please review:
> You introduced new methods put1 and put2 in Interface ImgFileWriter
> and SectionWriter implements them but I don't find where they are
> used.
>
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Donnerstag, 3. August 2017 10:07:30
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> Sorry - I did do "ant test", but this doesn't detect the missing
> methods unless attempt made to call then.
>
> New patch attached - I noticed that some of the new method comments
> where not quite correct so I've improved these.
>
> Regards
> Ticker
>
> On Thu, 2017-08-03 at 05:15 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > sounds good, but I cannot compile:
> > ant clean test
> > gives
> > build-test:
> >     [mkdir] Created dir: D:\mkgmap\build\test
> >     [javac] Compiling 73 source files to D:\mkgmap\build\test
> >     [javac] D:\mkgmap\test\func\lib\ArrayImgWriter.java:26: error:
> > ArrayImgWriter is not abstract and does not override abstract
> > method
> > putN(int,int) in ImgFileWriter
> >     [javac] public class ArrayImgWriter implements ImgFileWriter {
> >     [javac]        ^
> >     [javac] 1 error
> >
> > ciao,
> > Gerd
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Dienstag, 1. August 2017 14:31:57
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > I decided to both fix the code and put out a warning message if
> > there
> > are structures that switch to 3 byte pointers (zips, highways and
> > exit
> > facilities as well as cities)
> >
> > Address search now works when there are > 65535 cities.
> >
> > The code is a lot tidier - I've done it properly and it gets rid of
> > lots of messy bits of in-line testing against sizes > 255 etc.
> >
> > Ticker
> >
> >
> > On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:
> > > Hi Tcker,
> > >
> > > yes, good enough. I also thought about this alternative.
> > >
> > > Gerd
> > >
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Montag, 31. Juli 2017 13:09:49
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > I started fixing all the places where there is an assumption
> > > about
> > > the
> > > number of cities (and zips) being < 65535 and I'm only part way
> > > through
> > > my list of suspect files and have make a lot of changes so far.
> > >
> > > I'm thinking it is not worth while because:
> > >
> > > The overall MDR size jumps because all city entries change to 3
> > > bytes.
> > > I get a smaller GMAPSUPP by increasing the splitting so that it
> > > stays
> > > in the 2 byte limit.
> > >
> > > I might miss some subtle coding of the header flags and/or the
> > > input/output.
> > >
> > > It might not work anyway after I've made all the changes.
> > >
> > > The changes are a bit messy because I've just hacked each problem
> > > area
> > > with a fix. Really there should be some utility functions,
> > > enhancements
> > > to the readers/writers and pointerSize variables in the relevant
> > > classes.
> > >
> > > I propose just to signal an error in individual maps when the
> > > exceed
> > > the limit
> > >
> > > What do you think?
> > >
> > > Regards
> > > Ticker
> > >
> > > On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
> > > > Hi Gerd
> > > >
> > > > I'll just done a bit more investigation and see what seems to
> > > > be
> > > > the
> > > > the same assumption in imgfmt/app/net/RoadDef.java around line
> > > > 296
> > > >                                         if(numCities > 255)
> > > >
> > > >  writer.putChar(cityIndex);
> > > >                                         else
> > > >
> > > >  writer.put((byte)cityIndex);
> > > >
> > > > I'll fix and test tomorrow and if there are still have problems
> > > > I'll
> > > > assemble the problem map, style etc for you
> > > >
> > > > Thanks
> > > > Ticker
> > > >
> > > > On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > >
> > > > > hmm, I think the patch looks good and I have no idea why the
> > > > > search
> > > > > doesn't work
> > > > > for you. Yesterday I thought the patch handles the overall
> > > > > number
> > > > > of
> > > > > cities in the gmapsupp,
> > > > > but it is about the number of cities in a single map tile.
> > > > > Maybe there is another problem caused by the high number of
> > > > > cities.
> > > > > Maybe you can post a link to the tile with
> > > > > that high number?
> > > > >
> > > > > Gerd
> > > > >
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > Auftrag
> > > > > von Ticker Berkin <[hidden email]>
> > > > > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > > > > An: [hidden email]
> > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > Quite possibly, or maybe there are limits in the device.
> > > > >
> > > > > More testing reveals that although my lists of cities and
> > > > > streets
> > > > > within them appear correctly in Find>Address, it doesn't find
> > > > > anything.
> > > > >
> > > > > Ticker
> > > > >
> > > > > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > > > > Hi Tcker,
> > > > > >
> > > > > > thanks for the patch. I seem to remember that the number of
> > > > > > entries
> > > > > > in mdr5 is also used
> > > > > > elsewhere. Will have a closer look tomorrow.
> > > > > >
> > > > > > Gerd
> > > > > > ________________________________________
> > > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > > Auftrag
> > > > > > von Ticker Berkin <[hidden email]>
> > > > > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > > > > An: [hidden email]
> > > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > > >
> > > > > > Hi Gerd
> > > > > >
> > > > > > It looks as if can simply write 1/2/3 bytes as have
> > > > > > indicated
> > > > > > in
> > > > > > the
> > > > > > magic field in the MDR header and I've changed it to do
> > > > > > this.
> > > > > > All
> > > > > > seems
> > > > > > OK.
> > > > > >
> > > > > > Yes - I do misuse a City POI. But now, after having worked
> > > > > > out
> > > > > > how
> > > > > > the
> > > > > > MDR20 index works, I might be able to do what I want in a
> > > > > > better
> > > > > > way.
> > > > > >
> > > > > > Attached is patch to write the correct number of bytes.
> > > > > >
> > > > > > Regards
> > > > > > Ticker
> > > > > >
> > > > > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > > > > Hi Ticker,
> > > > > > >
> > > > > > > yes, sounds plausible. The comment seems to say that we
> > > > > > > don't
> > > > > > > know
> > > > > > > how to handle so many cities.
> > > > > > > Why do you have so many cities. Do you "abuse" the
> > > > > > > corresponding
> > > > > > > POI
> > > > > > > for something?
> > > > > > >
> > > > > > > Gerd
> > > > > > > ________________________________________
> > > > > > > Von: mkgmap-dev <[hidden email]>
> > > > > > > im
> > > > > > > Auftrag
> > > > > > > von Ticker Berkin <[hidden email]>
> > > > > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > > > > An: [hidden email]
> > > > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > > > >
> > > > > > > Hi Gerd
> > > > > > >
> > > > > > > I think I've found the problem:
> > > > > > >
> > > > > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > > > > >
> > > > > > >         private void putLocalCityIndex(ImgFileWriter
> > > > > > > writer,
> > > > > > > int
> > > > > > > cityIndex) {
> > > > > > >                 if (localCitySize == 2)
> > > > > > >                 // 3 probably not possible in actual
> > > > > > > maps.
> > > > > > >                         writer.putChar((char) cityIndex);
> > > > > > >                 else
> > > > > > >                         writer.put((byte) cityIndex);
> > > > > > >         }
> > > > > > >
> > > > > > > There were more than 63335 cities in the failing map.
> > > > > > >
> > > > > > > Ticker
> > > > > > >
> > > > > > >
> > > > > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > > > > Hi Gerd
> > > > > > > >
> > > > > > > > I've done a bit of investigation and debugging. It
> > > > > > > > seems
> > > > > > > > to
> > > > > > > > be
> > > > > > > > building
> > > > > > > > the indexes with enough cities and streets. MDR20 is
> > > > > > > > the
> > > > > > > > suspect
> > > > > > > > index.
> > > > > > > >
> > > > > > > > I'm now experimenting with mkgmap / Display
> > > > > > > >
> > > > > > > > With just the suspect map built in gmapsupp:
> > > > > > > >
> > > > > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > > > > filter[]
> > > > > > > > Copying .               65024
> > > > > > > > Copying MAKEGMAP.MPS    78
> > > > > > > > Copying 74440044.RGN    4154927
> > > > > > > > Copying 74440044.TRE    18772
> > > > > > > > Copying 74440044.LBL    2174485
> > > > > > > > Copying 74440044.NET    1626350
> > > > > > > > Copying 74440044.NOD    3566829
> > > > > > > > Copying 00007444.MDR    2979295
> > > > > > > > Copying 00007444.SRT    912
> > > > > > > >
> > > > > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > > > > initial values    7    2    e
> > > > > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > > > > >
> > > > > > > > seems fine but
> > > > > > > >
> > > > > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > > > > ---------- 00007444.MDR --------------------
> > > > > > > > EXTRA
> > > > > > > > remaining {}
> > > > > > > > # reading 74440044
> > > > > > > > mdr5 check
> > > > > > > > Exception in thread "main"
> > > > > > > > java.lang.IndexOutOfBoundsException:
> > > > > > > > Index:
> > > > > > > > 308736, Size: 70018
> > > > > > > >         at
> > > > > > > > java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > > > > >         at
> > > > > > > > test.display.check.MapDetails.getCity(MapDetails.java:1
> > > > > > > > 88
> > > > > > > > )
> > > > > > > >         at
> > > > > > > > test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > > > > >         at
> > > > > > > > test.display.MdrCheck.print(MdrCheck.java:87)
> > > > > > > >         at
> > > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:1
> > > > > > > > 71
> > > > > > > > )
> > > > > > > >         at
> > > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:1
> > > > > > > > 96
> > > > > > > > )
> > > > > > > >         at
> > > > > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > > > > >         at
> > > > > > > > test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > > > > + exit
> > > > > > > >
> > > > > > > > On another map from the split this runs OK, producing
> > > > > > > > masses
> > > > > > > > of
> > > > > > > > output:
> > > > > > > >
> > > > > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > > > > ---------- 00007429.MDR --------------------
> > > > > > > > EXTRA
> > > > > > > > remaining {}
> > > > > > > > # reading 74290047
> > > > > > > > ERROR: map 1: city 1: no name; index 1
> > > > > > > > mdr5 check
> > > > > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0
> > > > > > > > (UNITED
> > > > > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0
> > > > > > > > (UNITED
> > > > > > > > KINGDOM^]GBR)
> > > > > > > > ind20=0 rep=false
> > > > > > > > ... 87000 more lines ...
> > > > > > > >
> > > > > > > >
> > > > > > > > Do you think mkgmap:Display is finding a real problem
> > > > > > > > and
> > > > > > > > I
> > > > > > > > should
> > > > > > > > pursue this line, or could it be unreliable and this
> > > > > > > > exception
> > > > > > > > is
> > > > > > > > due
> > > > > > > > to it rather than the format of the index?
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Ticker
> > > > > > > >
> > > > > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann
> > > > > > > > wrote:
> > > > > > > > > Ticker Berkin wrote
> > > > > > > > > > With a particular split of 'british-isles', one map
> > > > > > > > > > from
> > > > > > > > > > the
> > > > > > > > > > split
> > > > > > > > > > causes the list of cities in the Find>Address
> > > > > > > > > > function
> > > > > > > > > > to
> > > > > > > > > > be
> > > > > > > > > > empty.
> > > > > > > > >
> > > > > > > > > Maybe you have the same problem here which I
> > > > > > > > > described
> > > > > > > > > for
> > > > > > > > > road
> > > > > > > > > name search when road names contain special
> > > > > > > > > characters.
> > > > > > > > > In
> > > > > > > > > your
> > > > > > > > > case
> > > > > > > > > this would mean special characters in the citiy
> > > > > > > > > names.
> > > > > > > > > I suggest to add debug code in
> > > > > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgF
> > > > > > > > > il
> > > > > > > > > eW
> > > > > > > > > ri
> > > > > > > > > te
> > > > > > > > > r
> > > > > > > > > writer)
> > > > > > > > > to print all city names which are written to the
> > > > > > > > > global
> > > > > > > > > index.
> > > > > > > > >
> > > > > > > > > Gerd
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > View this message in context:
> > > > > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > > > > Sent from the Mkgmap Development mailing list archive
> > > > > > > > > at
> > > > > > > > > Nabble.com.
> > > > > > > > > _______________________________________________
> > > > > > > > > mkgmap-dev mailing list
> > > > > > > > > [hidden email]
> > > > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > > _______________________________________________
> > > > > > > > mkgmap-dev mailing list
> > > > > > > > [hidden email]
> > > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Find Address - No cities

Gerd Petermann
Hi Ticker,

okay, I think I understand. I've reviewed all changes for copy+paste problems and did not find any.
I also created some test maps and found no difference in the img files besides the date fields, so I've
now committed it with r3978.

Gerd
________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Donnerstag, 3. August 2017 11:11:28
An: [hidden email]
Betreff: Re: [mkgmap-dev] Find Address - No cities

Hi Gerd

I introduced them as methods to be used where unsigned bytes/words need
to be written and the range verified with an assertion, rather than
(byte)/(char) casts in the main code that simply discard the unwanted
bytes.

Almost all put()/putChar() in imgfmt/* should use these new methods,
but I didn't want to make excess changes while solving the city...
problem.

Maybe the names of these, which I chose to match existing put3() and
the names in ImageFileReader - where different methods are really
required for signed and unsigned - should be made consistent.

Regards
Ticker

On Thu, 2017-08-03 at 08:38 +0000, Gerd Petermann wrote:

> Hi  Ticker,
>
> thanks. Please review:
> You introduced new methods put1 and put2 in Interface ImgFileWriter
> and SectionWriter implements them but I don't find where they are
> used.
>
> Gerd
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Donnerstag, 3. August 2017 10:07:30
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Find Address - No cities
>
> Hi Gerd
>
> Sorry - I did do "ant test", but this doesn't detect the missing
> methods unless attempt made to call then.
>
> New patch attached - I noticed that some of the new method comments
> where not quite correct so I've improved these.
>
> Regards
> Ticker
>
> On Thu, 2017-08-03 at 05:15 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > sounds good, but I cannot compile:
> > ant clean test
> > gives
> > build-test:
> >     [mkdir] Created dir: D:\mkgmap\build\test
> >     [javac] Compiling 73 source files to D:\mkgmap\build\test
> >     [javac] D:\mkgmap\test\func\lib\ArrayImgWriter.java:26: error:
> > ArrayImgWriter is not abstract and does not override abstract
> > method
> > putN(int,int) in ImgFileWriter
> >     [javac] public class ArrayImgWriter implements ImgFileWriter {
> >     [javac]        ^
> >     [javac] 1 error
> >
> > ciao,
> > Gerd
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Dienstag, 1. August 2017 14:31:57
> > An: [hidden email]
> > Betreff: Re: [mkgmap-dev] Find Address - No cities
> >
> > Hi Gerd
> >
> > I decided to both fix the code and put out a warning message if
> > there
> > are structures that switch to 3 byte pointers (zips, highways and
> > exit
> > facilities as well as cities)
> >
> > Address search now works when there are > 65535 cities.
> >
> > The code is a lot tidier - I've done it properly and it gets rid of
> > lots of messy bits of in-line testing against sizes > 255 etc.
> >
> > Ticker
> >
> >
> > On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:
> > > Hi Tcker,
> > >
> > > yes, good enough. I also thought about this alternative.
> > >
> > > Gerd
> > >
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <[hidden email]> im
> > > Auftrag
> > > von Ticker Berkin <[hidden email]>
> > > Gesendet: Montag, 31. Juli 2017 13:09:49
> > > An: [hidden email]
> > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > >
> > > Hi Gerd
> > >
> > > I started fixing all the places where there is an assumption
> > > about
> > > the
> > > number of cities (and zips) being < 65535 and I'm only part way
> > > through
> > > my list of suspect files and have make a lot of changes so far.
> > >
> > > I'm thinking it is not worth while because:
> > >
> > > The overall MDR size jumps because all city entries change to 3
> > > bytes.
> > > I get a smaller GMAPSUPP by increasing the splitting so that it
> > > stays
> > > in the 2 byte limit.
> > >
> > > I might miss some subtle coding of the header flags and/or the
> > > input/output.
> > >
> > > It might not work anyway after I've made all the changes.
> > >
> > > The changes are a bit messy because I've just hacked each problem
> > > area
> > > with a fix. Really there should be some utility functions,
> > > enhancements
> > > to the readers/writers and pointerSize variables in the relevant
> > > classes.
> > >
> > > I propose just to signal an error in individual maps when the
> > > exceed
> > > the limit
> > >
> > > What do you think?
> > >
> > > Regards
> > > Ticker
> > >
> > > On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
> > > > Hi Gerd
> > > >
> > > > I'll just done a bit more investigation and see what seems to
> > > > be
> > > > the
> > > > the same assumption in imgfmt/app/net/RoadDef.java around line
> > > > 296
> > > >                                         if(numCities > 255)
> > > >
> > > >  writer.putChar(cityIndex);
> > > >                                         else
> > > >
> > > >  writer.put((byte)cityIndex);
> > > >
> > > > I'll fix and test tomorrow and if there are still have problems
> > > > I'll
> > > > assemble the problem map, style etc for you
> > > >
> > > > Thanks
> > > > Ticker
> > > >
> > > > On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > >
> > > > > hmm, I think the patch looks good and I have no idea why the
> > > > > search
> > > > > doesn't work
> > > > > for you. Yesterday I thought the patch handles the overall
> > > > > number
> > > > > of
> > > > > cities in the gmapsupp,
> > > > > but it is about the number of cities in a single map tile.
> > > > > Maybe there is another problem caused by the high number of
> > > > > cities.
> > > > > Maybe you can post a link to the tile with
> > > > > that high number?
> > > > >
> > > > > Gerd
> > > > >
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > Auftrag
> > > > > von Ticker Berkin <[hidden email]>
> > > > > Gesendet: Samstag, 29. Juli 2017 21:12:27
> > > > > An: [hidden email]
> > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > Quite possibly, or maybe there are limits in the device.
> > > > >
> > > > > More testing reveals that although my lists of cities and
> > > > > streets
> > > > > within them appear correctly in Find>Address, it doesn't find
> > > > > anything.
> > > > >
> > > > > Ticker
> > > > >
> > > > > On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
> > > > > > Hi Tcker,
> > > > > >
> > > > > > thanks for the patch. I seem to remember that the number of
> > > > > > entries
> > > > > > in mdr5 is also used
> > > > > > elsewhere. Will have a closer look tomorrow.
> > > > > >
> > > > > > Gerd
> > > > > > ________________________________________
> > > > > > Von: mkgmap-dev <[hidden email]> im
> > > > > > Auftrag
> > > > > > von Ticker Berkin <[hidden email]>
> > > > > > Gesendet: Samstag, 29. Juli 2017 19:14:02
> > > > > > An: [hidden email]
> > > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > > >
> > > > > > Hi Gerd
> > > > > >
> > > > > > It looks as if can simply write 1/2/3 bytes as have
> > > > > > indicated
> > > > > > in
> > > > > > the
> > > > > > magic field in the MDR header and I've changed it to do
> > > > > > this.
> > > > > > All
> > > > > > seems
> > > > > > OK.
> > > > > >
> > > > > > Yes - I do misuse a City POI. But now, after having worked
> > > > > > out
> > > > > > how
> > > > > > the
> > > > > > MDR20 index works, I might be able to do what I want in a
> > > > > > better
> > > > > > way.
> > > > > >
> > > > > > Attached is patch to write the correct number of bytes.
> > > > > >
> > > > > > Regards
> > > > > > Ticker
> > > > > >
> > > > > > On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
> > > > > > > Hi Ticker,
> > > > > > >
> > > > > > > yes, sounds plausible. The comment seems to say that we
> > > > > > > don't
> > > > > > > know
> > > > > > > how to handle so many cities.
> > > > > > > Why do you have so many cities. Do you "abuse" the
> > > > > > > corresponding
> > > > > > > POI
> > > > > > > for something?
> > > > > > >
> > > > > > > Gerd
> > > > > > > ________________________________________
> > > > > > > Von: mkgmap-dev <[hidden email]>
> > > > > > > im
> > > > > > > Auftrag
> > > > > > > von Ticker Berkin <[hidden email]>
> > > > > > > Gesendet: Samstag, 29. Juli 2017 18:13:20
> > > > > > > An: [hidden email]
> > > > > > > Betreff: Re: [mkgmap-dev] Find Address - No cities
> > > > > > >
> > > > > > > Hi Gerd
> > > > > > >
> > > > > > > I think I've found the problem:
> > > > > > >
> > > > > > > /src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
> > > > > > >
> > > > > > >         private void putLocalCityIndex(ImgFileWriter
> > > > > > > writer,
> > > > > > > int
> > > > > > > cityIndex) {
> > > > > > >                 if (localCitySize == 2)
> > > > > > >                 // 3 probably not possible in actual
> > > > > > > maps.
> > > > > > >                         writer.putChar((char) cityIndex);
> > > > > > >                 else
> > > > > > >                         writer.put((byte) cityIndex);
> > > > > > >         }
> > > > > > >
> > > > > > > There were more than 63335 cities in the failing map.
> > > > > > >
> > > > > > > Ticker
> > > > > > >
> > > > > > >
> > > > > > > On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
> > > > > > > > Hi Gerd
> > > > > > > >
> > > > > > > > I've done a bit of investigation and debugging. It
> > > > > > > > seems
> > > > > > > > to
> > > > > > > > be
> > > > > > > > building
> > > > > > > > the indexes with enough cities and streets. MDR20 is
> > > > > > > > the
> > > > > > > > suspect
> > > > > > > > index.
> > > > > > > >
> > > > > > > > I'm now experimenting with mkgmap / Display
> > > > > > > >
> > > > > > > > With just the suspect map built in gmapsupp:
> > > > > > > >
> > > > > > > > $ ... test.ExtractFile gmapsupp.img
> > > > > > > > filter[]
> > > > > > > > Copying .               65024
> > > > > > > > Copying MAKEGMAP.MPS    78
> > > > > > > > Copying 74440044.RGN    4154927
> > > > > > > > Copying 74440044.TRE    18772
> > > > > > > > Copying 74440044.LBL    2174485
> > > > > > > > Copying 74440044.NET    1626350
> > > > > > > > Copying 74440044.NOD    3566829
> > > > > > > > Copying 00007444.MDR    2979295
> > > > > > > > Copying 00007444.SRT    912
> > > > > > > >
> > > > > > > > $ ... test.display.MdrSummary 00007444.MDR
> > > > > > > > initial values    7    2    e
> > > > > > > > MDR 1  NR=1     (000001) RS=4  magic=0x0
> > > > > > > > MDR 4  NR=101   (000065) RS=3  magic=0x0
> > > > > > > > MDR 5  NR=54456 (00d4b8) RS=9  magic=0x152
> > > > > > > > MDR 6  NR=4039  (000fc7) RS=3  magic=0x1
> > > > > > > > MDR 7  NR=17674 (00450a) RS=6  magic=0x66
> > > > > > > > MDR 9  NR=11    (00000b) RS=4  magic=0x0
> > > > > > > > MDR 10 DataSize=427616   (00068660) magic=0x0
> > > > > > > > MDR 11 NR=106904(01a198) RS=10 magic=0x95
> > > > > > > > MDR 12 NR=11    (00000b) RS=7  magic=0x40a
> > > > > > > > MDR 17 DataSize=111036   (0001b1bc) magic=0x0
> > > > > > > > MDR 18 NR=102   (000066) RS=5  magic=0x6
> > > > > > > > MDR 19 NR=106904(01a198) RS=3  magic=0x2
> > > > > > > > MDR 20 NR=20389 (004fa5) RS=6  magic=0xe
> > > > > > > > MDR 22 NR=18117 (0046c5) RS=6  magic=0xc000e
> > > > > > > > MDR 24 NR=2     (000002) RS=6  magic=0x0
> > > > > > > > MDR 25 NR=70016 (011180) RS=3  magic=0x0
> > > > > > > > MDR 29 NR=2     (000002) RS=8  magic=0x26
> > > > > > > >
> > > > > > > > seems fine but
> > > > > > > >
> > > > > > > > $ ... test.display.MdrCheck 00007444.MDR
> > > > > > > > ---------- 00007444.MDR --------------------
> > > > > > > > EXTRA
> > > > > > > > remaining {}
> > > > > > > > # reading 74440044
> > > > > > > > mdr5 check
> > > > > > > > Exception in thread "main"
> > > > > > > > java.lang.IndexOutOfBoundsException:
> > > > > > > > Index:
> > > > > > > > 308736, Size: 70018
> > > > > > > >         at
> > > > > > > > java.util.ArrayList.rangeCheck(ArrayList.java:653)
> > > > > > > >         at java.util.ArrayList.get(ArrayList.java:429)
> > > > > > > >         at
> > > > > > > > test.display.check.MapDetails.getCity(MapDetails.java:1
> > > > > > > > 88
> > > > > > > > )
> > > > > > > >         at
> > > > > > > > test.display.MdrCheck.check5(MdrCheck.java:282)
> > > > > > > >         at
> > > > > > > > test.display.MdrCheck.print(MdrCheck.java:87)
> > > > > > > >         at
> > > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:1
> > > > > > > > 71
> > > > > > > > )
> > > > > > > >         at
> > > > > > > > test.display.CommonDisplay.display(CommonDisplay.java:1
> > > > > > > > 96
> > > > > > > > )
> > > > > > > >         at
> > > > > > > > test.check.CommonCheck.runMain(CommonCheck.java:145)
> > > > > > > >         at
> > > > > > > > test.display.MdrCheck.main(MdrCheck.java:1789)
> > > > > > > > + exit
> > > > > > > >
> > > > > > > > On another map from the split this runs OK, producing
> > > > > > > > masses
> > > > > > > > of
> > > > > > > > output:
> > > > > > > >
> > > > > > > > $ ... test.display.MdrCheck 00007429.MDR
> > > > > > > > ---------- 00007429.MDR --------------------
> > > > > > > > EXTRA
> > > > > > > > remaining {}
> > > > > > > > # reading 74290047
> > > > > > > > ERROR: map 1: city 1: no name; index 1
> > > > > > > > mdr5 check
> > > > > > > > 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0
> > > > > > > > (UNITED
> > > > > > > > KINGDOM^]GBR) ind20=0 rep=false
> > > > > > > > 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0
> > > > > > > > (UNITED
> > > > > > > > KINGDOM^]GBR)
> > > > > > > > ind20=0 rep=false
> > > > > > > > ... 87000 more lines ...
> > > > > > > >
> > > > > > > >
> > > > > > > > Do you think mkgmap:Display is finding a real problem
> > > > > > > > and
> > > > > > > > I
> > > > > > > > should
> > > > > > > > pursue this line, or could it be unreliable and this
> > > > > > > > exception
> > > > > > > > is
> > > > > > > > due
> > > > > > > > to it rather than the format of the index?
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Ticker
> > > > > > > >
> > > > > > > > On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann
> > > > > > > > wrote:
> > > > > > > > > Ticker Berkin wrote
> > > > > > > > > > With a particular split of 'british-isles', one map
> > > > > > > > > > from
> > > > > > > > > > the
> > > > > > > > > > split
> > > > > > > > > > causes the list of cities in the Find>Address
> > > > > > > > > > function
> > > > > > > > > > to
> > > > > > > > > > be
> > > > > > > > > > empty.
> > > > > > > > >
> > > > > > > > > Maybe you have the same problem here which I
> > > > > > > > > described
> > > > > > > > > for
> > > > > > > > > road
> > > > > > > > > name search when road names contain special
> > > > > > > > > characters.
> > > > > > > > > In
> > > > > > > > > your
> > > > > > > > > case
> > > > > > > > > this would mean special characters in the citiy
> > > > > > > > > names.
> > > > > > > > > I suggest to add debug code in
> > > > > > > > > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgF
> > > > > > > > > il
> > > > > > > > > eW
> > > > > > > > > ri
> > > > > > > > > te
> > > > > > > > > r
> > > > > > > > > writer)
> > > > > > > > > to print all city names which are written to the
> > > > > > > > > global
> > > > > > > > > index.
> > > > > > > > >
> > > > > > > > > Gerd
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > View this message in context:
> > > > > > > > > http://gis.19327.n8.nabble.com/Find-Add
> > > > > > > > > ress-No-cities-tp5899775p5899880.html
> > > > > > > > > Sent from the Mkgmap Development mailing list archive
> > > > > > > > > at
> > > > > > > > > Nabble.com.
> > > > > > > > > _______________________________________________
> > > > > > > > > mkgmap-dev mailing list
> > > > > > > > > [hidden email]
> > > > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > > _______________________________________________
> > > > > > > > mkgmap-dev mailing list
> > > > > > > > [hidden email]
> > > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > _______________________________________________
> > > > > > > mkgmap-dev mailing list
> > > > > > > [hidden email]
> > > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > [hidden email]
> > > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > [hidden email]
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > [hidden email]
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > [hidden email]
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev