mkgmap r4025 implements --x-dem-poly option

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi all,

see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)

Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg

The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
I hope Frank has an idea.
It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
--x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
~300KB.

Gerd
P.S.
See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
I've moved some messages into the logger, so there will be less noise from mkgmap.
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Henning Scholland
Hi Gerd,

I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
files in Scandinavia. Unzipped it's all fine.

Btw. it would be nice to move the 'DEM file calculation for ... took 265
ms'-messages from stdout to logging Information.

Henning

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.nio.HeapByteBuffer.<init>(Unknown Source)
    at java.nio.ByteBuffer.allocate(Unknown Source)
    at
uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
    at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
    at
uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
    at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
    at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
    at
uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
    at
uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
    at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
    at
uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
    at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
    at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)

On 28.12.2017 18:18, Gerd Petermann wrote:

> Hi all,
>
> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>
> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>
> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
> I hope Frank has an idea.
> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
> ~300KB.
>
> Gerd
> P.S.
> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
> I've moved some messages into the logger, so there will be less noise from mkgmap.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Henning,
I think that the memory for normal Files is managed in a different, probably better way by the JRE.
Maybe I will find a better solution in the next days.
Gerd

---- Henning Scholland schrieb ----

Hi Gerd,

I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
files in Scandinavia. Unzipped it's all fine.

Btw. it would be nice to move the 'DEM file calculation for ... took 265
ms'-messages from stdout to logging Information.

Henning

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.nio.HeapByteBuffer.<init>(Unknown Source)
    at java.nio.ByteBuffer.allocate(Unknown Source)
    at
uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
    at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
    at
uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
    at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
    at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
    at
uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
    at
uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
    at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
    at
uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
    at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
    at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)

On 28.12.2017 18:18, Gerd Petermann wrote:
> Hi all,
>
> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>
> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>
> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
> I hope Frank has an idea.
> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
> ~300KB.
>
> Gerd
> P.S.
> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
> I've moved some messages into the logger, so there will be less noise from mkgmap.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Carlos Dávila-2
In reply to this post by Henning Scholland
--x-dem-poly option works like a charm, thanks!
I'm doing my tests with Bolivia map and there's some problem with
overview map generation. Log file is recording a lot of lines like "file
S19W069.hgt not found. Is expected to cover sea.", but the given hgt
*does exist* in the hgt folder. "Missing" hgt's are different in
different runs of mkgmap, see screenshots [1] and [2]. Watching at the
overview map, it seems mkgmap starts building DEM correctly, but at a
given point it fails to read hgt's and rest of DEM is not generated. DEM
for tiles in the "missing" areas are correctly generated. BuildDEMFile
creates correct overview, see [3].
[1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
|[2]
|||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
[3]
|||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||

El 31/12/17 a las 11:53, Henning Scholland escribió:

> Hi Gerd,
>
> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
> files in Scandinavia. Unzipped it's all fine.
>
> Btw. it would be nice to move the 'DEM file calculation for ... took 265
> ms'-messages from stdout to logging Information.
>
> Henning
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>      at java.nio.HeapByteBuffer.<init>(Unknown Source)
>      at java.nio.ByteBuffer.allocate(Unknown Source)
>      at
> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>      at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>      at
> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>      at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>      at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>      at
> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>      at
> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>      at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>      at
> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>      at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>      at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>
> On 28.12.2017 18:18, Gerd Petermann wrote:
>> Hi all,
>>
>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>
>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>
>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>> I hope Frank has an idea.
>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>> ~300KB.
>>
>> Gerd
>> P.S.
>> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

ligfietser

Thanks Gerd for implementing the poly option. It solves my issues with large ocean tiles too, by drawing a polygon around the (azores) islands I can keep the tiles big, the issues of memory errors are solved!



_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
In reply to this post by Carlos Dávila-2
Hi Carlos,

I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
and the try / catch block interprets the problem in the same way as a missing hgt file.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Dienstag, 2. Januar 2018 19:40:00
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

--x-dem-poly option works like a charm, thanks!
I'm doing my tests with Bolivia map and there's some problem with
overview map generation. Log file is recording a lot of lines like "file
S19W069.hgt not found. Is expected to cover sea.", but the given hgt
*does exist* in the hgt folder. "Missing" hgt's are different in
different runs of mkgmap, see screenshots [1] and [2]. Watching at the
overview map, it seems mkgmap starts building DEM correctly, but at a
given point it fails to read hgt's and rest of DEM is not generated. DEM
for tiles in the "missing" areas are correctly generated. BuildDEMFile
creates correct overview, see [3].
[1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
|[2]
|||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
[3]
|||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||

El 31/12/17 a las 11:53, Henning Scholland escribió:

> Hi Gerd,
>
> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
> files in Scandinavia. Unzipped it's all fine.
>
> Btw. it would be nice to move the 'DEM file calculation for ... took 265
> ms'-messages from stdout to logging Information.
>
> Henning
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>      at java.nio.HeapByteBuffer.<init>(Unknown Source)
>      at java.nio.ByteBuffer.allocate(Unknown Source)
>      at
> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>      at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>      at
> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>      at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>      at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>      at
> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>      at
> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>      at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>      at
> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>      at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>      at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>
> On 28.12.2017 18:18, Gerd Petermann wrote:
>> Hi all,
>>
>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>
>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>
>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>> I hope Frank has an idea.
>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>> ~300KB.
>>
>> Gerd
>> P.S.
>> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Carlos Dávila-2
Thanks Gerd. If you need input file or commands, just let me know.

El 03/01/18 a las 20:00, Gerd Petermann escribió:

> Hi Carlos,
>
> I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
> and the try / catch block interprets the problem in the same way as a missing hgt file.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Dienstag, 2. Januar 2018 19:40:00
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> --x-dem-poly option works like a charm, thanks!
> I'm doing my tests with Bolivia map and there's some problem with
> overview map generation. Log file is recording a lot of lines like "file
> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
> *does exist* in the hgt folder. "Missing" hgt's are different in
> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
> overview map, it seems mkgmap starts building DEM correctly, but at a
> given point it fails to read hgt's and rest of DEM is not generated. DEM
> for tiles in the "missing" areas are correctly generated. BuildDEMFile
> creates correct overview, see [3].
> [1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
> |[2]
> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
> [3]
> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>
> El 31/12/17 a las 11:53, Henning Scholland escribió:
>> Hi Gerd,
>>
>> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
>> files in Scandinavia. Unzipped it's all fine.
>>
>> Btw. it would be nice to move the 'DEM file calculation for ... took 265
>> ms'-messages from stdout to logging Information.
>>
>> Henning
>>
>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>       at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>       at java.nio.ByteBuffer.allocate(Unknown Source)
>>       at
>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>       at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>>       at
>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>>       at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>       at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>       at
>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>       at
>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>       at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>       at
>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>       at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>       at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>
>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>> Hi all,
>>>
>>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>>
>>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>
>>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>>> I hope Frank has an idea.
>>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>>> ~300KB.
>>>
>>> Gerd
>>> P.S.
>>> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>>
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Carlos,

I was not yet able to reproduce the problem. I've used the (unzipped) 3´´ files from http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
and my overview map is created with the default style. I tried different values for --x-overview-dem-dist and it worked without problems.
The only special option that I needed was --block-size=2048 for some tiles.
If you already use that please post further details about commands and maybe a link to your hgt files.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Mittwoch, 3. Januar 2018 20:15:14
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

Thanks Gerd. If you need input file or commands, just let me know.

El 03/01/18 a las 20:00, Gerd Petermann escribió:

> Hi Carlos,
>
> I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
> and the try / catch block interprets the problem in the same way as a missing hgt file.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Dienstag, 2. Januar 2018 19:40:00
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> --x-dem-poly option works like a charm, thanks!
> I'm doing my tests with Bolivia map and there's some problem with
> overview map generation. Log file is recording a lot of lines like "file
> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
> *does exist* in the hgt folder. "Missing" hgt's are different in
> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
> overview map, it seems mkgmap starts building DEM correctly, but at a
> given point it fails to read hgt's and rest of DEM is not generated. DEM
> for tiles in the "missing" areas are correctly generated. BuildDEMFile
> creates correct overview, see [3].
> [1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
> |[2]
> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
> [3]
> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>
> El 31/12/17 a las 11:53, Henning Scholland escribió:
>> Hi Gerd,
>>
>> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
>> files in Scandinavia. Unzipped it's all fine.
>>
>> Btw. it would be nice to move the 'DEM file calculation for ... took 265
>> ms'-messages from stdout to logging Information.
>>
>> Henning
>>
>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>       at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>       at java.nio.ByteBuffer.allocate(Unknown Source)
>>       at
>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>       at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>>       at
>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>>       at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>       at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>       at
>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>       at
>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>       at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>       at
>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>       at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>       at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>
>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>> Hi all,
>>>
>>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>>
>>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>
>>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>>> I hope Frank has an idea.
>>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>>> ~300KB.
>>>
>>> Gerd
>>> P.S.
>>> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>>
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

ligfietser
In reply to this post by Carlos Dávila-2
Hi Gerd,
Is it possible to skip dem processing for some specific tiles, for instance I have to add some contour tiles that needs to be processed each time with the same mkgmap version (cannot add it as already processed img).
I add them as overlay map on my Benelux map with this statement in my rendering process:
mkgmap.args -c %date%\splitter\template.args -c contours\contours.args

If I process it, those contours cause a map failure exception :

SEVERE (MapFailedException): contours\10010201.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048
SEVERE (BlockManager): contours\10010202.mp.gz: overflowed directory with max block 65534, current=65535
SEVERE (MapFailedException): contours\10010202.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048
SEVERE (BlockManager): contours\10010204.mp.gz: overflowed directory with max block 65534, current=65535
SEVERE (MapFailedException): contours\10010204.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048

In my contour.args file I have these lines:

mapname: 10010201
input-file: 10010201.mp.gz
mapname: 10010202
input-file: 10010202.mp.gz
mapname: 10010203
input-file: 10010203.mp.gz
mapname: 10010204
input-file: 10010204.mp.gz

How can I skip these files from DEM processing?


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Minko,

one way is to place the dem options in an extra file, e.g. dem-args, and use
mkgmap.args -c contours\contours.args -c dem-args -c %date%\splitter\template.args
so that the dem options are not active when the contour files are processed.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von lig fietser <[hidden email]>
Gesendet: Donnerstag, 4. Januar 2018 09:36:09
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

Hi Gerd,
Is it possible to skip dem processing for some specific tiles, for instance I have to add some contour tiles that needs to be processed each time with the same mkgmap version (cannot add it as already processed img).
I add them as overlay map on my Benelux map with this statement in my rendering process:
mkgmap.args -c %date%\splitter\template.args -c contours\contours.args

If I process it, those contours cause a map failure exception :

SEVERE (MapFailedException): contours\10010201.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048
SEVERE (BlockManager): contours\10010202.mp.gz: overflowed directory with max block 65534, current=65535
SEVERE (MapFailedException): contours\10010202.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048
SEVERE (BlockManager): contours\10010204.mp.gz: overflowed directory with max block 65534, current=65535
SEVERE (MapFailedException): contours\10010204.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048

In my contour.args file I have these lines:

mapname: 10010201
input-file: 10010201.mp.gz
mapname: 10010202
input-file: 10010202.mp.gz
mapname: 10010203
input-file: 10010203.mp.gz
mapname: 10010204
input-file: 10010204.mp.gz

How can I skip these files from DEM processing?

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

ligfietser

Thanks Gerd, I'll try that.


Van: mkgmap-dev <[hidden email]> namens Gerd Petermann <[hidden email]>
Verzonden: donderdag 4 januari 2018 01:24:51
Aan: Development list for mkgmap
Onderwerp: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
 
Hi Minko,

one way is to place the dem options in an extra file, e.g. dem-args, and use
mkgmap.args -c contours\contours.args -c dem-args -c %date%\splitter\template.args
so that the dem options are not active when the contour files are processed.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von lig fietser <[hidden email]>
Gesendet: Donnerstag, 4. Januar 2018 09:36:09
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

Hi Gerd,
Is it possible to skip dem processing for some specific tiles, for instance I have to add some contour tiles that needs to be processed each time with the same mkgmap version (cannot add it as already processed img).
I add them as overlay map on my Benelux map with this statement in my rendering process:
mkgmap.args -c %date%\splitter\template.args -c contours\contours.args

If I process it, those contours cause a map failure exception :

SEVERE (MapFailedException): contours\10010201.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048
SEVERE (BlockManager): contours\10010202.mp.gz: overflowed directory with max block 65534, current=65535
SEVERE (MapFailedException): contours\10010202.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048
SEVERE (BlockManager): contours\10010204.mp.gz: overflowed directory with max block 65534, current=65535
SEVERE (MapFailedException): contours\10010204.mp.gz: (thrown in BlockManager.allocate()) Too many blocks. Use a larger block size with an option such as --block-size=1024 or --block-size=2048

In my contour.args file I have these lines:

mapname: 10010201
input-file: 10010201.mp.gz
mapname: 10010202
input-file: 10010202.mp.gz
mapname: 10010203
input-file: 10010203.mp.gz
mapname: 10010204
input-file: 10010204.mp.gz

How can I skip these files from DEM processing?

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Carlos Dávila-2
In reply to this post by Gerd Petermann
Hi Gerd
I also use hgt files from viewfinderpanoramas. I split bolivia with
max-nodes=100000 and then run mkgmap as follows:
java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
--block-size=1024 --x-dem=../DEM/hgt --x-dem-poly=polygons/bolivia.poly
--x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
--gmapi --show-profiles=1 --output-dir=../mapas/$pais
--bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route --latin1
--code-page=1252 --country-name=$PAIS --country-abbr=$ABR
--area-name=$MAPA --family-name="OpenStreetMap $MAPA" --family-id=1$FID
--product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
--overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
--name-tag-list=$NAMETAG --index --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
--license-file=license_ODbL.txt --copyright-message="OpenStreetMap
contributors, ODbL. See: http://www.openstreetmap.org/copyright"
$LANGUAGE --road-name-config=$CONFIG
--x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
--remove-ovm-work-files=true --style-file=../resources/styles
--style=mio --check-styles -c $pais.args

Tomorrow I'll try with default style, just to see if it makes any difference

El 04/01/18 a las 07:35, Gerd Petermann escribió:

> Hi Carlos,
>
> I was not yet able to reproduce the problem. I've used the (unzipped) 3´´ files from http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
> and my overview map is created with the default style. I tried different values for --x-overview-dem-dist and it worked without problems.
> The only special option that I needed was --block-size=2048 for some tiles.
> If you already use that please post further details about commands and maybe a link to your hgt files.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> Thanks Gerd. If you need input file or commands, just let me know.
>
> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
>> and the try / catch block interprets the problem in the same way as a missing hgt file.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>> An: [hidden email]
>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>
>> --x-dem-poly option works like a charm, thanks!
>> I'm doing my tests with Bolivia map and there's some problem with
>> overview map generation. Log file is recording a lot of lines like "file
>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>> *does exist* in the hgt folder. "Missing" hgt's are different in
>> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
>> overview map, it seems mkgmap starts building DEM correctly, but at a
>> given point it fails to read hgt's and rest of DEM is not generated. DEM
>> for tiles in the "missing" areas are correctly generated. BuildDEMFile
>> creates correct overview, see [3].
>> [1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>> |[2]
>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>> [3]
>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>
>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>> Hi Gerd,
>>>
>>> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
>>> files in Scandinavia. Unzipped it's all fine.
>>>
>>> Btw. it would be nice to move the 'DEM file calculation for ... took 265
>>> ms'-messages from stdout to logging Information.
>>>
>>> Henning
>>>
>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>>        at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>        at java.nio.ByteBuffer.allocate(Unknown Source)
>>>        at
>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>        at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>>>        at
>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>>>        at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>        at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>        at
>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>        at
>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>        at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>        at
>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>        at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>        at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>
>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>> Hi all,
>>>>
>>>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>>>
>>>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>
>>>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>>>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>>>> I hope Frank has an idea.
>>>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>>>> ~300KB.
>>>>
>>>> Gerd
>>>> P.S.
>>>> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>>>
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Carlos,

I am now trying to use your options.
Maybe you use different options for splitter, with my options I have to use --block-size=2048 to avoid MapFailedException.
Please post the split-file created by splitter (areas.list).
Up to now I've tested with -Xmx6G and I think the value 2500m looks too low.
So, my 2nd suggestion is to try with a higher value.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Donnerstag, 4. Januar 2018 23:14:36
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

Hi Gerd
I also use hgt files from viewfinderpanoramas. I split bolivia with
max-nodes=100000 and then run mkgmap as follows:
java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
--block-size=1024 --x-dem=../DEM/hgt --x-dem-poly=polygons/bolivia.poly
--x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
--gmapi --show-profiles=1 --output-dir=../mapas/$pais
--bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route --latin1
--code-page=1252 --country-name=$PAIS --country-abbr=$ABR
--area-name=$MAPA --family-name="OpenStreetMap $MAPA" --family-id=1$FID
--product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
--overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
--name-tag-list=$NAMETAG --index --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
--license-file=license_ODbL.txt --copyright-message="OpenStreetMap
contributors, ODbL. See: http://www.openstreetmap.org/copyright"
$LANGUAGE --road-name-config=$CONFIG
--x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
--remove-ovm-work-files=true --style-file=../resources/styles
--style=mio --check-styles -c $pais.args

Tomorrow I'll try with default style, just to see if it makes any difference

El 04/01/18 a las 07:35, Gerd Petermann escribió:

> Hi Carlos,
>
> I was not yet able to reproduce the problem. I've used the (unzipped) 3´´ files from http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
> and my overview map is created with the default style. I tried different values for --x-overview-dem-dist and it worked without problems.
> The only special option that I needed was --block-size=2048 for some tiles.
> If you already use that please post further details about commands and maybe a link to your hgt files.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> Thanks Gerd. If you need input file or commands, just let me know.
>
> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
>> and the try / catch block interprets the problem in the same way as a missing hgt file.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>> An: [hidden email]
>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>
>> --x-dem-poly option works like a charm, thanks!
>> I'm doing my tests with Bolivia map and there's some problem with
>> overview map generation. Log file is recording a lot of lines like "file
>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>> *does exist* in the hgt folder. "Missing" hgt's are different in
>> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
>> overview map, it seems mkgmap starts building DEM correctly, but at a
>> given point it fails to read hgt's and rest of DEM is not generated. DEM
>> for tiles in the "missing" areas are correctly generated. BuildDEMFile
>> creates correct overview, see [3].
>> [1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>> |[2]
>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>> [3]
>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>
>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>> Hi Gerd,
>>>
>>> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
>>> files in Scandinavia. Unzipped it's all fine.
>>>
>>> Btw. it would be nice to move the 'DEM file calculation for ... took 265
>>> ms'-messages from stdout to logging Information.
>>>
>>> Henning
>>>
>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>>        at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>        at java.nio.ByteBuffer.allocate(Unknown Source)
>>>        at
>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>        at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>>>        at
>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>>>        at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>        at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>        at
>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>        at
>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>        at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>        at
>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>        at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>        at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>
>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>> Hi all,
>>>>
>>>> see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>>>
>>>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>
>>>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>>>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>>>> I hope Frank has an idea.
>>>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>>>> ~300KB.
>>>>
>>>> Gerd
>>>> P.S.
>>>> See also http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>>>
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Carlos Dávila-2
splitter options:
java -Xmx1500M -jar splitter.jar --max-nodes=$MAXNODES
--geonames-file=geonames/cities15000_$ABR.zip --mapid=551${FID}001
--output=o5m $pais.o5m

areas.list attached

Late this evening I'll try in another machine with more RAM available
and report.

El 05/01/18 a las 09:54, Gerd Petermann escribió:

> Hi Carlos,
>
> I am now trying to use your options.
> Maybe you use different options for splitter, with my options I have to use --block-size=2048 to avoid MapFailedException.
> Please post the split-file created by splitter (areas.list).
> Up to now I've tested with -Xmx6G and I think the value 2500m looks too low.
> So, my 2nd suggestion is to try with a higher value.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev<[hidden email]>  im Auftrag von Carlos Dávila<[hidden email]>
> Gesendet: Donnerstag, 4. Januar 2018 23:14:36
> An:[hidden email]
> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> Hi Gerd
> I also use hgt files from viewfinderpanoramas. I split bolivia with
> max-nodes=100000 and then run mkgmap as follows:
> java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
> --block-size=1024 --x-dem=../DEM/hgt --x-dem-poly=polygons/bolivia.poly
> --x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
> --gmapi --show-profiles=1 --output-dir=../mapas/$pais
> --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route --latin1
> --code-page=1252 --country-name=$PAIS --country-abbr=$ABR
> --area-name=$MAPA --family-name="OpenStreetMap $MAPA" --family-id=1$FID
> --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
> --overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
> --name-tag-list=$NAMETAG --index --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --license-file=license_ODbL.txt --copyright-message="OpenStreetMap
> contributors, ODbL. See:http://www.openstreetmap.org/copyright"
> $LANGUAGE --road-name-config=$CONFIG
> --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
> --remove-ovm-work-files=true --style-file=../resources/styles
> --style=mio --check-styles -c $pais.args
>
> Tomorrow I'll try with default style, just to see if it makes any difference
>
> El 04/01/18 a las 07:35, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I was not yet able to reproduce the problem. I've used the (unzipped) 3´´ files fromhttp://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
>> and my overview map is created with the default style. I tried different values for --x-overview-dem-dist and it worked without problems.
>> The only special option that I needed was --block-size=2048 for some tiles.
>> If you already use that please post further details about commands and maybe a link to your hgt files.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev<[hidden email]>  im Auftrag von Carlos Dávila<[hidden email]>
>> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
>> An:[hidden email]
>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>
>> Thanks Gerd. If you need input file or commands, just let me know.
>>
>> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
>>> and the try / catch block interprets the problem in the same way as a missing hgt file.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev<[hidden email]>  im Auftrag von Carlos Dávila<[hidden email]>
>>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>>> An:[hidden email]
>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>
>>> --x-dem-poly option works like a charm, thanks!
>>> I'm doing my tests with Bolivia map and there's some problem with
>>> overview map generation. Log file is recording a lot of lines like "file
>>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>>> *does exist* in the hgt folder. "Missing" hgt's are different in
>>> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
>>> overview map, it seems mkgmap starts building DEM correctly, but at a
>>> given point it fails to read hgt's and rest of DEM is not generated. DEM
>>> for tiles in the "missing" areas are correctly generated. BuildDEMFile
>>> creates correct overview, see [3].
>>> [1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>>> |[2]
>>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>>> [3]
>>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>>
>>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>>> Hi Gerd,
>>>>
>>>> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
>>>> files in Scandinavia. Unzipped it's all fine.
>>>>
>>>> Btw. it would be nice to move the 'DEM file calculation for ... took 265
>>>> ms'-messages from stdout to logging Information.
>>>>
>>>> Henning
>>>>
>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>>>         at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>>         at java.nio.ByteBuffer.allocate(Unknown Source)
>>>>         at
>>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>>         at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>>>>         at
>>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>>>>         at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>>         at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>>         at
>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>>         at
>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>>         at
>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>>
>>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>>> Hi all,
>>>>>
>>>>> seehttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>>>>
>>>>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>>
>>>>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>>>>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>>>>> I hope Frank has an idea.
>>>>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>>>>> ~300KB.
>>>>>
>>>>> Gerd
>>>>> P.S.
>>>>> See alsohttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>>>>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

areas.list (16K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Carlos,

ah, my mistake, I thought I read max-nodes=1000000 , which gives only 8 rather large tiles, you wrote 100000 -->  118 tiles
Trying again now..

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Freitag, 5. Januar 2018 12:46:03
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

splitter options:
java -Xmx1500M -jar splitter.jar --max-nodes=$MAXNODES
--geonames-file=geonames/cities15000_$ABR.zip --mapid=551${FID}001
--output=o5m $pais.o5m

areas.list attached

Late this evening I'll try in another machine with more RAM available
and report.

El 05/01/18 a las 09:54, Gerd Petermann escribió:

> Hi Carlos,
>
> I am now trying to use your options.
> Maybe you use different options for splitter, with my options I have to use --block-size=2048 to avoid MapFailedException.
> Please post the split-file created by splitter (areas.list).
> Up to now I've tested with -Xmx6G and I think the value 2500m looks too low.
> So, my 2nd suggestion is to try with a higher value.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev<[hidden email]>  im Auftrag von Carlos Dávila<[hidden email]>
> Gesendet: Donnerstag, 4. Januar 2018 23:14:36
> An:[hidden email]
> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> Hi Gerd
> I also use hgt files from viewfinderpanoramas. I split bolivia with
> max-nodes=100000 and then run mkgmap as follows:
> java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
> --block-size=1024 --x-dem=../DEM/hgt --x-dem-poly=polygons/bolivia.poly
> --x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
> --gmapi --show-profiles=1 --output-dir=../mapas/$pais
> --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route --latin1
> --code-page=1252 --country-name=$PAIS --country-abbr=$ABR
> --area-name=$MAPA --family-name="OpenStreetMap $MAPA" --family-id=1$FID
> --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
> --overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
> --name-tag-list=$NAMETAG --index --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --license-file=license_ODbL.txt --copyright-message="OpenStreetMap
> contributors, ODbL. See:http://www.openstreetmap.org/copyright"
> $LANGUAGE --road-name-config=$CONFIG
> --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
> --remove-ovm-work-files=true --style-file=../resources/styles
> --style=mio --check-styles -c $pais.args
>
> Tomorrow I'll try with default style, just to see if it makes any difference
>
> El 04/01/18 a las 07:35, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I was not yet able to reproduce the problem. I've used the (unzipped) 3´´ files fromhttp://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
>> and my overview map is created with the default style. I tried different values for --x-overview-dem-dist and it worked without problems.
>> The only special option that I needed was --block-size=2048 for some tiles.
>> If you already use that please post further details about commands and maybe a link to your hgt files.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev<[hidden email]>  im Auftrag von Carlos Dávila<[hidden email]>
>> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
>> An:[hidden email]
>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>
>> Thanks Gerd. If you need input file or commands, just let me know.
>>
>> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> I'll try to reproduce the problem tomorrow. Sounds as if something else goes wrong
>>> and the try / catch block interprets the problem in the same way as a missing hgt file.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev<[hidden email]>  im Auftrag von Carlos Dávila<[hidden email]>
>>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>>> An:[hidden email]
>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>
>>> --x-dem-poly option works like a charm, thanks!
>>> I'm doing my tests with Bolivia map and there's some problem with
>>> overview map generation. Log file is recording a lot of lines like "file
>>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>>> *does exist* in the hgt folder. "Missing" hgt's are different in
>>> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
>>> overview map, it seems mkgmap starts building DEM correctly, but at a
>>> given point it fails to read hgt's and rest of DEM is not generated. DEM
>>> for tiles in the "missing" areas are correctly generated. BuildDEMFile
>>> creates correct overview, see [3].
>>> [1] ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>>> |[2]
>>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>>> [3]
>>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>>
>>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>>> Hi Gerd,
>>>>
>>>> I got an OutOfMemory exception (see below) while reading zipped 1" SRTM
>>>> files in Scandinavia. Unzipped it's all fine.
>>>>
>>>> Btw. it would be nice to move the 'DEM file calculation for ... took 265
>>>> ms'-messages from stdout to logging Information.
>>>>
>>>> Henning
>>>>
>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>>>         at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>>         at java.nio.ByteBuffer.allocate(Unknown Source)
>>>>         at
>>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>>         at uk.me.parabola.mkgmap.reader.hgt.HGTReader.<init>(HGTReader.java:90)
>>>>         at
>>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.<init>(HGTConverter.java:66)
>>>>         at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>>         at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>>         at
>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>>         at
>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>>         at
>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>>
>>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>>> Hi all,
>>>>>
>>>>> seehttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>>> this is useful if you create a map for a country extract like nepal (nepal-latest.osm.pbf)
>>>>>
>>>>> Here is a screenshot of a map created with the option --x-dem-poly=nepal.poly
>>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>>
>>>>> The screenshot also shows some wrong reliefs, and I think I used files without many holes :-(
>>>>> So, next step is to find out what goes wrong with DEM in low resolutions. This might take longer,
>>>>> I hope Frank has an idea.
>>>>> It seems to be a good idea to use a rather high res for the overview map for now. For the screenshot I used
>>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the problems are gone and DEM file is still only
>>>>> ~300KB.
>>>>>
>>>>> Gerd
>>>>> P.S.
>>>>> See alsohttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>>> I've moved some messages into the logger, so there will be less noise from mkgmap.
>>>>>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Carlos,

I still cannot reproduce the problem. Maybe a difference in the JRE?
My java says
e:\bol2>java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

Gerd


Gerd Petermann wrote

> Hi Carlos,
>
> ah, my mistake, I thought I read max-nodes=1000000 , which gives only 8
> rather large tiles, you wrote 100000 -->  118 tiles
> Trying again now..
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev &lt;

> mkgmap-dev-bounces@.org

> &gt; im Auftrag von Carlos Dávila &lt;

> cdavilam@

> &gt;
> Gesendet: Freitag, 5. Januar 2018 12:46:03
> An:

> mkgmap-dev@.org

> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>
> splitter options:
> java -Xmx1500M -jar splitter.jar --max-nodes=$MAXNODES
> --geonames-file=geonames/cities15000_$ABR.zip --mapid=551${FID}001
> --output=o5m $pais.o5m
>
> areas.list attached
>
> Late this evening I'll try in another machine with more RAM available
> and report.
>
> El 05/01/18 a las 09:54, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I am now trying to use your options.
>> Maybe you use different options for splitter, with my options I have to
>> use --block-size=2048 to avoid MapFailedException.
>> Please post the split-file created by splitter (areas.list).
>> Up to now I've tested with -Xmx6G and I think the value 2500m looks too
>> low.
>> So, my 2nd suggestion is to try with a higher value.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev&lt;

> mkgmap-dev-bounces@.org

> &gt;  im Auftrag von Carlos Dávila&lt;

> cdavilam@

> &gt;
>> Gesendet: Donnerstag, 4. Januar 2018 23:14:36
>> An:

> mkgmap-dev@.org

>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>
>> Hi Gerd
>> I also use hgt files from viewfinderpanoramas. I split bolivia with
>> max-nodes=100000 and then run mkgmap as follows:
>> java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
>> --block-size=1024 --x-dem=../DEM/hgt --x-dem-poly=polygons/bolivia.poly
>> --x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
>> --gmapi --show-profiles=1 --output-dir=../mapas/$pais
>> --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route --latin1
>> --code-page=1252 --country-name=$PAIS --country-abbr=$ABR
>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA" --family-id=1$FID
>> --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
>> --overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
>> --name-tag-list=$NAMETAG --index --process-destination --process-exits
>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>> 18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>> --license-file=license_ODbL.txt --copyright-message="OpenStreetMap
>> contributors, ODbL. See:http://www.openstreetmap.org/copyright"
>> $LANGUAGE --road-name-config=$CONFIG
>> --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
>> --remove-ovm-work-files=true --style-file=../resources/styles
>> --style=mio --check-styles -c $pais.args
>>
>> Tomorrow I'll try with default style, just to see if it makes any
>> difference
>>
>> El 04/01/18 a las 07:35, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> I was not yet able to reproduce the problem. I've used the (unzipped)
>>> 3´´ files
>>> fromhttp://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
>>> and my overview map is created with the default style. I tried different
>>> values for --x-overview-dem-dist and it worked without problems.
>>> The only special option that I needed was --block-size=2048 for some
>>> tiles.
>>> If you already use that please post further details about commands and
>>> maybe a link to your hgt files.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev&lt;

> mkgmap-dev-bounces@.org

> &gt;  im Auftrag von Carlos Dávila&lt;

> cdavilam@

> &gt;
>>> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
>>> An:

> mkgmap-dev@.org

>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>
>>> Thanks Gerd. If you need input file or commands, just let me know.
>>>
>>> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> I'll try to reproduce the problem tomorrow. Sounds as if something else
>>>> goes wrong
>>>> and the try / catch block interprets the problem in the same way as a
>>>> missing hgt file.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev&lt;

> mkgmap-dev-bounces@.org

> &gt;  im Auftrag von Carlos Dávila&lt;

> cdavilam@

> &gt;
>>>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>>>> An:

> mkgmap-dev@.org

>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>
>>>> --x-dem-poly option works like a charm, thanks!
>>>> I'm doing my tests with Bolivia map and there's some problem with
>>>> overview map generation. Log file is recording a lot of lines like
>>>> "file
>>>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>>>> *does exist* in the hgt folder. "Missing" hgt's are different in
>>>> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
>>>> overview map, it seems mkgmap starts building DEM correctly, but at a
>>>> given point it fails to read hgt's and rest of DEM is not generated.
>>>> DEM
>>>> for tiles in the "missing" areas are correctly generated. BuildDEMFile
>>>> creates correct overview, see [3].
>>>> [1]
>>>> ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>>>> |[2]
>>>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>>>> [3]
>>>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>>>
>>>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>>>> Hi Gerd,
>>>>>
>>>>> I got an OutOfMemory exception (see below) while reading zipped 1"
>>>>> SRTM
>>>>> files in Scandinavia. Unzipped it's all fine.
>>>>>
>>>>> Btw. it would be nice to move the 'DEM file calculation for ... took
>>>>> 265
>>>>> ms'-messages from stdout to logging Information.
>>>>>
>>>>> Henning
>>>>>
>>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>>>>         at java.nio.HeapByteBuffer.
> <init>
> (Unknown Source)
>>>>>         at java.nio.ByteBuffer.allocate(Unknown Source)
>>>>>         at
>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>>>         at uk.me.parabola.mkgmap.reader.hgt.HGTReader.
> <init>
> (HGTReader.java:90)
>>>>>         at
>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.
> <init>
> (HGTConverter.java:66)
>>>>>         at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>>>         at
>>>>> uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>>>         at
>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>>>         at
>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>>>         at
>>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>>>
>>>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> seehttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>>>> this is useful if you create a map for a country extract like nepal
>>>>>> (nepal-latest.osm.pbf)
>>>>>>
>>>>>> Here is a screenshot of a map created with the option
>>>>>> --x-dem-poly=nepal.poly
>>>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>>>
>>>>>> The screenshot also shows some wrong reliefs, and I think I used
>>>>>> files without many holes :-(
>>>>>> So, next step is to find out what goes wrong with DEM in low
>>>>>> resolutions. This might take longer,
>>>>>> I hope Frank has an idea.
>>>>>> It seems to be a good idea to use a rather high res for the overview
>>>>>> map for now. For the screenshot I used
>>>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the
>>>>>> problems are gone and DEM file is still only
>>>>>> ~300KB.
>>>>>>
>>>>>> Gerd
>>>>>> P.S.
>>>>>> See
>>>>>> alsohttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>>>> I've moved some messages into the logger, so there will be less noise
>>>>>> from mkgmap.
>>>>>>
>
> _______________________________________________
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Carlos Dávila-2
My java:
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK Server VM (build 25.151-b12, mixed mode)

El 05/01/18 a las 15:23, Gerd Petermann escribió:

> Hi Carlos,
>
> I still cannot reproduce the problem. Maybe a difference in the JRE?
> My java says
> e:\bol2>java -version
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
>
> Gerd
>
>
> Gerd Petermann wrote
>> Hi Carlos,
>>
>> ah, my mistake, I thought I read max-nodes=1000000 , which gives only 8
>> rather large tiles, you wrote 100000 -->  118 tiles
>> Trying again now..
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev &lt;
>> mkgmap-dev-bounces@.org
>> &gt; im Auftrag von Carlos Dávila &lt;
>> cdavilam@
>> &gt;
>> Gesendet: Freitag, 5. Januar 2018 12:46:03
>> An:
>> mkgmap-dev@.org
>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>
>> splitter options:
>> java -Xmx1500M -jar splitter.jar --max-nodes=$MAXNODES
>> --geonames-file=geonames/cities15000_$ABR.zip --mapid=551${FID}001
>> --output=o5m $pais.o5m
>>
>> areas.list attached
>>
>> Late this evening I'll try in another machine with more RAM available
>> and report.
>>
>> El 05/01/18 a las 09:54, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> I am now trying to use your options.
>>> Maybe you use different options for splitter, with my options I have to
>>> use --block-size=2048 to avoid MapFailedException.
>>> Please post the split-file created by splitter (areas.list).
>>> Up to now I've tested with -Xmx6G and I think the value 2500m looks too
>>> low.
>>> So, my 2nd suggestion is to try with a higher value.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev&lt;
>> mkgmap-dev-bounces@.org
>> &gt;  im Auftrag von Carlos Dávila&lt;
>> cdavilam@
>> &gt;
>>> Gesendet: Donnerstag, 4. Januar 2018 23:14:36
>>> An:
>> mkgmap-dev@.org
>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>
>>> Hi Gerd
>>> I also use hgt files from viewfinderpanoramas. I split bolivia with
>>> max-nodes=100000 and then run mkgmap as follows:
>>> java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
>>> --block-size=1024 --x-dem=../DEM/hgt --x-dem-poly=polygons/bolivia.poly
>>> --x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
>>> --gmapi --show-profiles=1 --output-dir=../mapas/$pais
>>> --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route --latin1
>>> --code-page=1252 --country-name=$PAIS --country-abbr=$ABR
>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA" --family-id=1$FID
>>> --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
>>> --overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
>>> --name-tag-list=$NAMETAG --index --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>> --license-file=license_ODbL.txt --copyright-message="OpenStreetMap
>>> contributors, ODbL. See:http://www.openstreetmap.org/copyright"
>>> $LANGUAGE --road-name-config=$CONFIG
>>> --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
>>> --remove-ovm-work-files=true --style-file=../resources/styles
>>> --style=mio --check-styles -c $pais.args
>>>
>>> Tomorrow I'll try with default style, just to see if it makes any
>>> difference
>>>
>>> El 04/01/18 a las 07:35, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> I was not yet able to reproduce the problem. I've used the (unzipped)
>>>> 3´´ files
>>>> fromhttp://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
>>>> and my overview map is created with the default style. I tried different
>>>> values for --x-overview-dem-dist and it worked without problems.
>>>> The only special option that I needed was --block-size=2048 for some
>>>> tiles.
>>>> If you already use that please post further details about commands and
>>>> maybe a link to your hgt files.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev&lt;
>> mkgmap-dev-bounces@.org
>> &gt;  im Auftrag von Carlos Dávila&lt;
>> cdavilam@
>> &gt;
>>>> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
>>>> An:
>> mkgmap-dev@.org
>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>
>>>> Thanks Gerd. If you need input file or commands, just let me know.
>>>>
>>>> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>>>>> Hi Carlos,
>>>>>
>>>>> I'll try to reproduce the problem tomorrow. Sounds as if something else
>>>>> goes wrong
>>>>> and the try / catch block interprets the problem in the same way as a
>>>>> missing hgt file.
>>>>>
>>>>> Gerd
>>>>>
>>>>> ________________________________________
>>>>> Von: mkgmap-dev&lt;
>> mkgmap-dev-bounces@.org
>> &gt;  im Auftrag von Carlos Dávila&lt;
>> cdavilam@
>> &gt;
>>>>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>>>>> An:
>> mkgmap-dev@.org
>>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>>
>>>>> --x-dem-poly option works like a charm, thanks!
>>>>> I'm doing my tests with Bolivia map and there's some problem with
>>>>> overview map generation. Log file is recording a lot of lines like
>>>>> "file
>>>>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>>>>> *does exist* in the hgt folder. "Missing" hgt's are different in
>>>>> different runs of mkgmap, see screenshots [1] and [2]. Watching at the
>>>>> overview map, it seems mkgmap starts building DEM correctly, but at a
>>>>> given point it fails to read hgt's and rest of DEM is not generated.
>>>>> DEM
>>>>> for tiles in the "missing" areas are correctly generated. BuildDEMFile
>>>>> creates correct overview, see [3].
>>>>> [1]
>>>>> ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>>>>> |[2]
>>>>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>>>>> [3]
>>>>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>>>>
>>>>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>>>>> Hi Gerd,
>>>>>>
>>>>>> I got an OutOfMemory exception (see below) while reading zipped 1"
>>>>>> SRTM
>>>>>> files in Scandinavia. Unzipped it's all fine.
>>>>>>
>>>>>> Btw. it would be nice to move the 'DEM file calculation for ... took
>>>>>> 265
>>>>>> ms'-messages from stdout to logging Information.
>>>>>>
>>>>>> Henning
>>>>>>
>>>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>>>>>>          at java.nio.HeapByteBuffer.
>> <init>
>> (Unknown Source)
>>>>>>          at java.nio.ByteBuffer.allocate(Unknown Source)
>>>>>>          at
>>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>>>>          at uk.me.parabola.mkgmap.reader.hgt.HGTReader.
>> <init>
>> (HGTReader.java:90)
>>>>>>          at
>>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.
>> <init>
>> (HGTConverter.java:66)
>>>>>>          at uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>>>>          at
>>>>>> uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>>>>          at
>>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>>>>          at
>>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>>>>          at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>>>>          at
>>>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>>>>          at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>>>>          at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>>>>
>>>>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> seehttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>>>>> this is useful if you create a map for a country extract like nepal
>>>>>>> (nepal-latest.osm.pbf)
>>>>>>>
>>>>>>> Here is a screenshot of a map created with the option
>>>>>>> --x-dem-poly=nepal.poly
>>>>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>>>>
>>>>>>> The screenshot also shows some wrong reliefs, and I think I used
>>>>>>> files without many holes :-(
>>>>>>> So, next step is to find out what goes wrong with DEM in low
>>>>>>> resolutions. This might take longer,
>>>>>>> I hope Frank has an idea.
>>>>>>> It seems to be a good idea to use a rather high res for the overview
>>>>>>> map for now. For the screenshot I used
>>>>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the
>>>>>>> problems are gone and DEM file is still only
>>>>>>> ~300KB.
>>>>>>>
>>>>>>> Gerd
>>>>>>> P.S.
>>>>>>> See
>>>>>>> alsohttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>>>>> I've moved some messages into the logger, so there will be less noise
>>>>>>> from mkgmap.
>>>>>>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev@.org
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Carlos Dávila-2
I have now built Bolivia map in my server with the same command and hgt
files and it worked fine. Java is:
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-1~bpo8+1-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)

El 05/01/18 a las 16:01, Carlos Dávila escribió:

> My java:
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
> OpenJDK Server VM (build 25.151-b12, mixed mode)
>
> El 05/01/18 a las 15:23, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I still cannot reproduce the problem. Maybe a difference in the JRE?
>> My java says
>> e:\bol2>java -version
>> java version "1.8.0_144"
>> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
>>
>> Gerd
>>
>>
>> Gerd Petermann wrote
>>> Hi Carlos,
>>>
>>> ah, my mistake, I thought I read max-nodes=1000000 , which gives only 8
>>> rather large tiles, you wrote 100000 -->  118 tiles
>>> Trying again now..
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev &lt;
>>> mkgmap-dev-bounces@.org
>>> &gt; im Auftrag von Carlos Dávila &lt;
>>> cdavilam@
>>> &gt;
>>> Gesendet: Freitag, 5. Januar 2018 12:46:03
>>> An:
>>> mkgmap-dev@.org
>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>
>>> splitter options:
>>> java -Xmx1500M -jar splitter.jar --max-nodes=$MAXNODES
>>> --geonames-file=geonames/cities15000_$ABR.zip --mapid=551${FID}001
>>> --output=o5m $pais.o5m
>>>
>>> areas.list attached
>>>
>>> Late this evening I'll try in another machine with more RAM available
>>> and report.
>>>
>>> El 05/01/18 a las 09:54, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> I am now trying to use your options.
>>>> Maybe you use different options for splitter, with my options I
>>>> have to
>>>> use --block-size=2048 to avoid MapFailedException.
>>>> Please post the split-file created by splitter (areas.list).
>>>> Up to now I've tested with -Xmx6G and I think the value 2500m looks
>>>> too
>>>> low.
>>>> So, my 2nd suggestion is to try with a higher value.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev&lt;
>>> mkgmap-dev-bounces@.org
>>> &gt;  im Auftrag von Carlos Dávila&lt;
>>> cdavilam@
>>> &gt;
>>>> Gesendet: Donnerstag, 4. Januar 2018 23:14:36
>>>> An:
>>> mkgmap-dev@.org
>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>
>>>> Hi Gerd
>>>> I also use hgt files from viewfinderpanoramas. I split bolivia with
>>>> max-nodes=100000 and then run mkgmap as follows:
>>>> java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
>>>> --block-size=1024 --x-dem=../DEM/hgt
>>>> --x-dem-poly=polygons/bolivia.poly
>>>> --x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
>>>> --gmapi --show-profiles=1 --output-dir=../mapas/$pais
>>>> --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route
>>>> --latin1
>>>> --code-page=1252 --country-name=$PAIS --country-abbr=$ABR
>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>> --family-id=1$FID
>>>> --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
>>>> --overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
>>>> --name-tag-list=$NAMETAG --index --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>> --license-file=license_ODbL.txt --copyright-message="OpenStreetMap
>>>> contributors, ODbL. See:http://www.openstreetmap.org/copyright"
>>>> $LANGUAGE --road-name-config=$CONFIG
>>>> --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
>>>>
>>>> --remove-ovm-work-files=true --style-file=../resources/styles
>>>> --style=mio --check-styles -c $pais.args
>>>>
>>>> Tomorrow I'll try with default style, just to see if it makes any
>>>> difference
>>>>
>>>> El 04/01/18 a las 07:35, Gerd Petermann escribió:
>>>>> Hi Carlos,
>>>>>
>>>>> I was not yet able to reproduce the problem. I've used the (unzipped)
>>>>> 3´´ files
>>>>> fromhttp://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
>>>>>
>>>>> and my overview map is created with the default style. I tried
>>>>> different
>>>>> values for --x-overview-dem-dist and it worked without problems.
>>>>> The only special option that I needed was --block-size=2048 for some
>>>>> tiles.
>>>>> If you already use that please post further details about commands
>>>>> and
>>>>> maybe a link to your hgt files.
>>>>>
>>>>> Gerd
>>>>>
>>>>> ________________________________________
>>>>> Von: mkgmap-dev&lt;
>>> mkgmap-dev-bounces@.org
>>> &gt;  im Auftrag von Carlos Dávila&lt;
>>> cdavilam@
>>> &gt;
>>>>> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
>>>>> An:
>>> mkgmap-dev@.org
>>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>>
>>>>> Thanks Gerd. If you need input file or commands, just let me know.
>>>>>
>>>>> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>>>>>> Hi Carlos,
>>>>>>
>>>>>> I'll try to reproduce the problem tomorrow. Sounds as if
>>>>>> something else
>>>>>> goes wrong
>>>>>> and the try / catch block interprets the problem in the same way
>>>>>> as a
>>>>>> missing hgt file.
>>>>>>
>>>>>> Gerd
>>>>>>
>>>>>> ________________________________________
>>>>>> Von: mkgmap-dev&lt;
>>> mkgmap-dev-bounces@.org
>>> &gt;  im Auftrag von Carlos Dávila&lt;
>>> cdavilam@
>>> &gt;
>>>>>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>>>>>> An:
>>> mkgmap-dev@.org
>>>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly
>>>>>> option
>>>>>>
>>>>>> --x-dem-poly option works like a charm, thanks!
>>>>>> I'm doing my tests with Bolivia map and there's some problem with
>>>>>> overview map generation. Log file is recording a lot of lines like
>>>>>> "file
>>>>>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>>>>>> *does exist* in the hgt folder. "Missing" hgt's are different in
>>>>>> different runs of mkgmap, see screenshots [1] and [2]. Watching
>>>>>> at the
>>>>>> overview map, it seems mkgmap starts building DEM correctly, but
>>>>>> at a
>>>>>> given point it fails to read hgt's and rest of DEM is not generated.
>>>>>> DEM
>>>>>> for tiles in the "missing" areas are correctly generated.
>>>>>> BuildDEMFile
>>>>>> creates correct overview, see [3].
>>>>>> [1]
>>>>>> ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png 
>>>>>>
>>>>>> |[2]
>>>>>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png 
>>>>>>
>>>>>> [3]
>>>>>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>>>>>
>>>>>>
>>>>>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>>>>>> Hi Gerd,
>>>>>>>
>>>>>>> I got an OutOfMemory exception (see below) while reading zipped 1"
>>>>>>> SRTM
>>>>>>> files in Scandinavia. Unzipped it's all fine.
>>>>>>>
>>>>>>> Btw. it would be nice to move the 'DEM file calculation for ...
>>>>>>> took
>>>>>>> 265
>>>>>>> ms'-messages from stdout to logging Information.
>>>>>>>
>>>>>>> Henning
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>>>>>>> space
>>>>>>>          at java.nio.HeapByteBuffer.
>>> <init>
>>> (Unknown Source)
>>>>>>>          at java.nio.ByteBuffer.allocate(Unknown Source)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>>>>>
>>>>>>>          at uk.me.parabola.mkgmap.reader.hgt.HGTReader.
>>> <init>
>>> (HGTReader.java:90)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.
>>> <init>
>>> (HGTConverter.java:66)
>>>>>>>          at
>>>>>>> uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>>>>>
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>>>>>
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>>>>>
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>>>>>          at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>>>>>
>>>>>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> seehttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>>>>>>
>>>>>>>> this is useful if you create a map for a country extract like
>>>>>>>> nepal
>>>>>>>> (nepal-latest.osm.pbf)
>>>>>>>>
>>>>>>>> Here is a screenshot of a map created with the option
>>>>>>>> --x-dem-poly=nepal.poly
>>>>>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>>>>>
>>>>>>>> The screenshot also shows some wrong reliefs, and I think I used
>>>>>>>> files without many holes :-(
>>>>>>>> So, next step is to find out what goes wrong with DEM in low
>>>>>>>> resolutions. This might take longer,
>>>>>>>> I hope Frank has an idea.
>>>>>>>> It seems to be a good idea to use a rather high res for the
>>>>>>>> overview
>>>>>>>> map for now. For the screenshot I used
>>>>>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the
>>>>>>>> problems are gone and DEM file is still only
>>>>>>>> ~300KB.
>>>>>>>>
>>>>>>>> Gerd
>>>>>>>> P.S.
>>>>>>>> See
>>>>>>>> alsohttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>>>>>>
>>>>>>>> I've moved some messages into the logger, so there will be less
>>>>>>>> noise
>>>>>>>> from mkgmap.

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: mkgmap r4025 implements --x-dem-poly option

Gerd Petermann
Hi Carlos,

I've also tried r4033 with Oracle JRE "1.8.0_152" now and found no problems.
No idea what to do here.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Freitag, 5. Januar 2018 20:31:39
An: [hidden email]
Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option

I have now built Bolivia map in my server with the same command and hgt
files and it worked fine. Java is:
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-1~bpo8+1-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)

El 05/01/18 a las 16:01, Carlos Dávila escribió:

> My java:
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
> OpenJDK Server VM (build 25.151-b12, mixed mode)
>
> El 05/01/18 a las 15:23, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I still cannot reproduce the problem. Maybe a difference in the JRE?
>> My java says
>> e:\bol2>java -version
>> java version "1.8.0_144"
>> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
>>
>> Gerd
>>
>>
>> Gerd Petermann wrote
>>> Hi Carlos,
>>>
>>> ah, my mistake, I thought I read max-nodes=1000000 , which gives only 8
>>> rather large tiles, you wrote 100000 -->  118 tiles
>>> Trying again now..
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev &lt;
>>> mkgmap-dev-bounces@.org
>>> &gt; im Auftrag von Carlos Dávila &lt;
>>> cdavilam@
>>> &gt;
>>> Gesendet: Freitag, 5. Januar 2018 12:46:03
>>> An:
>>> mkgmap-dev@.org
>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>
>>> splitter options:
>>> java -Xmx1500M -jar splitter.jar --max-nodes=$MAXNODES
>>> --geonames-file=geonames/cities15000_$ABR.zip --mapid=551${FID}001
>>> --output=o5m $pais.o5m
>>>
>>> areas.list attached
>>>
>>> Late this evening I'll try in another machine with more RAM available
>>> and report.
>>>
>>> El 05/01/18 a las 09:54, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> I am now trying to use your options.
>>>> Maybe you use different options for splitter, with my options I
>>>> have to
>>>> use --block-size=2048 to avoid MapFailedException.
>>>> Please post the split-file created by splitter (areas.list).
>>>> Up to now I've tested with -Xmx6G and I think the value 2500m looks
>>>> too
>>>> low.
>>>> So, my 2nd suggestion is to try with a higher value.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev&lt;
>>> mkgmap-dev-bounces@.org
>>> &gt;  im Auftrag von Carlos Dávila&lt;
>>> cdavilam@
>>> &gt;
>>>> Gesendet: Donnerstag, 4. Januar 2018 23:14:36
>>>> An:
>>> mkgmap-dev@.org
>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>
>>>> Hi Gerd
>>>> I also use hgt files from viewfinderpanoramas. I split bolivia with
>>>> max-nodes=100000 and then run mkgmap as follows:
>>>> java -Xmx2500m -ea -Dlog.config=logging.properties -jar mkgmap-dem.jar
>>>> --block-size=1024 --x-dem=../DEM/hgt
>>>> --x-dem-poly=polygons/bolivia.poly
>>>> --x-dem-dists=3312,5846,8948,12646,20282 --x-overview-dem-dist=128000
>>>> --gmapi --show-profiles=1 --output-dir=../mapas/$pais
>>>> --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs=1 --route
>>>> --latin1
>>>> --code-page=1252 --country-name=$PAIS --country-abbr=$ABR
>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>> --family-id=1$FID
>>>> --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA"
>>>> --overview-mapname=$ABR-1${FID} --overview-mapnumber=551${FID}000
>>>> --name-tag-list=$NAMETAG --index --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --add-pois-to-areas --link-pois-to-ways
>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>> --license-file=license_ODbL.txt --copyright-message="OpenStreetMap
>>>> contributors, ODbL. See:http://www.openstreetmap.org/copyright"
>>>> $LANGUAGE --road-name-config=$CONFIG
>>>> --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED
>>>>
>>>> --remove-ovm-work-files=true --style-file=../resources/styles
>>>> --style=mio --check-styles -c $pais.args
>>>>
>>>> Tomorrow I'll try with default style, just to see if it makes any
>>>> difference
>>>>
>>>> El 04/01/18 a las 07:35, Gerd Petermann escribió:
>>>>> Hi Carlos,
>>>>>
>>>>> I was not yet able to reproduce the problem. I've used the (unzipped)
>>>>> 3´´ files
>>>>> fromhttp://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
>>>>>
>>>>> and my overview map is created with the default style. I tried
>>>>> different
>>>>> values for --x-overview-dem-dist and it worked without problems.
>>>>> The only special option that I needed was --block-size=2048 for some
>>>>> tiles.
>>>>> If you already use that please post further details about commands
>>>>> and
>>>>> maybe a link to your hgt files.
>>>>>
>>>>> Gerd
>>>>>
>>>>> ________________________________________
>>>>> Von: mkgmap-dev&lt;
>>> mkgmap-dev-bounces@.org
>>> &gt;  im Auftrag von Carlos Dávila&lt;
>>> cdavilam@
>>> &gt;
>>>>> Gesendet: Mittwoch, 3. Januar 2018 20:15:14
>>>>> An:
>>> mkgmap-dev@.org
>>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly option
>>>>>
>>>>> Thanks Gerd. If you need input file or commands, just let me know.
>>>>>
>>>>> El 03/01/18 a las 20:00, Gerd Petermann escribió:
>>>>>> Hi Carlos,
>>>>>>
>>>>>> I'll try to reproduce the problem tomorrow. Sounds as if
>>>>>> something else
>>>>>> goes wrong
>>>>>> and the try / catch block interprets the problem in the same way
>>>>>> as a
>>>>>> missing hgt file.
>>>>>>
>>>>>> Gerd
>>>>>>
>>>>>> ________________________________________
>>>>>> Von: mkgmap-dev&lt;
>>> mkgmap-dev-bounces@.org
>>> &gt;  im Auftrag von Carlos Dávila&lt;
>>> cdavilam@
>>> &gt;
>>>>>> Gesendet: Dienstag, 2. Januar 2018 19:40:00
>>>>>> An:
>>> mkgmap-dev@.org
>>>>>> Betreff: Re: [mkgmap-dev] mkgmap r4025 implements --x-dem-poly
>>>>>> option
>>>>>>
>>>>>> --x-dem-poly option works like a charm, thanks!
>>>>>> I'm doing my tests with Bolivia map and there's some problem with
>>>>>> overview map generation. Log file is recording a lot of lines like
>>>>>> "file
>>>>>> S19W069.hgt not found. Is expected to cover sea.", but the given hgt
>>>>>> *does exist* in the hgt folder. "Missing" hgt's are different in
>>>>>> different runs of mkgmap, see screenshots [1] and [2]. Watching
>>>>>> at the
>>>>>> overview map, it seems mkgmap starts building DEM correctly, but
>>>>>> at a
>>>>>> given point it fails to read hgt's and rest of DEM is not generated.
>>>>>> DEM
>>>>>> for tiles in the "missing" areas are correctly generated.
>>>>>> BuildDEMFile
>>>>>> creates correct overview, see [3].
>>>>>> [1]
>>>>>> ||http://files.mkgmap.org.uk/download/381/Screenshot_20180102_181555.png
>>>>>>
>>>>>> |[2]
>>>>>> |||http://files.mkgmap.org.uk/download/380/Screenshot_20180102_185917.png
>>>>>>
>>>>>> [3]
>>>>>> |||||http://files.mkgmap.org.uk/download/382/Screenshot_20180102_192228.png|||
>>>>>>
>>>>>>
>>>>>> El 31/12/17 a las 11:53, Henning Scholland escribió:
>>>>>>> Hi Gerd,
>>>>>>>
>>>>>>> I got an OutOfMemory exception (see below) while reading zipped 1"
>>>>>>> SRTM
>>>>>>> files in Scandinavia. Unzipped it's all fine.
>>>>>>>
>>>>>>> Btw. it would be nice to move the 'DEM file calculation for ...
>>>>>>> took
>>>>>>> 265
>>>>>>> ms'-messages from stdout to logging Information.
>>>>>>>
>>>>>>> Henning
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>>>>>>> space
>>>>>>>          at java.nio.HeapByteBuffer.
>>> <init>
>>> (Unknown Source)
>>>>>>>          at java.nio.ByteBuffer.allocate(Unknown Source)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTReader.extractFromZip(HGTReader.java:123)
>>>>>>>
>>>>>>>          at uk.me.parabola.mkgmap.reader.hgt.HGTReader.
>>> <init>
>>> (HGTReader.java:90)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.reader.hgt.HGTConverter.
>>> <init>
>>> (HGTConverter.java:66)
>>>>>>>          at
>>>>>>> uk.me.parabola.imgfmt.app.dem.DEMFile.calc(DEMFile.java:47)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:334)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:191)
>>>>>>>
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:103)
>>>>>>>
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:617)
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:128)
>>>>>>>
>>>>>>>          at
>>>>>>> uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:136)
>>>>>>>          at uk.me.parabola.mkgmap.main.Main.main(Main.java:107)
>>>>>>>
>>>>>>> On 28.12.2017 18:18, Gerd Petermann wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> seehttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4025
>>>>>>>>
>>>>>>>> this is useful if you create a map for a country extract like
>>>>>>>> nepal
>>>>>>>> (nepal-latest.osm.pbf)
>>>>>>>>
>>>>>>>> Here is a screenshot of a map created with the option
>>>>>>>> --x-dem-poly=nepal.poly
>>>>>>>> http://files.mkgmap.org.uk/download/378/nepal-with-poly.jpg
>>>>>>>>
>>>>>>>> The screenshot also shows some wrong reliefs, and I think I used
>>>>>>>> files without many holes :-(
>>>>>>>> So, next step is to find out what goes wrong with DEM in low
>>>>>>>> resolutions. This might take longer,
>>>>>>>> I hope Frank has an idea.
>>>>>>>> It seems to be a good idea to use a rather high res for the
>>>>>>>> overview
>>>>>>>> map for now. For the screenshot I used
>>>>>>>> --x-overview-dem-dist=276160, with --x-overview-dem-dist=88000 the
>>>>>>>> problems are gone and DEM file is still only
>>>>>>>> ~300KB.
>>>>>>>>
>>>>>>>> Gerd
>>>>>>>> P.S.
>>>>>>>> See
>>>>>>>> alsohttp://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&path=%2F&rev=4024
>>>>>>>>
>>>>>>>> I've moved some messages into the logger, so there will be less
>>>>>>>> noise
>>>>>>>> from mkgmap.

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev