Tiles pruned in DEM map

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Tiles pruned in DEM map

Carlos Dávila-4
I build several types of map (OSM, OSM+contour lines, maps for trucks
and OSM+DEM) for the same area, using same input files. I split
country.o5m with splitter and then use the same template.args output by
splitter for all maps, just changing mapname for the different types of
map. Given that, all resulting mapsets should have the same tiles, but
tiles in DEM map are smaller than in the other types. They share part of
the boundaries (usually two of them) with other types, but other
boundaries are moved in, reducing displayed tile size. See attached
screenshots. However, file size is the same (discounting *.DEM) for each
tile in *.gmap subfolders.

Command is quite similar for OSM and OSM+DEM maps:

java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
--precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
--area-name=$MAPA --family-name="OpenStreetMap $MAPA"
--family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
--overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
--name-tag-list=$NAMETAG --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
--report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
--check-roundabouts --check-roundabout-flares --license-file=license_OSM
--copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
--style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt

java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
--dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
--overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
--show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
--country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
--family-name="OSM $MAPA DEM" --family-id=5${FID}
--product-version=$VERSION --series-name="OSM-$MAPA-DEM"
--overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
--name-tag-list=$NAMETAG --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
--link-pois-to-ways --location-autofill=is_in,nearest
--drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
--license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
${pais}-DEM.args tmp/${ABR}5${FID}.txt

Both OSM and OSM+DEM maps can be downloaded from
https://alternativaslibres.org/en/downloads.php#Oceania

Any idea why this happens?


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Australia DEM.png (351K) Download Attachment
Australia OK.png (105K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Carlos,

not sure if I understand what the problem is. The two screenshots show completely different tile boundaries, so they were not created from the same splitter output.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Donnerstag, 17. Dezember 2020 23:44
An: Development list for mkgmap
Betreff: [mkgmap-dev] Tiles pruned in DEM map

I build several types of map (OSM, OSM+contour lines, maps for trucks
and OSM+DEM) for the same area, using same input files. I split
country.o5m with splitter and then use the same template.args output by
splitter for all maps, just changing mapname for the different types of
map. Given that, all resulting mapsets should have the same tiles, but
tiles in DEM map are smaller than in the other types. They share part of
the boundaries (usually two of them) with other types, but other
boundaries are moved in, reducing displayed tile size. See attached
screenshots. However, file size is the same (discounting *.DEM) for each
tile in *.gmap subfolders.

Command is quite similar for OSM and OSM+DEM maps:

java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
--precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
--area-name=$MAPA --family-name="OpenStreetMap $MAPA"
--family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
--overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
--name-tag-list=$NAMETAG --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
--report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
--check-roundabouts --check-roundabout-flares --license-file=license_OSM
--copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
--style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt

java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
--dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
--overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
--show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
--country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
--family-name="OSM $MAPA DEM" --family-id=5${FID}
--product-version=$VERSION --series-name="OSM-$MAPA-DEM"
--overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
--name-tag-list=$NAMETAG --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
--link-pois-to-ways --location-autofill=is_in,nearest
--drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
--license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
${pais}-DEM.args tmp/${ABR}5${FID}.txt

Both OSM and OSM+DEM maps can be downloaded from
https://alternativaslibres.org/en/downloads.php#Oceania

Any idea why this happens?

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Carlos Dávila-5
In theory, they are created from same splitter output, that's the problem.

El 18/12/20 a las 11:48, Gerd Petermann escribió:

> Hi Carlos,
>
> not sure if I understand what the problem is. The two screenshots show completely different tile boundaries, so they were not created from the same splitter output.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Donnerstag, 17. Dezember 2020 23:44
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>
> I build several types of map (OSM, OSM+contour lines, maps for trucks
> and OSM+DEM) for the same area, using same input files. I split
> country.o5m with splitter and then use the same template.args output by
> splitter for all maps, just changing mapname for the different types of
> map. Given that, all resulting mapsets should have the same tiles, but
> tiles in DEM map are smaller than in the other types. They share part of
> the boundaries (usually two of them) with other types, but other
> boundaries are moved in, reducing displayed tile size. See attached
> screenshots. However, file size is the same (discounting *.DEM) for each
> tile in *.gmap subfolders.
>
> Command is quite similar for OSM and OSM+DEM maps:
>
> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
> --precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
> --family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
> --overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
> --name-tag-list=$NAMETAG --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --check-roundabouts --check-roundabout-flares --license-file=license_OSM
> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>
> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
> --family-name="OSM $MAPA DEM" --family-id=5${FID}
> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
> --overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
> --name-tag-list=$NAMETAG --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
> --link-pois-to-ways --location-autofill=is_in,nearest
> --drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>
> Both OSM and OSM+DEM maps can be downloaded from
> https://alternativaslibres.org/en/downloads.php#Oceania
>
> Any idea why this happens?
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Carlos Dávila-5
In reply to this post by Gerd Petermann
The problem seems to be caused by overview DEM. If I remove
--overview-dem-dist option tile is complete in MapSource. The issue can
be reproduced with this tile
<https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
vierfinderpanoramas3

El 18/12/20 a las 11:48, Gerd Petermann escribió:

> Hi Carlos,
>
> not sure if I understand what the problem is. The two screenshots show completely different tile boundaries, so they were not created from the same splitter output.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Donnerstag, 17. Dezember 2020 23:44
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>
> I build several types of map (OSM, OSM+contour lines, maps for trucks
> and OSM+DEM) for the same area, using same input files. I split
> country.o5m with splitter and then use the same template.args output by
> splitter for all maps, just changing mapname for the different types of
> map. Given that, all resulting mapsets should have the same tiles, but
> tiles in DEM map are smaller than in the other types. They share part of
> the boundaries (usually two of them) with other types, but other
> boundaries are moved in, reducing displayed tile size. See attached
> screenshots. However, file size is the same (discounting *.DEM) for each
> tile in *.gmap subfolders.
>
> Command is quite similar for OSM and OSM+DEM maps:
>
> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
> --precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
> --family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
> --overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
> --name-tag-list=$NAMETAG --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --check-roundabouts --check-roundabout-flares --license-file=license_OSM
> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>
> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
> --family-name="OSM $MAPA DEM" --family-id=5${FID}
> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
> --overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
> --name-tag-list=$NAMETAG --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
> --link-pois-to-ways --location-autofill=is_in,nearest
> --drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>
> Both OSM and OSM+DEM maps can be downloaded from
> https://alternativaslibres.org/en/downloads.php#Oceania
>
> Any idea why this happens?
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Carlos,

It seems I still don't understand what the problem is or how to reproduce it.
I tried this command and the overview map looks OK:
java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --overview-dem-dist=128000 --show-profiles=1 --gmapi f:\dwnload\temp\51145001.o5m

If it is related to the DEM data I probably cannot help much. You may try a slightly different value for the --overview-dem-dist option or a modified polygon
given with the --dem-poly option.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Montag, 21. Dezember 2020 22:09
An: [hidden email]
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

The problem seems to be caused by overview DEM. If I remove
--overview-dem-dist option tile is complete in MapSource. The issue can
be reproduced with this tile
<https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
vierfinderpanoramas3

El 18/12/20 a las 11:48, Gerd Petermann escribió:

> Hi Carlos,
>
> not sure if I understand what the problem is. The two screenshots show completely different tile boundaries, so they were not created from the same splitter output.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Donnerstag, 17. Dezember 2020 23:44
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>
> I build several types of map (OSM, OSM+contour lines, maps for trucks
> and OSM+DEM) for the same area, using same input files. I split
> country.o5m with splitter and then use the same template.args output by
> splitter for all maps, just changing mapname for the different types of
> map. Given that, all resulting mapsets should have the same tiles, but
> tiles in DEM map are smaller than in the other types. They share part of
> the boundaries (usually two of them) with other types, but other
> boundaries are moved in, reducing displayed tile size. See attached
> screenshots. However, file size is the same (discounting *.DEM) for each
> tile in *.gmap subfolders.
>
> Command is quite similar for OSM and OSM+DEM maps:
>
> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
> --precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
> --family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
> --overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
> --name-tag-list=$NAMETAG --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --check-roundabouts --check-roundabout-flares --license-file=license_OSM
> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>
> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
> --family-name="OSM $MAPA DEM" --family-id=5${FID}
> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
> --overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
> --name-tag-list=$NAMETAG --process-destination --process-exits
> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
> --link-pois-to-ways --location-autofill=is_in,nearest
> --drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>
> Both OSM and OSM+DEM maps can be downloaded from
> https://alternativaslibres.org/en/downloads.php#Oceania
>
> Any idea why this happens?
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Carlos Dávila-5
I'm sorry, probably I didn't explain well enough.

Overview is always correct, the problem affects only tiles. As you saw
in the screenshots of my first mail, they are different in size, but
they are generated from the same input files, so they should have the
same size. If you zoom in to an area that should be included in a tile,
only overview map is shown, no detailed map. Even more, when you zoom
in, at the given point where detailed map appears, tile boundary jumps
to the correct size, but nothing but overview is displayed outside the
"pruned" tile.

You can download correct and wrong files from the links below and play
with them to get a better idea of the problem. They correspond to first
tile of Australia as shown in my screenshots.

https://alternativaslibres.org/tmp/11-error.zip

https://alternativaslibres.org/tmp/12-OK.zip

Error was generated with java -Xmx27G -ea
-Dlog.config=logging.properties -jar mkgmap.jar
--dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
opciones_comunes $CODEPAGE --gmapi --show-profiles=1 --bounds=bounds.zip
--precomp-sea=sea.zip --route --family-name="OSM $MAPA DEM"
--family-id=5${FID} --overview-mapname=${ABR}5${FID}
--overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
--polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
51145001.o5m tmp/${ABR}5${FID}.txt

OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
-jar mkgmap.jar
--dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
opciones_comunes $CODEPAGE --gmapi --show-profiles=1 --bounds=bounds.zip
--precomp-sea=sea.zip --route --family-name="OSM $MAPA DEM"
--family-id=5${FID} --overview-mapname=${ABR}5${FID}
--overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
--polygon-size-limits="24:12, 18:10, 16:0"
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
51145001.o5m tmp/${ABR}5${FID}.txt

Although removing --overview-dem-dist solved the issue in my first test
(see command below, correct output), after a lot of tests removing
options one by one from the original command, finally it seems the
problem is caused by option --order-by-decreasing-area (or the
combination of both).

java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
--dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
--show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
--country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
--family-name="OSM $MAPA DEM" --family-id=5${FID}
--product-version=$VERSION --series-name="OSM-$MAPA-DEM"
--overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
--name-tag-list=$NAMETAG --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
--link-pois-to-ways --location-autofill=is_in,nearest
--drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
--license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
51145001.o5m tmp/${ABR}5${FID}.txt

El 22/12/20 a las 10:15, Gerd Petermann escribió:

> Hi Carlos,
>
> It seems I still don't understand what the problem is or how to reproduce it.
> I tried this command and the overview map looks OK:
> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --overview-dem-dist=128000 --show-profiles=1 --gmapi f:\dwnload\temp\51145001.o5m
>
> If it is related to the DEM data I probably cannot help much. You may try a slightly different value for the --overview-dem-dist option or a modified polygon
> given with the --dem-poly option.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Montag, 21. Dezember 2020 22:09
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> The problem seems to be caused by overview DEM. If I remove
> --overview-dem-dist option tile is complete in MapSource. The issue can
> be reproduced with this tile
> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
> vierfinderpanoramas3
>
> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> not sure if I understand what the problem is. The two screenshots show completely different tile boundaries, so they were not created from the same splitter output.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>> An: Development list for mkgmap
>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>
>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>> and OSM+DEM) for the same area, using same input files. I split
>> country.o5m with splitter and then use the same template.args output by
>> splitter for all maps, just changing mapname for the different types of
>> map. Given that, all resulting mapsets should have the same tiles, but
>> tiles in DEM map are smaller than in the other types. They share part of
>> the boundaries (usually two of them) with other types, but other
>> boundaries are moved in, reducing displayed tile size. See attached
>> screenshots. However, file size is the same (discounting *.DEM) for each
>> tile in *.gmap subfolders.
>>
>> Command is quite similar for OSM and OSM+DEM maps:
>>
>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>> --precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>> --family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
>> --overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
>> --name-tag-list=$NAMETAG --process-destination --process-exits
>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>> --check-roundabouts --check-roundabout-flares --license-file=license_OSM
>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>
>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>> --overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
>> --name-tag-list=$NAMETAG --process-destination --process-exits
>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>> --link-pois-to-ways --location-autofill=is_in,nearest
>> --drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>
>> Both OSM and OSM+DEM maps can be downloaded from
>> https://alternativaslibres.org/en/downloads.php#Oceania
>>
>> Any idea why this happens?
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Carlos,

OK, I think I understand what problem you see. I used JaVaWa MapConverter to install the map in 11-error.zip and 12-OK.zip  played with it. With 11-error.zip only the data from the overview map is displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip shows more details (this is with MapSource, in Basecamp I see details in both maps).

I tried to analyse your files and I see three suspicious things so far:
1) The routing data seems to be wrong, NodCheck reports "Could not find node for road 38a61 nod2=124c8 " for both tiles. I don't see such an error with the default style.
2) The bad overview map contains a lot more 0x4a polygons. This is probably caused by the --order-by-decreasing-area, and I am not sure why this happens.
3) You use a special version of mkgmap, so please try also with the latest version.

My first guess was that the bad NOD file may cause this but the problem doesn't disappear when I remove the file 51145001.NOD, so this is probably not to blame. Same for the 51145001.DEM file.

I tried to reproduce the possible problem with the 0x4a tiles using the default style with this command
java -Xmx4G -jar map.jar --route --gmapi --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest f:\dwnload\temp\51145001.o5m
but my overview map contains the same number of 0x4a tiles  as your good map.

I cannot reproduce your DEM data because I don't know the polygon file (polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you create the test files again without --remove-ovm-work-files=true  .

Gerd


________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Dienstag, 22. Dezember 2020 18:05
An: [hidden email]
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

I'm sorry, probably I didn't explain well enough.

Overview is always correct, the problem affects only tiles. As you saw
in the screenshots of my first mail, they are different in size, but
they are generated from the same input files, so they should have the
same size. If you zoom in to an area that should be included in a tile,
only overview map is shown, no detailed map. Even more, when you zoom
in, at the given point where detailed map appears, tile boundary jumps
to the correct size, but nothing but overview is displayed outside the
"pruned" tile.

You can download correct and wrong files from the links below and play
with them to get a better idea of the problem. They correspond to first
tile of Australia as shown in my screenshots.

https://alternativaslibres.org/tmp/11-error.zip

https://alternativaslibres.org/tmp/12-OK.zip

Error was generated with java -Xmx27G -ea
-Dlog.config=logging.properties -jar mkgmap.jar
--dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
opciones_comunes $CODEPAGE --gmapi --show-profiles=1 --bounds=bounds.zip
--precomp-sea=sea.zip --route --family-name="OSM $MAPA DEM"
--family-id=5${FID} --overview-mapname=${ABR}5${FID}
--overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
--polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
51145001.o5m tmp/${ABR}5${FID}.txt

OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
-jar mkgmap.jar
--dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
opciones_comunes $CODEPAGE --gmapi --show-profiles=1 --bounds=bounds.zip
--precomp-sea=sea.zip --route --family-name="OSM $MAPA DEM"
--family-id=5${FID} --overview-mapname=${ABR}5${FID}
--overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
--polygon-size-limits="24:12, 18:10, 16:0"
--location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
51145001.o5m tmp/${ABR}5${FID}.txt

Although removing --overview-dem-dist solved the issue in my first test
(see command below, correct output), after a lot of tests removing
options one by one from the original command, finally it seems the
problem is caused by option --order-by-decreasing-area (or the
combination of both).

java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
--dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
--dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
--show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
--country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
--family-name="OSM $MAPA DEM" --family-id=5${FID}
--product-version=$VERSION --series-name="OSM-$MAPA-DEM"
--overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
--name-tag-list=$NAMETAG --process-destination --process-exits
--housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
--link-pois-to-ways --location-autofill=is_in,nearest
--drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
--license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
--road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
51145001.o5m tmp/${ABR}5${FID}.txt

El 22/12/20 a las 10:15, Gerd Petermann escribió:

> Hi Carlos,
>
> It seems I still don't understand what the problem is or how to reproduce it.
> I tried this command and the overview map looks OK:
> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --overview-dem-dist=128000 --show-profiles=1 --gmapi f:\dwnload\temp\51145001.o5m
>
> If it is related to the DEM data I probably cannot help much. You may try a slightly different value for the --overview-dem-dist option or a modified polygon
> given with the --dem-poly option.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Montag, 21. Dezember 2020 22:09
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> The problem seems to be caused by overview DEM. If I remove
> --overview-dem-dist option tile is complete in MapSource. The issue can
> be reproduced with this tile
> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
> vierfinderpanoramas3
>
> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> not sure if I understand what the problem is. The two screenshots show completely different tile boundaries, so they were not created from the same splitter output.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>> An: Development list for mkgmap
>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>
>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>> and OSM+DEM) for the same area, using same input files. I split
>> country.o5m with splitter and then use the same template.args output by
>> splitter for all maps, just changing mapname for the different types of
>> map. Given that, all resulting mapsets should have the same tiles, but
>> tiles in DEM map are smaller than in the other types. They share part of
>> the boundaries (usually two of them) with other types, but other
>> boundaries are moved in, reducing displayed tile size. See attached
>> screenshots. However, file size is the same (discounting *.DEM) for each
>> tile in *.gmap subfolders.
>>
>> Command is quite similar for OSM and OSM+DEM maps:
>>
>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>> --precomp-sea=sea.zip --route --country-name=$PAIS --country-abbr=${ABR}
>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>> --family-id=1${FID} --product-version=$VERSION --series-name="OSM-$MAPA"
>> --overview-mapname=${ABR}1${FID} --overview-mapnumber=${GRUPO}1${FID}000
>> --name-tag-list=$NAMETAG --process-destination --process-exits
>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>> --check-roundabouts --check-roundabout-flares --license-file=license_OSM
>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>
>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>> --overview-mapname=${ABR}5${FID} --overview-mapnumber=${GRUPO}5${FID}000
>> --name-tag-list=$NAMETAG --process-destination --process-exits
>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>> --link-pois-to-ways --location-autofill=is_in,nearest
>> --drive-on=detect,$DRIVEON --check-roundabouts --check-roundabout-flares
>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true -c
>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>
>> Both OSM and OSM+DEM maps can be downloaded from
>> https://alternativaslibres.org/en/downloads.php#Oceania
>>
>> Any idea why this happens?
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Carlos Dávila-5
In reply to this post by Carlos Dávila-5
I'm sorry for the late response and for breaking the thread, but I
didn't receive Gerd's last message, so I've had to copy it from mailing
list archive. Reply inline.

OK, I think I understand what problem you see. I used JaVaWa
MapConverter to install the map in 11-error.zip and 12-OK.zip played
with it. With 11-error.zip only the data from the overview map is
displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
shows more details (this is with MapSource, in Basecamp I see details in
both maps).

Yes, everything to the North of approx. S32.39125 is missing.

I tried to analyse your files and I see three suspicious things so far:

1) The routing data seems to be wrong, NodCheck reports "Could not find
node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
error with the default style.

How can I check that? If both tiles are affected, probably this is not
related with current issue, but other maps produced with the same style
will also be wrong regarding routing.

2) The bad overview map contains a lot more 0x4a polygons. This is
probably caused by the --order-by-decreasing-area, and I am not sure why
this happens.

Do you think the problem is in the overview map or may be in tile map?
If I open tile in MapEdit++ the number of non polygon elements is almost
the same in wrong and correct maps. For example, number of roads is
exactly the same. It seems as if something is masking part of the tile
in MapSource (also in BaseCamp in my case); elements are there, but you
can't see them.

3) You use a special version of mkgmap, so please try also with the
latest version.

With latest mkgmap and default style problem persists.

My first guess was that the bad NOD file may cause this but the problem
doesn't disappear when I remove the file 51145001.NOD, so this is
probably not to blame. Same for the 51145001.DEM file.

I tried to reproduce the possible problem with the 0x4a tiles using the
default style with this command java -Xmx4G -jar map.jar --route --gmapi
--bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
--reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
--order-by-decreasing-area --location-autofill=is_in,nearest
f:\dwnload\temp\51145001.o5m but my overview map contains the same
number of 0x4a tiles as your good map.

I cannot reproduce your DEM data because I don't know the polygon file
(polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
create the test files again without --remove-ovm-work-files=true.

You can download polygon file from here
<https://alternativaslibres.org/tmp/australia.poly> and ovm from here
<https://alternativaslibres.org/tmp/ovm_51145001.img>.

El 22/12/20 a las 18:05, Carlos Dávila escribió:

> I'm sorry, probably I didn't explain well enough.
>
> Overview is always correct, the problem affects only tiles. As you saw
> in the screenshots of my first mail, they are different in size, but
> they are generated from the same input files, so they should have the
> same size. If you zoom in to an area that should be included in a
> tile, only overview map is shown, no detailed map. Even more, when you
> zoom in, at the given point where detailed map appears, tile boundary
> jumps to the correct size, but nothing but overview is displayed
> outside the "pruned" tile.
>
> You can download correct and wrong files from the links below and play
> with them to get a better idea of the problem. They correspond to
> first tile of Australia as shown in my screenshots.
>
> https://alternativaslibres.org/tmp/11-error.zip
>
> https://alternativaslibres.org/tmp/12-OK.zip
>
> Error was generated with java -Xmx27G -ea
> -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
> -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
> --polygon-size-limits="24:12, 18:10, 16:0"
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> Although removing --overview-dem-dist solved the issue in my first
> test (see command below, correct output), after a lot of tests
> removing options one by one from the original command, finally it
> seems the problem is caused by option --order-by-decreasing-area (or
> the combination of both).
>
> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
> --family-name="OSM $MAPA DEM" --family-id=5${FID}
> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
> --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
> --process-destination --process-exits --housenumbers
> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --check-roundabouts --check-roundabout-flares
> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> It seems I still don't understand what the problem is or how to
>> reproduce it.
>> I tried this command and the overview map looks OK:
>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>> f:\dwnload\temp\51145001.o5m
>>
>> If it is related to the DEM data I probably cannot help much. You may
>> try a slightly different value for the --overview-dem-dist option or
>> a modified polygon
>> given with the --dem-poly option.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag
>> von Carlos Dávila <[hidden email]>
>> Gesendet: Montag, 21. Dezember 2020 22:09
>> An: [hidden email]
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> The problem seems to be caused by overview DEM. If I remove
>> --overview-dem-dist option tile is complete in MapSource. The issue can
>> be reproduced with this tile
>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>> vierfinderpanoramas3
>>
>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> not sure if I understand what the problem is. The two screenshots
>>> show completely different tile boundaries, so they were not created
>>> from the same splitter output.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>> von Carlos Dávila <[hidden email]>
>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>> An: Development list for mkgmap
>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>
>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>> and OSM+DEM) for the same area, using same input files. I split
>>> country.o5m with splitter and then use the same template.args output by
>>> splitter for all maps, just changing mapname for the different types of
>>> map. Given that, all resulting mapsets should have the same tiles, but
>>> tiles in DEM map are smaller than in the other types. They share
>>> part of
>>> the boundaries (usually two of them) with other types, but other
>>> boundaries are moved in, reducing displayed tile size. See attached
>>> screenshots. However, file size is the same (discounting *.DEM) for
>>> each
>>> tile in *.gmap subfolders.
>>>
>>> Command is quite similar for OSM and OSM+DEM maps:
>>>
>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>> --country-abbr=${ABR}
>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>> --family-id=1${FID} --product-version=$VERSION
>>> --series-name="OSM-$MAPA"
>>> --overview-mapname=${ABR}1${FID}
>>> --overview-mapnumber=${GRUPO}1${FID}000
>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>> --check-roundabouts --check-roundabout-flares
>>> --license-file=license_OSM
>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>
>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>> --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000
>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>> --check-roundabout-flares
>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio
>>> --remove-ovm-work-files=true -c
>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>
>>> Both OSM and OSM+DEM maps can be downloaded from
>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>
>>> Any idea why this happens?
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Carlos,

reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
It would be easier if you would post a link to your style.

I am now able to reproduce the display error using this command
java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m

and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Samstag, 26. Dezember 2020 19:12
An: [hidden email]
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

I'm sorry for the late response and for breaking the thread, but I
didn't receive Gerd's last message, so I've had to copy it from mailing
list archive. Reply inline.

OK, I think I understand what problem you see. I used JaVaWa
MapConverter to install the map in 11-error.zip and 12-OK.zip played
with it. With 11-error.zip only the data from the overview map is
displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
shows more details (this is with MapSource, in Basecamp I see details in
both maps).

Yes, everything to the North of approx. S32.39125 is missing.

I tried to analyse your files and I see three suspicious things so far:

1) The routing data seems to be wrong, NodCheck reports "Could not find
node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
error with the default style.

How can I check that? If both tiles are affected, probably this is not
related with current issue, but other maps produced with the same style
will also be wrong regarding routing.

2) The bad overview map contains a lot more 0x4a polygons. This is
probably caused by the --order-by-decreasing-area, and I am not sure why
this happens.

Do you think the problem is in the overview map or may be in tile map?
If I open tile in MapEdit++ the number of non polygon elements is almost
the same in wrong and correct maps. For example, number of roads is
exactly the same. It seems as if something is masking part of the tile
in MapSource (also in BaseCamp in my case); elements are there, but you
can't see them.

3) You use a special version of mkgmap, so please try also with the
latest version.

With latest mkgmap and default style problem persists.

My first guess was that the bad NOD file may cause this but the problem
doesn't disappear when I remove the file 51145001.NOD, so this is
probably not to blame. Same for the 51145001.DEM file.

I tried to reproduce the possible problem with the 0x4a tiles using the
default style with this command java -Xmx4G -jar map.jar --route --gmapi
--bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
--reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
--order-by-decreasing-area --location-autofill=is_in,nearest
f:\dwnload\temp\51145001.o5m but my overview map contains the same
number of 0x4a tiles as your good map.

I cannot reproduce your DEM data because I don't know the polygon file
(polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
create the test files again without --remove-ovm-work-files=true.

You can download polygon file from here
<https://alternativaslibres.org/tmp/australia.poly> and ovm from here
<https://alternativaslibres.org/tmp/ovm_51145001.img>.

El 22/12/20 a las 18:05, Carlos Dávila escribió:

> I'm sorry, probably I didn't explain well enough.
>
> Overview is always correct, the problem affects only tiles. As you saw
> in the screenshots of my first mail, they are different in size, but
> they are generated from the same input files, so they should have the
> same size. If you zoom in to an area that should be included in a
> tile, only overview map is shown, no detailed map. Even more, when you
> zoom in, at the given point where detailed map appears, tile boundary
> jumps to the correct size, but nothing but overview is displayed
> outside the "pruned" tile.
>
> You can download correct and wrong files from the links below and play
> with them to get a better idea of the problem. They correspond to
> first tile of Australia as shown in my screenshots.
>
> https://alternativaslibres.org/tmp/11-error.zip
>
> https://alternativaslibres.org/tmp/12-OK.zip
>
> Error was generated with java -Xmx27G -ea
> -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
> -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
> --polygon-size-limits="24:12, 18:10, 16:0"
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> Although removing --overview-dem-dist solved the issue in my first
> test (see command below, correct output), after a lot of tests
> removing options one by one from the original command, finally it
> seems the problem is caused by option --order-by-decreasing-area (or
> the combination of both).
>
> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
> --family-name="OSM $MAPA DEM" --family-id=5${FID}
> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
> --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
> --process-destination --process-exits --housenumbers
> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --check-roundabouts --check-roundabout-flares
> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> It seems I still don't understand what the problem is or how to
>> reproduce it.
>> I tried this command and the overview map looks OK:
>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>> f:\dwnload\temp\51145001.o5m
>>
>> If it is related to the DEM data I probably cannot help much. You may
>> try a slightly different value for the --overview-dem-dist option or
>> a modified polygon
>> given with the --dem-poly option.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag
>> von Carlos Dávila <[hidden email]>
>> Gesendet: Montag, 21. Dezember 2020 22:09
>> An: [hidden email]
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> The problem seems to be caused by overview DEM. If I remove
>> --overview-dem-dist option tile is complete in MapSource. The issue can
>> be reproduced with this tile
>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>> vierfinderpanoramas3
>>
>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> not sure if I understand what the problem is. The two screenshots
>>> show completely different tile boundaries, so they were not created
>>> from the same splitter output.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>> von Carlos Dávila <[hidden email]>
>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>> An: Development list for mkgmap
>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>
>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>> and OSM+DEM) for the same area, using same input files. I split
>>> country.o5m with splitter and then use the same template.args output by
>>> splitter for all maps, just changing mapname for the different types of
>>> map. Given that, all resulting mapsets should have the same tiles, but
>>> tiles in DEM map are smaller than in the other types. They share
>>> part of
>>> the boundaries (usually two of them) with other types, but other
>>> boundaries are moved in, reducing displayed tile size. See attached
>>> screenshots. However, file size is the same (discounting *.DEM) for
>>> each
>>> tile in *.gmap subfolders.
>>>
>>> Command is quite similar for OSM and OSM+DEM maps:
>>>
>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>> --country-abbr=${ABR}
>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>> --family-id=1${FID} --product-version=$VERSION
>>> --series-name="OSM-$MAPA"
>>> --overview-mapname=${ABR}1${FID}
>>> --overview-mapnumber=${GRUPO}1${FID}000
>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>> --check-roundabouts --check-roundabout-flares
>>> --license-file=license_OSM
>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>
>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>> --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000
>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>> --check-roundabout-flares
>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio
>>> --remove-ovm-work-files=true -c
>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>
>>> Both OSM and OSM+DEM maps can be downloaded from
>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>
>>> Any idea why this happens?
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Carlos,

I agree that --order-by-decreasing-area is to blame. This option seems to disturb the calculation of the overview map, and maybe the extreme size of the tile plays another role here.

I hope Ticker can help here, I think the option supresses the merging of shapes, and this is probably not a good idea for the 0x4a shapes.

As a work-around for you:
Add option --no-order-by-decreasing-area at the end(!) of the parameter list so that the overview map is created without this option. This seems to fix the problem on my machine.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Gerd Petermann <[hidden email]>
Gesendet: Sonntag, 27. Dezember 2020 09:36
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

Hi Carlos,

reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
It would be easier if you would post a link to your style.

I am now able to reproduce the display error using this command
java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m

and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Samstag, 26. Dezember 2020 19:12
An: [hidden email]
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

I'm sorry for the late response and for breaking the thread, but I
didn't receive Gerd's last message, so I've had to copy it from mailing
list archive. Reply inline.

OK, I think I understand what problem you see. I used JaVaWa
MapConverter to install the map in 11-error.zip and 12-OK.zip played
with it. With 11-error.zip only the data from the overview map is
displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
shows more details (this is with MapSource, in Basecamp I see details in
both maps).

Yes, everything to the North of approx. S32.39125 is missing.

I tried to analyse your files and I see three suspicious things so far:

1) The routing data seems to be wrong, NodCheck reports "Could not find
node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
error with the default style.

How can I check that? If both tiles are affected, probably this is not
related with current issue, but other maps produced with the same style
will also be wrong regarding routing.

2) The bad overview map contains a lot more 0x4a polygons. This is
probably caused by the --order-by-decreasing-area, and I am not sure why
this happens.

Do you think the problem is in the overview map or may be in tile map?
If I open tile in MapEdit++ the number of non polygon elements is almost
the same in wrong and correct maps. For example, number of roads is
exactly the same. It seems as if something is masking part of the tile
in MapSource (also in BaseCamp in my case); elements are there, but you
can't see them.

3) You use a special version of mkgmap, so please try also with the
latest version.

With latest mkgmap and default style problem persists.

My first guess was that the bad NOD file may cause this but the problem
doesn't disappear when I remove the file 51145001.NOD, so this is
probably not to blame. Same for the 51145001.DEM file.

I tried to reproduce the possible problem with the 0x4a tiles using the
default style with this command java -Xmx4G -jar map.jar --route --gmapi
--bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
--reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
--order-by-decreasing-area --location-autofill=is_in,nearest
f:\dwnload\temp\51145001.o5m but my overview map contains the same
number of 0x4a tiles as your good map.

I cannot reproduce your DEM data because I don't know the polygon file
(polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
create the test files again without --remove-ovm-work-files=true.

You can download polygon file from here
<https://alternativaslibres.org/tmp/australia.poly> and ovm from here
<https://alternativaslibres.org/tmp/ovm_51145001.img>.

El 22/12/20 a las 18:05, Carlos Dávila escribió:

> I'm sorry, probably I didn't explain well enough.
>
> Overview is always correct, the problem affects only tiles. As you saw
> in the screenshots of my first mail, they are different in size, but
> they are generated from the same input files, so they should have the
> same size. If you zoom in to an area that should be included in a
> tile, only overview map is shown, no detailed map. Even more, when you
> zoom in, at the given point where detailed map appears, tile boundary
> jumps to the correct size, but nothing but overview is displayed
> outside the "pruned" tile.
>
> You can download correct and wrong files from the links below and play
> with them to get a better idea of the problem. They correspond to
> first tile of Australia as shown in my screenshots.
>
> https://alternativaslibres.org/tmp/11-error.zip
>
> https://alternativaslibres.org/tmp/12-OK.zip
>
> Error was generated with java -Xmx27G -ea
> -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
> -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
> --polygon-size-limits="24:12, 18:10, 16:0"
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> Although removing --overview-dem-dist solved the issue in my first
> test (see command below, correct output), after a lot of tests
> removing options one by one from the original command, finally it
> seems the problem is caused by option --order-by-decreasing-area (or
> the combination of both).
>
> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
> --family-name="OSM $MAPA DEM" --family-id=5${FID}
> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
> --overview-mapname=${ABR}5${FID}
> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
> --process-destination --process-exits --housenumbers
> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
> --check-roundabouts --check-roundabout-flares
> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
> 51145001.o5m tmp/${ABR}5${FID}.txt
>
> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> It seems I still don't understand what the problem is or how to
>> reproduce it.
>> I tried this command and the overview map looks OK:
>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>> f:\dwnload\temp\51145001.o5m
>>
>> If it is related to the DEM data I probably cannot help much. You may
>> try a slightly different value for the --overview-dem-dist option or
>> a modified polygon
>> given with the --dem-poly option.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag
>> von Carlos Dávila <[hidden email]>
>> Gesendet: Montag, 21. Dezember 2020 22:09
>> An: [hidden email]
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> The problem seems to be caused by overview DEM. If I remove
>> --overview-dem-dist option tile is complete in MapSource. The issue can
>> be reproduced with this tile
>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>> vierfinderpanoramas3
>>
>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> not sure if I understand what the problem is. The two screenshots
>>> show completely different tile boundaries, so they were not created
>>> from the same splitter output.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>> von Carlos Dávila <[hidden email]>
>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>> An: Development list for mkgmap
>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>
>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>> and OSM+DEM) for the same area, using same input files. I split
>>> country.o5m with splitter and then use the same template.args output by
>>> splitter for all maps, just changing mapname for the different types of
>>> map. Given that, all resulting mapsets should have the same tiles, but
>>> tiles in DEM map are smaller than in the other types. They share
>>> part of
>>> the boundaries (usually two of them) with other types, but other
>>> boundaries are moved in, reducing displayed tile size. See attached
>>> screenshots. However, file size is the same (discounting *.DEM) for
>>> each
>>> tile in *.gmap subfolders.
>>>
>>> Command is quite similar for OSM and OSM+DEM maps:
>>>
>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>> --country-abbr=${ABR}
>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>> --family-id=1${FID} --product-version=$VERSION
>>> --series-name="OSM-$MAPA"
>>> --overview-mapname=${ABR}1${FID}
>>> --overview-mapnumber=${GRUPO}1${FID}000
>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>> --check-roundabouts --check-roundabout-flares
>>> --license-file=license_OSM
>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>
>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>> --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000
>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>> --check-roundabout-flares
>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio
>>> --remove-ovm-work-files=true -c
>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>
>>> Both OSM and OSM+DEM maps can be downloaded from
>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>
>>> Any idea why this happens?
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Ticker Berkin
Hi Gerd & Carlos

I've started looking at this, but can't do anything serious until
tomorrow.

Ticker

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Carlos Dávila-5
In reply to this post by Gerd Petermann
I had removed --order-by-decreasing-area from my command for DEM maps,
but --no-order-by-decreasing-area seems to fix the problem by now. By
the way, this option is not documented in options file and I can't find
it in the code. Is --no a general switch for all options or is it a
particular case for order-by-decreasing-area? I don't remember having
seen it before.

El 27/12/20 a las 10:36, Gerd Petermann escribió:

> Hi Carlos,
>
> I agree that --order-by-decreasing-area is to blame. This option seems to disturb the calculation of the overview map, and maybe the extreme size of the tile plays another role here.
>
> I hope Ticker can help here, I think the option supresses the merging of shapes, and this is probably not a good idea for the 0x4a shapes.
>
> As a work-around for you:
> Add option --no-order-by-decreasing-area at the end(!) of the parameter list so that the overview map is created without this option. This seems to fix the problem on my machine.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Gerd Petermann <[hidden email]>
> Gesendet: Sonntag, 27. Dezember 2020 09:36
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Carlos,
>
> reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
> java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
> It would be easier if you would post a link to your style.
>
> I am now able to reproduce the display error using this command
> java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m
>
> and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Samstag, 26. Dezember 2020 19:12
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> I'm sorry for the late response and for breaking the thread, but I
> didn't receive Gerd's last message, so I've had to copy it from mailing
> list archive. Reply inline.
>
> OK, I think I understand what problem you see. I used JaVaWa
> MapConverter to install the map in 11-error.zip and 12-OK.zip played
> with it. With 11-error.zip only the data from the overview map is
> displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
> shows more details (this is with MapSource, in Basecamp I see details in
> both maps).
>
> Yes, everything to the North of approx. S32.39125 is missing.
>
> I tried to analyse your files and I see three suspicious things so far:
>
> 1) The routing data seems to be wrong, NodCheck reports "Could not find
> node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
> error with the default style.
>
> How can I check that? If both tiles are affected, probably this is not
> related with current issue, but other maps produced with the same style
> will also be wrong regarding routing.
>
> 2) The bad overview map contains a lot more 0x4a polygons. This is
> probably caused by the --order-by-decreasing-area, and I am not sure why
> this happens.
>
> Do you think the problem is in the overview map or may be in tile map?
> If I open tile in MapEdit++ the number of non polygon elements is almost
> the same in wrong and correct maps. For example, number of roads is
> exactly the same. It seems as if something is masking part of the tile
> in MapSource (also in BaseCamp in my case); elements are there, but you
> can't see them.
>
> 3) You use a special version of mkgmap, so please try also with the
> latest version.
>
> With latest mkgmap and default style problem persists.
>
> My first guess was that the bad NOD file may cause this but the problem
> doesn't disappear when I remove the file 51145001.NOD, so this is
> probably not to blame. Same for the 51145001.DEM file.
>
> I tried to reproduce the possible problem with the 0x4a tiles using the
> default style with this command java -Xmx4G -jar map.jar --route --gmapi
> --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
> --order-by-decreasing-area --location-autofill=is_in,nearest
> f:\dwnload\temp\51145001.o5m but my overview map contains the same
> number of 0x4a tiles as your good map.
>
> I cannot reproduce your DEM data because I don't know the polygon file
> (polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
> create the test files again without --remove-ovm-work-files=true.
>
> You can download polygon file from here
> <https://alternativaslibres.org/tmp/australia.poly> and ovm from here
> <https://alternativaslibres.org/tmp/ovm_51145001.img>.
>
> El 22/12/20 a las 18:05, Carlos Dávila escribió:
>> I'm sorry, probably I didn't explain well enough.
>>
>> Overview is always correct, the problem affects only tiles. As you saw
>> in the screenshots of my first mail, they are different in size, but
>> they are generated from the same input files, so they should have the
>> same size. If you zoom in to an area that should be included in a
>> tile, only overview map is shown, no detailed map. Even more, when you
>> zoom in, at the given point where detailed map appears, tile boundary
>> jumps to the correct size, but nothing but overview is displayed
>> outside the "pruned" tile.
>>
>> You can download correct and wrong files from the links below and play
>> with them to get a better idea of the problem. They correspond to
>> first tile of Australia as shown in my screenshots.
>>
>> https://alternativaslibres.org/tmp/11-error.zip
>>
>> https://alternativaslibres.org/tmp/12-OK.zip
>>
>> Error was generated with java -Xmx27G -ea
>> -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
>> -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>> --polygon-size-limits="24:12, 18:10, 16:0"
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> Although removing --overview-dem-dist solved the issue in my first
>> test (see command below, correct output), after a lot of tests
>> removing options one by one from the original command, finally it
>> seems the problem is caused by option --order-by-decreasing-area (or
>> the combination of both).
>>
>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>> --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
>> --process-destination --process-exits --housenumbers
>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>> --check-roundabouts --check-roundabout-flares
>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> It seems I still don't understand what the problem is or how to
>>> reproduce it.
>>> I tried this command and the overview map looks OK:
>>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>>> f:\dwnload\temp\51145001.o5m
>>>
>>> If it is related to the DEM data I probably cannot help much. You may
>>> try a slightly different value for the --overview-dem-dist option or
>>> a modified polygon
>>> given with the --dem-poly option.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>> von Carlos Dávila <[hidden email]>
>>> Gesendet: Montag, 21. Dezember 2020 22:09
>>> An: [hidden email]
>>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>>
>>> The problem seems to be caused by overview DEM. If I remove
>>> --overview-dem-dist option tile is complete in MapSource. The issue can
>>> be reproduced with this tile
>>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>>> vierfinderpanoramas3
>>>
>>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> not sure if I understand what the problem is. The two screenshots
>>>> show completely different tile boundaries, so they were not created
>>>> from the same splitter output.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>>> von Carlos Dávila <[hidden email]>
>>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>>> An: Development list for mkgmap
>>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>>
>>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>>> and OSM+DEM) for the same area, using same input files. I split
>>>> country.o5m with splitter and then use the same template.args output by
>>>> splitter for all maps, just changing mapname for the different types of
>>>> map. Given that, all resulting mapsets should have the same tiles, but
>>>> tiles in DEM map are smaller than in the other types. They share
>>>> part of
>>>> the boundaries (usually two of them) with other types, but other
>>>> boundaries are moved in, reducing displayed tile size. See attached
>>>> screenshots. However, file size is the same (discounting *.DEM) for
>>>> each
>>>> tile in *.gmap subfolders.
>>>>
>>>> Command is quite similar for OSM and OSM+DEM maps:
>>>>
>>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>>> --country-abbr=${ABR}
>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>> --family-id=1${FID} --product-version=$VERSION
>>>> --series-name="OSM-$MAPA"
>>>> --overview-mapname=${ABR}1${FID}
>>>> --overview-mapnumber=${GRUPO}1${FID}000
>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>> --check-roundabouts --check-roundabout-flares
>>>> --license-file=license_OSM
>>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>>
>>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>>> --overview-mapname=${ABR}5${FID}
>>>> --overview-mapnumber=${GRUPO}5${FID}000
>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>>> --check-roundabout-flares
>>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>>> --road-name-config=${CONFIG} --style=mio
>>>> --remove-ovm-work-files=true -c
>>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>>
>>>> Both OSM and OSM+DEM maps can be downloaded from
>>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>>
>>>> Any idea why this happens?
>>>>
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> [hidden email]
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Carlos,

most of the options can be prefixed with no- since many years. Just to make sure: My suggestion was to keep --order-by-decreasing-area for the tiles and add --no-order-by-decreasing-area at the end for the combiners, esp. the overview map.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Sonntag, 27. Dezember 2020 19:07
An: [hidden email]
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

I had removed --order-by-decreasing-area from my command for DEM maps,
but --no-order-by-decreasing-area seems to fix the problem by now. By
the way, this option is not documented in options file and I can't find
it in the code. Is --no a general switch for all options or is it a
particular case for order-by-decreasing-area? I don't remember having
seen it before.

El 27/12/20 a las 10:36, Gerd Petermann escribió:

> Hi Carlos,
>
> I agree that --order-by-decreasing-area is to blame. This option seems to disturb the calculation of the overview map, and maybe the extreme size of the tile plays another role here.
>
> I hope Ticker can help here, I think the option supresses the merging of shapes, and this is probably not a good idea for the 0x4a shapes.
>
> As a work-around for you:
> Add option --no-order-by-decreasing-area at the end(!) of the parameter list so that the overview map is created without this option. This seems to fix the problem on my machine.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Gerd Petermann <[hidden email]>
> Gesendet: Sonntag, 27. Dezember 2020 09:36
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Carlos,
>
> reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
> java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
> It would be easier if you would post a link to your style.
>
> I am now able to reproduce the display error using this command
> java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m
>
> and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Samstag, 26. Dezember 2020 19:12
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> I'm sorry for the late response and for breaking the thread, but I
> didn't receive Gerd's last message, so I've had to copy it from mailing
> list archive. Reply inline.
>
> OK, I think I understand what problem you see. I used JaVaWa
> MapConverter to install the map in 11-error.zip and 12-OK.zip played
> with it. With 11-error.zip only the data from the overview map is
> displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
> shows more details (this is with MapSource, in Basecamp I see details in
> both maps).
>
> Yes, everything to the North of approx. S32.39125 is missing.
>
> I tried to analyse your files and I see three suspicious things so far:
>
> 1) The routing data seems to be wrong, NodCheck reports "Could not find
> node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
> error with the default style.
>
> How can I check that? If both tiles are affected, probably this is not
> related with current issue, but other maps produced with the same style
> will also be wrong regarding routing.
>
> 2) The bad overview map contains a lot more 0x4a polygons. This is
> probably caused by the --order-by-decreasing-area, and I am not sure why
> this happens.
>
> Do you think the problem is in the overview map or may be in tile map?
> If I open tile in MapEdit++ the number of non polygon elements is almost
> the same in wrong and correct maps. For example, number of roads is
> exactly the same. It seems as if something is masking part of the tile
> in MapSource (also in BaseCamp in my case); elements are there, but you
> can't see them.
>
> 3) You use a special version of mkgmap, so please try also with the
> latest version.
>
> With latest mkgmap and default style problem persists.
>
> My first guess was that the bad NOD file may cause this but the problem
> doesn't disappear when I remove the file 51145001.NOD, so this is
> probably not to blame. Same for the 51145001.DEM file.
>
> I tried to reproduce the possible problem with the 0x4a tiles using the
> default style with this command java -Xmx4G -jar map.jar --route --gmapi
> --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
> --order-by-decreasing-area --location-autofill=is_in,nearest
> f:\dwnload\temp\51145001.o5m but my overview map contains the same
> number of 0x4a tiles as your good map.
>
> I cannot reproduce your DEM data because I don't know the polygon file
> (polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
> create the test files again without --remove-ovm-work-files=true.
>
> You can download polygon file from here
> <https://alternativaslibres.org/tmp/australia.poly> and ovm from here
> <https://alternativaslibres.org/tmp/ovm_51145001.img>.
>
> El 22/12/20 a las 18:05, Carlos Dávila escribió:
>> I'm sorry, probably I didn't explain well enough.
>>
>> Overview is always correct, the problem affects only tiles. As you saw
>> in the screenshots of my first mail, they are different in size, but
>> they are generated from the same input files, so they should have the
>> same size. If you zoom in to an area that should be included in a
>> tile, only overview map is shown, no detailed map. Even more, when you
>> zoom in, at the given point where detailed map appears, tile boundary
>> jumps to the correct size, but nothing but overview is displayed
>> outside the "pruned" tile.
>>
>> You can download correct and wrong files from the links below and play
>> with them to get a better idea of the problem. They correspond to
>> first tile of Australia as shown in my screenshots.
>>
>> https://alternativaslibres.org/tmp/11-error.zip
>>
>> https://alternativaslibres.org/tmp/12-OK.zip
>>
>> Error was generated with java -Xmx27G -ea
>> -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
>> -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>> --polygon-size-limits="24:12, 18:10, 16:0"
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> Although removing --overview-dem-dist solved the issue in my first
>> test (see command below, correct output), after a lot of tests
>> removing options one by one from the original command, finally it
>> seems the problem is caused by option --order-by-decreasing-area (or
>> the combination of both).
>>
>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>> --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
>> --process-destination --process-exits --housenumbers
>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>> --check-roundabouts --check-roundabout-flares
>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> It seems I still don't understand what the problem is or how to
>>> reproduce it.
>>> I tried this command and the overview map looks OK:
>>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>>> f:\dwnload\temp\51145001.o5m
>>>
>>> If it is related to the DEM data I probably cannot help much. You may
>>> try a slightly different value for the --overview-dem-dist option or
>>> a modified polygon
>>> given with the --dem-poly option.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>> von Carlos Dávila <[hidden email]>
>>> Gesendet: Montag, 21. Dezember 2020 22:09
>>> An: [hidden email]
>>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>>
>>> The problem seems to be caused by overview DEM. If I remove
>>> --overview-dem-dist option tile is complete in MapSource. The issue can
>>> be reproduced with this tile
>>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>>> vierfinderpanoramas3
>>>
>>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> not sure if I understand what the problem is. The two screenshots
>>>> show completely different tile boundaries, so they were not created
>>>> from the same splitter output.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>>> von Carlos Dávila <[hidden email]>
>>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>>> An: Development list for mkgmap
>>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>>
>>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>>> and OSM+DEM) for the same area, using same input files. I split
>>>> country.o5m with splitter and then use the same template.args output by
>>>> splitter for all maps, just changing mapname for the different types of
>>>> map. Given that, all resulting mapsets should have the same tiles, but
>>>> tiles in DEM map are smaller than in the other types. They share
>>>> part of
>>>> the boundaries (usually two of them) with other types, but other
>>>> boundaries are moved in, reducing displayed tile size. See attached
>>>> screenshots. However, file size is the same (discounting *.DEM) for
>>>> each
>>>> tile in *.gmap subfolders.
>>>>
>>>> Command is quite similar for OSM and OSM+DEM maps:
>>>>
>>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>>> --country-abbr=${ABR}
>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>> --family-id=1${FID} --product-version=$VERSION
>>>> --series-name="OSM-$MAPA"
>>>> --overview-mapname=${ABR}1${FID}
>>>> --overview-mapnumber=${GRUPO}1${FID}000
>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>> --check-roundabouts --check-roundabout-flares
>>>> --license-file=license_OSM
>>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>>
>>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>>> --overview-mapname=${ABR}5${FID}
>>>> --overview-mapnumber=${GRUPO}5${FID}000
>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>>> --check-roundabout-flares
>>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>>> --road-name-config=${CONFIG} --style=mio
>>>> --remove-ovm-work-files=true -c
>>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>>
>>>> Both OSM and OSM+DEM maps can be downloaded from
>>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>>
>>>> Any idea why this happens?
>>>>
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> [hidden email]
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Carlos Dávila-5

El 27/12/20 a las 19:13, Gerd Petermann escribió:
> Hi Carlos,
>
> most of the options can be prefixed with no- since many years.
I didn't remember that
> Just to make sure: My suggestion was to keep --order-by-decreasing-area for the tiles and add --no-order-by-decreasing-area at the end for the combiners, esp. the overview map.
I had understood. I built Australia map that way and worked fine, thanks.

>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Sonntag, 27. Dezember 2020 19:07
> An: [hidden email]
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> I had removed --order-by-decreasing-area from my command for DEM maps,
> but --no-order-by-decreasing-area seems to fix the problem by now. By
> the way, this option is not documented in options file and I can't find
> it in the code. Is --no a general switch for all options or is it a
> particular case for order-by-decreasing-area? I don't remember having
> seen it before.
>
> El 27/12/20 a las 10:36, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I agree that --order-by-decreasing-area is to blame. This option seems to disturb the calculation of the overview map, and maybe the extreme size of the tile plays another role here.
>>
>> I hope Ticker can help here, I think the option supresses the merging of shapes, and this is probably not a good idea for the 0x4a shapes.
>>
>> As a work-around for you:
>> Add option --no-order-by-decreasing-area at the end(!) of the parameter list so that the overview map is created without this option. This seems to fix the problem on my machine.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag von Gerd Petermann <[hidden email]>
>> Gesendet: Sonntag, 27. Dezember 2020 09:36
>> An: Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> Hi Carlos,
>>
>> reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
>> java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
>> It would be easier if you would post a link to your style.
>>
>> I am now able to reproduce the display error using this command
>> java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m
>>
>> and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
>> Gesendet: Samstag, 26. Dezember 2020 19:12
>> An: [hidden email]
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> I'm sorry for the late response and for breaking the thread, but I
>> didn't receive Gerd's last message, so I've had to copy it from mailing
>> list archive. Reply inline.
>>
>> OK, I think I understand what problem you see. I used JaVaWa
>> MapConverter to install the map in 11-error.zip and 12-OK.zip played
>> with it. With 11-error.zip only the data from the overview map is
>> displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
>> shows more details (this is with MapSource, in Basecamp I see details in
>> both maps).
>>
>> Yes, everything to the North of approx. S32.39125 is missing.
>>
>> I tried to analyse your files and I see three suspicious things so far:
>>
>> 1) The routing data seems to be wrong, NodCheck reports "Could not find
>> node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
>> error with the default style.
>>
>> How can I check that? If both tiles are affected, probably this is not
>> related with current issue, but other maps produced with the same style
>> will also be wrong regarding routing.
>>
>> 2) The bad overview map contains a lot more 0x4a polygons. This is
>> probably caused by the --order-by-decreasing-area, and I am not sure why
>> this happens.
>>
>> Do you think the problem is in the overview map or may be in tile map?
>> If I open tile in MapEdit++ the number of non polygon elements is almost
>> the same in wrong and correct maps. For example, number of roads is
>> exactly the same. It seems as if something is masking part of the tile
>> in MapSource (also in BaseCamp in my case); elements are there, but you
>> can't see them.
>>
>> 3) You use a special version of mkgmap, so please try also with the
>> latest version.
>>
>> With latest mkgmap and default style problem persists.
>>
>> My first guess was that the bad NOD file may cause this but the problem
>> doesn't disappear when I remove the file 51145001.NOD, so this is
>> probably not to blame. Same for the 51145001.DEM file.
>>
>> I tried to reproduce the possible problem with the 0x4a tiles using the
>> default style with this command java -Xmx4G -jar map.jar --route --gmapi
>> --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>> --order-by-decreasing-area --location-autofill=is_in,nearest
>> f:\dwnload\temp\51145001.o5m but my overview map contains the same
>> number of 0x4a tiles as your good map.
>>
>> I cannot reproduce your DEM data because I don't know the polygon file
>> (polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
>> create the test files again without --remove-ovm-work-files=true.
>>
>> You can download polygon file from here
>> <https://alternativaslibres.org/tmp/australia.poly> and ovm from here
>> <https://alternativaslibres.org/tmp/ovm_51145001.img>.
>>
>> El 22/12/20 a las 18:05, Carlos Dávila escribió:
>>> I'm sorry, probably I didn't explain well enough.
>>>
>>> Overview is always correct, the problem affects only tiles. As you saw
>>> in the screenshots of my first mail, they are different in size, but
>>> they are generated from the same input files, so they should have the
>>> same size. If you zoom in to an area that should be included in a
>>> tile, only overview map is shown, no detailed map. Even more, when you
>>> zoom in, at the given point where detailed map appears, tile boundary
>>> jumps to the correct size, but nothing but overview is displayed
>>> outside the "pruned" tile.
>>>
>>> You can download correct and wrong files from the links below and play
>>> with them to get a better idea of the problem. They correspond to
>>> first tile of Australia as shown in my screenshots.
>>>
>>> https://alternativaslibres.org/tmp/11-error.zip
>>>
>>> https://alternativaslibres.org/tmp/12-OK.zip
>>>
>>> Error was generated with java -Xmx27G -ea
>>> -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>>> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>>
>>> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
>>> -jar mkgmap.jar
>>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>>> --polygon-size-limits="24:12, 18:10, 16:0"
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>>
>>> Although removing --overview-dem-dist solved the issue in my first
>>> test (see command below, correct output), after a lot of tests
>>> removing options one by one from the original command, finally it
>>> seems the problem is caused by option --order-by-decreasing-area (or
>>> the combination of both).
>>>
>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>> --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
>>> --process-destination --process-exits --housenumbers
>>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>>> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>> --check-roundabouts --check-roundabout-flares
>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>>
>>> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> It seems I still don't understand what the problem is or how to
>>>> reproduce it.
>>>> I tried this command and the overview map looks OK:
>>>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>>>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>>>> f:\dwnload\temp\51145001.o5m
>>>>
>>>> If it is related to the DEM data I probably cannot help much. You may
>>>> try a slightly different value for the --overview-dem-dist option or
>>>> a modified polygon
>>>> given with the --dem-poly option.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>>> von Carlos Dávila <[hidden email]>
>>>> Gesendet: Montag, 21. Dezember 2020 22:09
>>>> An: [hidden email]
>>>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>>>
>>>> The problem seems to be caused by overview DEM. If I remove
>>>> --overview-dem-dist option tile is complete in MapSource. The issue can
>>>> be reproduced with this tile
>>>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>>>> vierfinderpanoramas3
>>>>
>>>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>>>> Hi Carlos,
>>>>>
>>>>> not sure if I understand what the problem is. The two screenshots
>>>>> show completely different tile boundaries, so they were not created
>>>>> from the same splitter output.
>>>>>
>>>>> Gerd
>>>>>
>>>>> ________________________________________
>>>>> Von: mkgmap-dev <[hidden email]> im Auftrag
>>>>> von Carlos Dávila <[hidden email]>
>>>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>>>> An: Development list for mkgmap
>>>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>>>
>>>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>>>> and OSM+DEM) for the same area, using same input files. I split
>>>>> country.o5m with splitter and then use the same template.args output by
>>>>> splitter for all maps, just changing mapname for the different types of
>>>>> map. Given that, all resulting mapsets should have the same tiles, but
>>>>> tiles in DEM map are smaller than in the other types. They share
>>>>> part of
>>>>> the boundaries (usually two of them) with other types, but other
>>>>> boundaries are moved in, reducing displayed tile size. See attached
>>>>> screenshots. However, file size is the same (discounting *.DEM) for
>>>>> each
>>>>> tile in *.gmap subfolders.
>>>>>
>>>>> Command is quite similar for OSM and OSM+DEM maps:
>>>>>
>>>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>>>> --country-abbr=${ABR}
>>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>>> --family-id=1${FID} --product-version=$VERSION
>>>>> --series-name="OSM-$MAPA"
>>>>> --overview-mapname=${ABR}1${FID}
>>>>> --overview-mapnumber=${GRUPO}1${FID}000
>>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>>> --check-roundabouts --check-roundabout-flares
>>>>> --license-file=license_OSM
>>>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>>>
>>>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>>>> --overview-mapname=${ABR}5${FID}
>>>>> --overview-mapnumber=${GRUPO}5${FID}000
>>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>>>> --check-roundabout-flares
>>>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>>>> --road-name-config=${CONFIG} --style=mio
>>>>> --remove-ovm-work-files=true -c
>>>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>>>
>>>>> Both OSM and OSM+DEM maps can be downloaded from
>>>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>>>
>>>>> Any idea why this happens?
>>>>>
>>>>> _______________________________________________
>>>>> mkgmap-dev mailing list
>>>>> [hidden email]
>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> [hidden email]
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> [hidden email]
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> [hidden email]
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Ticker Berkin
Hi Gerd & Carlos

Reading and trying to understand the code, I'm finding a few strange
things with the Overview map generation, DEM, 0x4a etc

The most significant is that the MapBuilder invocation for the combined
overview map normally runs without any options being passed to it. Only
if --overview-dem-dist is supplied are all the other options (including
--order-by-decreasing-area) passed in. I'm not sure if would be a good
idea to supply all every time or just be more selective and filter out
all but the necessary DEM options.

I'm still investigating the overview map levels. The ovm_ files are
produced with all the overview-levels as specified by options. When
this is read back by the overview combiner, a 0x4a polygon is added
covering each ovm_ tile, but it looks like it is at all levels, so, for
a tile with a large area or lots of detail is very likely to be split
(if --order-by) or shunted around into another subdivision and multiple
copies might exist.

My understanding of the purpose of the 0x4a is that, in the overview
map, there should be exactly 1 per detailed tile. It would be sensible
to set its maxResolution so it only occurs at one level.

After the 0x4a polygon has been added, a couple of bits of code scan
for them in all the overview polygons. It might be possible to improve
this, given they have just been added in the same processing phase.

If --order-by-decreasing-area is used, the overview map combiner
shouldn't attempt to respect it because it doesn't have the full size
information of polygons that cross a tile boundary. Rather it should
respect the polygon ordering in each ovm_ tile. I'm not sure yet this
is feasible; Maybe something like the equivalent of --preserve-element
order for this phase and look at all the logic paths of polygons to
stop any other order changes due to merging etc.

Ticker


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Ticker,

thanks for the hints. I agree that the code in OverviewBuilder is not very clear. I'll have a closer look tomorrow, but at least we should remove the -order-by-decreasing-area when copying the options. Or maybe I change the code so that only the hgt/dem options are copied. I guess I was too lazy there.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Dienstag, 29. Dezember 2020 10:50
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

Hi Gerd & Carlos

Reading and trying to understand the code, I'm finding a few strange
things with the Overview map generation, DEM, 0x4a etc

The most significant is that the MapBuilder invocation for the combined
overview map normally runs without any options being passed to it. Only
if --overview-dem-dist is supplied are all the other options (including
--order-by-decreasing-area) passed in. I'm not sure if would be a good
idea to supply all every time or just be more selective and filter out
all but the necessary DEM options.

I'm still investigating the overview map levels. The ovm_ files are
produced with all the overview-levels as specified by options. When
this is read back by the overview combiner, a 0x4a polygon is added
covering each ovm_ tile, but it looks like it is at all levels, so, for
a tile with a large area or lots of detail is very likely to be split
(if --order-by) or shunted around into another subdivision and multiple
copies might exist.

My understanding of the purpose of the 0x4a is that, in the overview
map, there should be exactly 1 per detailed tile. It would be sensible
to set its maxResolution so it only occurs at one level.

After the 0x4a polygon has been added, a couple of bits of code scan
for them in all the overview polygons. It might be possible to improve
this, given they have just been added in the same processing phase.

If --order-by-decreasing-area is used, the overview map combiner
shouldn't attempt to respect it because it doesn't have the full size
information of polygons that cross a tile boundary. Rather it should
respect the polygon ordering in each ovm_ tile. I'm not sure yet this
is feasible; Maybe something like the equivalent of --preserve-element
order for this phase and look at all the logic paths of polygons to
stop any other order changes due to merging etc.

Ticker


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Ticker Berkin
Hi Gerd

I'm going to experiment with the combined overview map and which
options cause problems. Also look at the effect of 0x4a polygons at
just 1 level.

I also notices that the combined overview (osmmap.img) is a fraction of
the size (~1%) of the sum of the ovm_ images that are used to build it.

Ticker

On Tue, 2020-12-29 at 15:52 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> thanks for the hints. I agree that the code in OverviewBuilder is not
> very clear. I'll have a closer look tomorrow, but at least we should
> remove the -order-by-decreasing-area when copying the options. Or
> maybe I change the code so that only the hgt/dem options are copied.
> I guess I was too lazy there.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Dienstag, 29. Dezember 2020 10:50
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Gerd & Carlos
>
> Reading and trying to understand the code, I'm finding a few strange
> things with the Overview map generation, DEM, 0x4a etc
>
> The most significant is that the MapBuilder invocation for the
> combined
> overview map normally runs without any options being passed to it.
> Only
> if --overview-dem-dist is supplied are all the other options
> (including
> --order-by-decreasing-area) passed in. I'm not sure if would be a
> good
> idea to supply all every time or just be more selective and filter
> out
> all but the necessary DEM options.
>
> I'm still investigating the overview map levels. The ovm_ files are
> produced with all the overview-levels as specified by options. When
> this is read back by the overview combiner, a 0x4a polygon is added
> covering each ovm_ tile, but it looks like it is at all levels, so,
> for
> a tile with a large area or lots of detail is very likely to be split
> (if --order-by) or shunted around into another subdivision and
> multiple
> copies might exist.
>
> My understanding of the purpose of the 0x4a is that, in the overview
> map, there should be exactly 1 per detailed tile. It would be
> sensible
> to set its maxResolution so it only occurs at one level.
>
> After the 0x4a polygon has been added, a couple of bits of code scan
> for them in all the overview polygons. It might be possible to
> improve
> this, given they have just been added in the same processing phase.
>
> If --order-by-decreasing-area is used, the overview map combiner
> shouldn't attempt to respect it because it doesn't have the full size
> information of polygons that cross a tile boundary. Rather it should
> respect the polygon ordering in each ovm_ tile. I'm not sure yet this
> is feasible; Maybe something like the equivalent of --preserve
> -element
> order for this phase and look at all the logic paths of polygons to
> stop any other order changes due to merging etc.
>
> Ticker
>
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Ticker,

yes, this code needs review, thanks for this. For now I've just disabled the --order-by-decreasing-area option for the overview map in r4590

I am not sure if it would be better to always pass the args to MapBuilder or only those for DEM. I'd prefer to always pass them but maybe there are other side effects
Reg. size 1%: My understanding was that the merging of shapes is responsible for all of this, but I might be wrong.

I am working on a routing issue that I found while looking at Carlos' problem. It only happens with --x-no-force-end-nodes-routing-nodes.

Gerd


________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Mittwoch, 30. Dezember 2020 09:50
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

Hi Gerd

I'm going to experiment with the combined overview map and which
options cause problems. Also look at the effect of 0x4a polygons at
just 1 level.

I also notices that the combined overview (osmmap.img) is a fraction of
the size (~1%) of the sum of the ovm_ images that are used to build it.

Ticker

On Tue, 2020-12-29 at 15:52 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> thanks for the hints. I agree that the code in OverviewBuilder is not
> very clear. I'll have a closer look tomorrow, but at least we should
> remove the -order-by-decreasing-area when copying the options. Or
> maybe I change the code so that only the hgt/dem options are copied.
> I guess I was too lazy there.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Dienstag, 29. Dezember 2020 10:50
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Gerd & Carlos
>
> Reading and trying to understand the code, I'm finding a few strange
> things with the Overview map generation, DEM, 0x4a etc
>
> The most significant is that the MapBuilder invocation for the
> combined
> overview map normally runs without any options being passed to it.
> Only
> if --overview-dem-dist is supplied are all the other options
> (including
> --order-by-decreasing-area) passed in. I'm not sure if would be a
> good
> idea to supply all every time or just be more selective and filter
> out
> all but the necessary DEM options.
>
> I'm still investigating the overview map levels. The ovm_ files are
> produced with all the overview-levels as specified by options. When
> this is read back by the overview combiner, a 0x4a polygon is added
> covering each ovm_ tile, but it looks like it is at all levels, so,
> for
> a tile with a large area or lots of detail is very likely to be split
> (if --order-by) or shunted around into another subdivision and
> multiple
> copies might exist.
>
> My understanding of the purpose of the 0x4a is that, in the overview
> map, there should be exactly 1 per detailed tile. It would be
> sensible
> to set its maxResolution so it only occurs at one level.
>
> After the 0x4a polygon has been added, a couple of bits of code scan
> for them in all the overview polygons. It might be possible to
> improve
> this, given they have just been added in the same processing phase.
>
> If --order-by-decreasing-area is used, the overview map combiner
> shouldn't attempt to respect it because it doesn't have the full size
> information of polygons that cross a tile boundary. Rather it should
> respect the polygon ordering in each ovm_ tile. I'm not sure yet this
> is feasible; Maybe something like the equivalent of --preserve
> -element
> order for this phase and look at all the logic paths of polygons to
> stop any other order changes due to merging etc.
>
> Ticker
>
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Ticker Berkin
Hi Gerd

I found another unexpected case: giving the --dem... option caused the
overview map to have NET and NOD sections

To stops surprises, we should either pass in all the args and let the
receiver of them decide what to do with them or pass in only the
presumed significant options.

I think it much better to pass them all in; the logic to decide is then
all in the place where it is significant. I've made some changes to
this effect - attached.

In the case of --order-by-decreasing area, the final overview map needs
to know that this is wanted so it can keep the polygons in the same
order in each detail tile.

The large ovm_ size is because it has a LBL section containing all the
names for all levels. Unused ones don't make it into the final overview
so I don't think this is worth bothering with.

I've removed the scans for 0x4a polygons and do the equivalent when
they are inserted per detail tile. Also I've also removed some
misleading comments

Ticker

On Wed, 2020-12-30 at 11:21 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> yes, this code needs review, thanks for this. For now I've just
> disabled the --order-by-decreasing-area option for the overview map
> in r4590
>
> I am not sure if it would be better to always pass the args to
> MapBuilder or only those for DEM. I'd prefer to always pass them but
> maybe there are other side effects
> Reg. size 1%: My understanding was that the merging of shapes is
> responsible for all of this, but I might be wrong.
>
> I am working on a routing issue that I found while looking at Carlos'
> problem. It only happens with --x-no-force-end-nodes-routing-nodes.
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 30. Dezember 2020 09:50
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Gerd
>
> I'm going to experiment with the combined overview map and which
> options cause problems. Also look at the effect of 0x4a polygons at
> just 1 level.
>
> I also notices that the combined overview (osmmap.img) is a fraction
> of
> the size (~1%) of the sum of the ovm_ images that are used to build
> it.
>
> Ticker
>
> On Tue, 2020-12-29 at 15:52 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > thanks for the hints. I agree that the code in OverviewBuilder is
> > not
> > very clear. I'll have a closer look tomorrow, but at least we
> > should
> > remove the -order-by-decreasing-area when copying the options. Or
> > maybe I change the code so that only the hgt/dem options are
> > copied.
> > I guess I was too lazy there.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Dienstag, 29. Dezember 2020 10:50
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
> >
> > Hi Gerd & Carlos
> >
> > Reading and trying to understand the code, I'm finding a few
> > strange
> > things with the Overview map generation, DEM, 0x4a etc
> >
> > The most significant is that the MapBuilder invocation for the
> > combined
> > overview map normally runs without any options being passed to it.
> > Only
> > if --overview-dem-dist is supplied are all the other options
> > (including
> > --order-by-decreasing-area) passed in. I'm not sure if would be a
> > good
> > idea to supply all every time or just be more selective and filter
> > out
> > all but the necessary DEM options.
> >
> > I'm still investigating the overview map levels. The ovm_ files are
> > produced with all the overview-levels as specified by options. When
> > this is read back by the overview combiner, a 0x4a polygon is added
> > covering each ovm_ tile, but it looks like it is at all levels, so,
> > for
> > a tile with a large area or lots of detail is very likely to be
> > split
> > (if --order-by) or shunted around into another subdivision and
> > multiple
> > copies might exist.
> >
> > My understanding of the purpose of the 0x4a is that, in the
> > overview
> > map, there should be exactly 1 per detailed tile. It would be
> > sensible
> > to set its maxResolution so it only occurs at one level.
> >
> > After the 0x4a polygon has been added, a couple of bits of code
> > scan
> > for them in all the overview polygons. It might be possible to
> > improve
> > this, given they have just been added in the same processing phase.
> >
> > If --order-by-decreasing-area is used, the overview map combiner
> > shouldn't attempt to respect it because it doesn't have the full
> > size
> > information of polygons that cross a tile boundary. Rather it
> > should
> > respect the polygon ordering in each ovm_ tile. I'm not sure yet
> > this
> > is feasible; Maybe something like the equivalent of --preserve
> > -element
> > order for this phase and look at all the logic paths of polygons to
> > stop any other order changes due to merging etc.
> >
> > Ticker
> >
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

overview.patch (16K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Tiles pruned in DEM map

Gerd Petermann
Hi Ticker,

thanks for the patch. I'll have a closer look during the next days. I don't yet understand why shapes aren't always merged.
What is the impact on the size of the overview map? What happens for those users who create the overview map in an extra step that doesn't have the --order-by-decreasing-area option? What happens when it's the other way around, no --order-by-decreasing-area option for the tiles but --order-by-decreasing-area for the overview map?

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Ticker Berkin <[hidden email]>
Gesendet: Montag, 4. Januar 2021 19:16
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map

Hi Gerd

I found another unexpected case: giving the --dem... option caused the
overview map to have NET and NOD sections

To stops surprises, we should either pass in all the args and let the
receiver of them decide what to do with them or pass in only the
presumed significant options.

I think it much better to pass them all in; the logic to decide is then
all in the place where it is significant. I've made some changes to
this effect - attached.

In the case of --order-by-decreasing area, the final overview map needs
to know that this is wanted so it can keep the polygons in the same
order in each detail tile.

The large ovm_ size is because it has a LBL section containing all the
names for all levels. Unused ones don't make it into the final overview
so I don't think this is worth bothering with.

I've removed the scans for 0x4a polygons and do the equivalent when
they are inserted per detail tile. Also I've also removed some
misleading comments

Ticker

On Wed, 2020-12-30 at 11:21 +0000, Gerd Petermann wrote:

> Hi Ticker,
>
> yes, this code needs review, thanks for this. For now I've just
> disabled the --order-by-decreasing-area option for the overview map
> in r4590
>
> I am not sure if it would be better to always pass the args to
> MapBuilder or only those for DEM. I'd prefer to always pass them but
> maybe there are other side effects
> Reg. size 1%: My understanding was that the merging of shapes is
> responsible for all of this, but I might be wrong.
>
> I am working on a routing issue that I found while looking at Carlos'
> problem. It only happens with --x-no-force-end-nodes-routing-nodes.
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag
> von Ticker Berkin <[hidden email]>
> Gesendet: Mittwoch, 30. Dezember 2020 09:50
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Gerd
>
> I'm going to experiment with the combined overview map and which
> options cause problems. Also look at the effect of 0x4a polygons at
> just 1 level.
>
> I also notices that the combined overview (osmmap.img) is a fraction
> of
> the size (~1%) of the sum of the ovm_ images that are used to build
> it.
>
> Ticker
>
> On Tue, 2020-12-29 at 15:52 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > thanks for the hints. I agree that the code in OverviewBuilder is
> > not
> > very clear. I'll have a closer look tomorrow, but at least we
> > should
> > remove the -order-by-decreasing-area when copying the options. Or
> > maybe I change the code so that only the hgt/dem options are
> > copied.
> > I guess I was too lazy there.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[hidden email]> im Auftrag
> > von Ticker Berkin <[hidden email]>
> > Gesendet: Dienstag, 29. Dezember 2020 10:50
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
> >
> > Hi Gerd & Carlos
> >
> > Reading and trying to understand the code, I'm finding a few
> > strange
> > things with the Overview map generation, DEM, 0x4a etc
> >
> > The most significant is that the MapBuilder invocation for the
> > combined
> > overview map normally runs without any options being passed to it.
> > Only
> > if --overview-dem-dist is supplied are all the other options
> > (including
> > --order-by-decreasing-area) passed in. I'm not sure if would be a
> > good
> > idea to supply all every time or just be more selective and filter
> > out
> > all but the necessary DEM options.
> >
> > I'm still investigating the overview map levels. The ovm_ files are
> > produced with all the overview-levels as specified by options. When
> > this is read back by the overview combiner, a 0x4a polygon is added
> > covering each ovm_ tile, but it looks like it is at all levels, so,
> > for
> > a tile with a large area or lots of detail is very likely to be
> > split
> > (if --order-by) or shunted around into another subdivision and
> > multiple
> > copies might exist.
> >
> > My understanding of the purpose of the 0x4a is that, in the
> > overview
> > map, there should be exactly 1 per detailed tile. It would be
> > sensible
> > to set its maxResolution so it only occurs at one level.
> >
> > After the 0x4a polygon has been added, a couple of bits of code
> > scan
> > for them in all the overview polygons. It might be possible to
> > improve
> > this, given they have just been added in the same processing phase.
> >
> > If --order-by-decreasing-area is used, the overview map combiner
> > shouldn't attempt to respect it because it doesn't have the full
> > size
> > information of polygons that cross a tile boundary. Rather it
> > should
> > respect the polygon ordering in each ovm_ tile. I'm not sure yet
> > this
> > is feasible; Maybe something like the equivalent of --preserve
> > -element
> > order for this phase and look at all the logic paths of polygons to
> > stop any other order changes due to merging etc.
> >
> > Ticker
> >
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
12