Split gmapsupp.img

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Split gmapsupp.img

Carlos Dávila-5
As far a I know, mkgmap is able to produce unlimited size gmapsupp
files, but SD cards and internal device memories have a limit of 4 GB in
a single file. To overcome that limit, recent versions of Garmin City
Navigator maps are split into several gmapsupp that work as a single one
in devices which include them. Would it be possible to add an option to
mkgmap, so that it automatically splits gmapsupp when you process a map
that is larger than 4 GB?

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Gerd Petermann
Hi Carlos,

see http://gis.19327.n8.nabble.com/Split-gmapsupp-img-files-over-4GB-tc5967656.html

I tried to code this at that time but got stuck because I didn't have the time to create such a large map to find out where MapInstall places the other files (e.g. MDR, TYP and if any other file or flag is needed.
Maybe I'll give it another try with a download of Europe made in 2019. Splitter will stress my PC for quite a while ...

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
Gesendet: Sonntag, 18. Oktober 2020 09:14
An: Development list for mkgmap
Betreff: [mkgmap-dev] Split gmapsupp.img

As far a I know, mkgmap is able to produce unlimited size gmapsupp
files, but SD cards and internal device memories have a limit of 4 GB in
a single file. To overcome that limit, recent versions of Garmin City
Navigator maps are split into several gmapsupp that work as a single one
in devices which include them. Would it be possible to add an option to
mkgmap, so that it automatically splits gmapsupp when you process a map
that is larger than 4 GB?

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Carlos Dávila-5
I'll have a look at that thread later. I can process large files in my
server if needed.

El 18/10/20 a las 9:50, Gerd Petermann escribió:

> Hi Carlos,
>
> see http://gis.19327.n8.nabble.com/Split-gmapsupp-img-files-over-4GB-tc5967656.html
>
> I tried to code this at that time but got stuck because I didn't have the time to create such a large map to find out where MapInstall places the other files (e.g. MDR, TYP and if any other file or flag is needed.
> Maybe I'll give it another try with a download of Europe made in 2019. Splitter will stress my PC for quite a while ...
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Sonntag, 18. Oktober 2020 09:14
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Split gmapsupp.img
>
> As far a I know, mkgmap is able to produce unlimited size gmapsupp
> files, but SD cards and internal device memories have a limit of 4 GB in
> a single file. To overcome that limit, recent versions of Garmin City
> Navigator maps are split into several gmapsupp that work as a single one
> in devices which include them. Would it be possible to add an option to
> mkgmap, so that it automatically splits gmapsupp when you process a map
> that is larger than 4 GB?
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Felix Hartmann-2
The other files have to be split as well I am pretty sure. Each map should also work independently. So it's just a matter of way take 3700MB for .IMG files to have enough overhead for the MDR/MDX. Except if no address search, then take 4050MB for split.

On Sun, 18 Oct 2020, 11:16 Carlos Dávila <[hidden email]> wrote:
I'll have a look at that thread later. I can process large files in my
server if needed.

El 18/10/20 a las 9:50, Gerd Petermann escribió:
> Hi Carlos,
>
> see http://gis.19327.n8.nabble.com/Split-gmapsupp-img-files-over-4GB-tc5967656.html
>
> I tried to code this at that time but got stuck because I didn't have the time to create such a large map to find out where MapInstall places the other files (e.g. MDR, TYP and if any other file or flag is needed.
> Maybe I'll give it another try with a download of Europe made in 2019. Splitter will stress my PC for quite a while ...
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <[hidden email]> im Auftrag von Carlos Dávila <[hidden email]>
> Gesendet: Sonntag, 18. Oktober 2020 09:14
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Split gmapsupp.img
>
> As far a I know, mkgmap is able to produce unlimited size gmapsupp
> files, but SD cards and internal device memories have a limit of 4 GB in
> a single file. To overcome that limit, recent versions of Garmin City
> Navigator maps are split into several gmapsupp that work as a single one
> in devices which include them. Would it be possible to add an option to
> mkgmap, so that it automatically splits gmapsupp when you process a map
> that is larger than 4 GB?
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Randolph J. Herber
In reply to this post by Carlos Dávila-5

The 4GB limit is a result of the fact that Garmin devices use FAT16B without LFS (only the very oldest) and FAT32 (all the rest) filesystems. Garmin devices do not support exFAT; though, devices with rewritable firmware could be made to support exFAT. The reason that FAT file systems are used are they are freely available and work on Linux, Windows and Apple operating systems.

Please, see:
https://en.wikipedia.org/wiki/File_Allocation_Table
https://physics.nist.gov/cuu/Units/binary.html
https://en.wikipedia.org/wiki/ExFAT

There are several sizes of concern: cluster size, maximum file size, maximum number of files, and maximum file system size. Each of these sizes are powers of two or powers of 2 less one. Because of the powers of 2, SI binary prefixes will be used. Remember only one file can be stored in a cluster and a file may have more than one cluster. The space at the end of the last cluster of a file is wasted.

FAT12  4KiB and 8KiB clusters, file size limited by volume size, 4068 files with 8 KiB clusters, 16MiB with 4KiB and 32MiB with 8 KiB clusters.

FAT16B  minimum file system sizes 8 MiB with 128, 32 MiB with 512*, and 256 MiB 4 KiB disk logical sector sizes, 2GiB-1 without LFS and 4GiB-1 with LFS, cluster sizes are powers of 2 from 512 to 32768, 65460 files with 32 KiB clusters, maximum file systems size is 65525 clusters.

FAT32  32 MiB-4.5 KiB (with 65525 clusters and 512 byte sectors) and 256 MiB-36 KiB (with 65525 clusters and 4 KiB sectors), same filesize limits as FAT16B, cluster sizes that are powers of 2 from 512 to 2,097,152.

exFAT  maximum file size practical 128 PiB-1, structural limit 16 EiB-1, maximum volume size practical 128 PiB, maximum number of files per directory 2,796,202.

* almost all non CHR disks are 512 byte sectored.

People will tell you that you can not use a SD card larger than a certain size dependent on Garmin receiver model. This is not true. The actual requirement is that a Garmin receiver only looks that the first primary partition on a SD card. Simply, divide the SD card into two partitions (in Windows, the disk manager can do this), format that partition as the appropriate FAT format, and put the map or maps there. The second partition can be formatted as one desires and used as desired. This is a handy space in which to store other previously made maps or even a complete map building system. I have done this on Garmin receivers as old as Garmin StreetPilot 2610 with SD cards as large as 256 GB (the card uses some of the memory for its management purposes).

On 10/18/2020 2:14 AM, Carlos Dávila wrote:
As far as I know, mkgmap is able to produce unlimited size gmapsupp files, but SD cards and internal device memories have a limit of 4 GB in a single file. To overcome that limit, recent versions of Garmin City Navigator maps are split into several gmapsupp that work as a single one in devices which include them. Would it be possible to add an option to mkgmap, so that it automatically splits gmapsupp when you process a map that is larger than 4 GB?

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

popej
In reply to this post by Carlos Dávila-5
Hi Carlos,

splitting could be useful, when you'd like to create img in one step.
Other option is to install map on PC and then use MapInstall to create
img. MapInstall does splitting for big maps.

--
Best regards,
Andrzej
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Carlos Dávila-5
El 19/10/20 a las 0:30, Andrzej Popowski escribió:
> Hi Carlos,
>
> splitting could be useful, when you'd like to create img in one step.
> Other option is to install map on PC and then use MapInstall to create
> img. MapInstall does splitting for big maps.
>
Hi Andrej

I know I can use MapSource/MapInstall to send parts of a map to device,
but many people have problems using this method, so I would like to have
way to provide ready to use gmapsupp files of my maps

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Felix Hartmann-2
MapInstall cannot be scripted to create a gmapsupp.img (plus setting it up on linux is a pain). I'm pretty sure the <4GB gmapsupp part files - shouldn't be too difficult if mkgmap can associate the address index for the choses tiles only. 
Right now for scripting I can only see how to create gmapsupp part files without address search.

Cutting Europe into smaller parts using osmcut or similar takes quite a lot of time, plus if you compile a full map of Europe anyhow already - would kinda double the work. And you would need to adapt the area rather often as osm data increases...
Having mkgmap split the map into parts would be great therefore (and also put out some sort of parts.list similar to the areas.list from splitter would be even better).

On Mon, 19 Oct 2020 at 17:15, Carlos Dávila <[hidden email]> wrote:
El 19/10/20 a las 0:30, Andrzej Popowski escribió:
> Hi Carlos,
>
> splitting could be useful, when you'd like to create img in one step.
> Other option is to install map on PC and then use MapInstall to create
> img. MapInstall does splitting for big maps.
>
Hi Andrej

I know I can use MapSource/MapInstall to send parts of a map to device,
but many people have problems using this method, so I would like to have
way to provide ready to use gmapsupp files of my maps

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


--
Felix Hartman - Openmtbmap.org & VeloMap.org


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

Gerd Petermann
Hi Felix,

I also thought about this alternative. Sad news that it doesn't work :(
Yesterday I managed to download a map for Europe, so I think I have now the data to play with this.

Gerd

________________________________________
Von: mkgmap-dev <[hidden email]> im Auftrag von Felix Hartmann <[hidden email]>
Gesendet: Montag, 19. Oktober 2020 20:37
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Split gmapsupp.img

MapInstall cannot be scripted to create a gmapsupp.img (plus setting it up on linux is a pain). I'm pretty sure the <4GB gmapsupp part files - shouldn't be too difficult if mkgmap can associate the address index for the choses tiles only.
Right now for scripting I can only see how to create gmapsupp part files without address search.

Cutting Europe into smaller parts using osmcut or similar takes quite a lot of time, plus if you compile a full map of Europe anyhow already - would kinda double the work. And you would need to adapt the area rather often as osm data increases...
Having mkgmap split the map into parts would be great therefore (and also put out some sort of parts.list similar to the areas.list from splitter would be even better).

On Mon, 19 Oct 2020 at 17:15, Carlos Dávila <[hidden email]<mailto:[hidden email]>> wrote:
El 19/10/20 a las 0:30, Andrzej Popowski escribió:
> Hi Carlos,
>
> splitting could be useful, when you'd like to create img in one step.
> Other option is to install map on PC and then use MapInstall to create
> img. MapInstall does splitting for big maps.
>
Hi Andrej

I know I can use MapSource/MapInstall to send parts of a map to device,
but many people have problems using this method, so I would like to have
way to provide ready to use gmapsupp files of my maps

_______________________________________________
mkgmap-dev mailing list
[hidden email]<mailto:[hidden email]>
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Split gmapsupp.img

popej
In reply to this post by Felix Hartmann-2
Hi Felix,

 > Right now for scripting I can only see how to create gmapsupp part
 > files without address search.

Doesn't mkgmap combine *.img with correct index? I mean you can get like
1000 of tiles after compiling a big area, but you can combine first 500
and next 500 separately.

I'm doing something similar for my maps. I split and compile area like
Africa in one go and then I run separate tasks to create parts of
Africa, without recompiling tiles. Mkgmap calculates indexes and preview
maps basing on provided list of img.

What could be a problem with partial maps is that one country can be
split into multiple parts. This can prohibit for correct address search,
if your current position is in one part, but searched address in
another. But I haven't tested that.

--
Best regards,
Andrzej
_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev