IO-problem with actual trunk

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

IO-problem with actual trunk

osm-8
Hi,

while generating a map of Oman I got the attached error-message (see
below 1010). The map is generated out of planet.o5m.

Henning

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap_error.log (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,
please post a link to the o5m input file.

Gerd

Henning Scholland wrote
Hi,

while generating a map of Oman I got the attached error-message (see
below 1010). The map is generated out of planet.o5m.

Henning

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap_error.log (3K) <http://gis.19327.n5.nabble.com/attachment/5786885/0/mkgmap_error.log>
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

osm-8
Here it is: http://www.aighes.de/data/1010.tar.gz

Henning



Am 22.11.2013 19:28, schrieb GerdP:

> Hi Henning,
> please post a link to the o5m input file.
>
> Gerd
>
>
> Henning Scholland wrote
>> Hi,
>>
>> while generating a map of Oman I got the attached error-message (see
>> below 1010). The map is generated out of planet.o5m.
>>
>> Henning
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>
>> mkgmap-dev@.org
>
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> mkgmap_error.log (3K)
>> &lt;http://gis.19327.n5.nabble.com/attachment/5786885/0/mkgmap_error.log&gt;
>
>
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/IO-problem-with-actual-trunk-tp5786885p5786888.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,

hmm, it seems that all files are corrupted, since osmconvert cannot read them as well
without printing error messages, so I don't think the problem was caused by an IO error.
I assume the files were created with latest splitter?
Did you create the planet file using update procedures or download it ?
If the latter is true, I should be able to reproduce it when I download latest planet.

Gerd

Henning Scholland wrote
Here it is: http://www.aighes.de/data/1010.tar.gz

Henning



Am 22.11.2013 19:28, schrieb GerdP:
> Hi Henning,
> please post a link to the o5m input file.
>
> Gerd
>
>
> Henning Scholland wrote
>> Hi,
>>
>> while generating a map of Oman I got the attached error-message (see
>> below 1010). The map is generated out of planet.o5m.
>>
>> Henning
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>
>> mkgmap-dev@.org
>
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> mkgmap_error.log (3K)
>> <http://gis.19327.n5.nabble.com/attachment/5786885/0/mkgmap_error.log>
>
>
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/IO-problem-with-actual-trunk-tp5786885p5786888.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

osm-8
No it's a combination of an updated planet and a srtm file. I started
downloading a fresh planet and will try that tomorrow.

Splitting was done with latest splitter.

Henning

Am 22.11.2013 23:08, schrieb GerdP:

> Hi Henning,
>
> hmm, it seems that all files are corrupted, since osmconvert cannot read
> them as well
> without printing error messages, so I don't think the problem was caused by
> an IO error.
> I assume the files were created with latest splitter?
> Did you create the planet file using update procedures or download it ?
> If the latter is true, I should be able to reproduce it when I download
> latest planet.
>
> Gerd
>
>
> Henning Scholland wrote
>> Here it is: http://www.aighes.de/data/1010.tar.gz
>>
>> Henning
>>
>>
>>
>> Am 22.11.2013 19:28, schrieb GerdP:
>>> Hi Henning,
>>> please post a link to the o5m input file.
>>>
>>> Gerd
>>>
>>>
>>> Henning Scholland wrote
>>>> Hi,
>>>>
>>>> while generating a map of Oman I got the attached error-message (see
>>>> below 1010). The map is generated out of planet.o5m.
>>>>
>>>> Henning
>>>>
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>
>>>> mkgmap-dev@.org
>>>
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>>> mkgmap_error.log (3K)
>>>> &lt;http://gis.19327.n5.nabble.com/attachment/5786885/0/mkgmap_error.log&gt;
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gis.19327.n5.nabble.com/IO-problem-with-actual-trunk-tp5786885p5786888.html
>>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>>
>
>> mkgmap-dev@.org
>
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>
>> mkgmap-dev@.org
>
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
>
> --
> View this message in context: http://gis.19327.n5.nabble.com/IO-problem-with-actual-trunk-tp5786885p5786913.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,

okay, maybe it was an error in the update process. I see that
e.g. OSM stats also had problems:
http://osmstats.altogetherlost.com/index.php?item=nodes

Anyway, it is obvious the code in splitter is missing a check,
either in the o5m read or in the write routine (or both) :-(

If you can reproduce the problem with the downloaded planet,
maybe try to use --output=pbf first.

Gerd

> Date: Fri, 22 Nov 2013 23:37:02 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] IO-problem with actual trunk
>
> No it's a combination of an updated planet and a srtm file. I started
> downloading a fresh planet and will try that tomorrow.
>
> Splitting was done with latest splitter.
>
> Henning
>
> Am 22.11.2013 23:08, schrieb GerdP:
> > Hi Henning,
> >
> > hmm, it seems that all files are corrupted, since osmconvert cannot read
> > them as well
> > without printing error messages, so I don't think the problem was caused by
> > an IO error.
> > I assume the files were created with latest splitter?
> > Did you create the planet file using update procedures or download it ?
> > If the latter is true, I should be able to reproduce it when I download
> > latest planet.
> >
> > Gerd
> >
> >
> > Henning Scholland wrote
> >> Here it is: http://www.aighes.de/data/1010.tar.gz
> >>
> >> Henning
> >>
> >>
> >>
> >> Am 22.11.2013 19:28, schrieb GerdP:
> >>> Hi Henning,
> >>> please post a link to the o5m input file.
> >>>
> >>> Gerd
> >>>
> >>>
> >>> Henning Scholland wrote
> >>>> Hi,
> >>>>
> >>>> while generating a map of Oman I got the attached error-message (see
> >>>> below 1010). The map is generated out of planet.o5m.
> >>>>
> >>>> Henning
> >>>>
> >>>> _______________________________________________
> >>>> mkgmap-dev mailing list
> >>>
> >>>> mkgmap-dev@.org
> >>>
> >>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >>>>
> >>>> mkgmap_error.log (3K)
> >>>> &lt;http://gis.19327.n5.nabble.com/attachment/5786885/0/mkgmap_error.log&gt;
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>> http://gis.19327.n5.nabble.com/IO-problem-with-actual-trunk-tp5786885p5786888.html
> >>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> >>> _______________________________________________
> >>> mkgmap-dev mailing list
> >>>
> >
> >> mkgmap-dev@.org
> >
> >>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >>>
> >>
> >> _______________________________________________
> >> mkgmap-dev mailing list
> >
> >> mkgmap-dev@.org
> >
> >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
> >
> >
> >
> >
> > --
> > View this message in context: http://gis.19327.n5.nabble.com/IO-problem-with-actual-trunk-tp5786885p5786913.html
> > Sent from the Mkgmap Development mailing list archive at Nabble.com.
> > _______________________________________________
> > mkgmap-dev mailing list
> > [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

osm-8
Am 23.11.2013 09:36, schrieb Gerd Petermann:

> Hi Henning,
>
> okay, maybe it was an error in the update process. I see that
> e.g. OSM stats also had problems:
> http://osmstats.altogetherlost.com/index.php?item=nodes
>
> Anyway, it is obvious the code in splitter is missing a check,
> either in the o5m read or in the write routine (or both) :-(
>
> If you can reproduce the problem with the downloaded planet,
> maybe try to use --output=pbf first.
>
> Gerd

Hi Gerd,
I'm not that familiar with o5m-format, but it is possible,, that only a
part of the world is corrupted?
Maybe you remember, that I'm splitting all my maps at ones. And all
other maps are correct. If it's not possible, I think splitter have a
problem with this.

Also I can update the used planet-file with osmupdate.

Henning

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,

> I'm not that familiar with o5m-format, but it is possible,, that only a
> part of the world is corrupted?
yes, I think so. The o5m format doesn't contain checksums or so,
so the read routine tends to skip unexpected data until it finds
something readable.

> Maybe you remember, that I'm splitting all my maps at ones. And all
> other maps are correct. If it's not possible, I think splitter have a
> problem with this.

okay, that implies that the Oman area contains a special case,
maybe a special string.
Please test:
What happens if you create the Oman files with --output=pbf?
If the created files are okay for osmconvert, the problem is
probably in splitters o5m write routine.

Gerd


_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
In reply to this post by osm-8
Hi Henning,

I think the problem is in the splitter o5m write routine.
It seems to write wrong string references for the
SRTM data.  Maybe I can reproduce the problem
with a small test routine.
Gerd



> Date: Sat, 23 Nov 2013 10:26:07 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] IO-problem with actual trunk
>
> Am 23.11.2013 09:36, schrieb Gerd Petermann:
> > Hi Henning,
> >
> > okay, maybe it was an error in the update process. I see that
> > e.g. OSM stats also had problems:
> > http://osmstats.altogetherlost.com/index.php?item=nodes
> >
> > Anyway, it is obvious the code in splitter is missing a check,
> > either in the o5m read or in the write routine (or both) :-(
> >
> > If you can reproduce the problem with the downloaded planet,
> > maybe try to use --output=pbf first.
> >
> > Gerd
>
> Hi Gerd,
> I'm not that familiar with o5m-format, but it is possible,, that only a
> part of the world is corrupted?
> Maybe you remember, that I'm splitting all my maps at ones. And all
> other maps are correct. If it's not possible, I think splitter have a
> problem with this.
>
> Also I can update the used planet-file with osmupdate.
>
> Henning
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
In reply to this post by osm-8
Hi Henning,

I was not able to reproduce the problem, but I found a potential
error in the o5m write routine: it did not write enough reset bytes.
I've changed that in r313. Please try if this helps.

Gerd

> Date: Sat, 23 Nov 2013 10:26:07 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] IO-problem with actual trunk
>
> Am 23.11.2013 09:36, schrieb Gerd Petermann:
> > Hi Henning,
> >
> > okay, maybe it was an error in the update process. I see that
> > e.g. OSM stats also had problems:
> > http://osmstats.altogetherlost.com/index.php?item=nodes
> >
> > Anyway, it is obvious the code in splitter is missing a check,
> > either in the o5m read or in the write routine (or both) :-(
> >
> > If you can reproduce the problem with the downloaded planet,
> > maybe try to use --output=pbf first.
> >
> > Gerd
>
> Hi Gerd,
> I'm not that familiar with o5m-format, but it is possible,, that only a
> part of the world is corrupted?
> Maybe you remember, that I'm splitting all my maps at ones. And all
> other maps are correct. If it's not possible, I think splitter have a
> problem with this.
>
> Also I can update the used planet-file with osmupdate.
>
> Henning
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

osm-8
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Gerd,
works as expected.

Henning

Am 23.11.2013 15:11, schrieb Gerd Petermann:

> Hi Henning,
>
> I was not able to reproduce the problem, but I found a potential
> error in the o5m write routine: it did not write enough reset
> bytes. I've changed that in r313. Please try if this helps.
>
> Gerd
>
>> Date: Sat, 23 Nov 2013 10:26:07 +0100 From: [hidden email] To:
>> [hidden email] Subject: Re: [mkgmap-dev]
>> IO-problem with actual trunk
>>
>> Am 23.11.2013 09:36, schrieb Gerd Petermann:
>>> Hi Henning,
>>>
>>> okay, maybe it was an error in the update process. I see that
>>> e.g. OSM stats also had problems:
>>> http://osmstats.altogetherlost.com/index.php?item=nodes
>>>
>>> Anyway, it is obvious the code in splitter is missing a check,
>>> either in the o5m read or in the write routine (or both) :-(
>>>
>>> If you can reproduce the problem with the downloaded planet,
>>> maybe try to use --output=pbf first.
>>>
>>> Gerd
>>
>> Hi Gerd, I'm not that familiar with o5m-format, but it is
>> possible,, that only a part of the world is corrupted? Maybe you
>> remember, that I'm splitting all my maps at ones. And all other
>> maps are correct. If it's not possible, I think splitter have a
>> problem with this.
>>
>> Also I can update the used planet-file with osmupdate.
>>
>> Henning
>>
>> _______________________________________________ mkgmap-dev
>> mailing list [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________ mkgmap-dev mailing
> list [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSkSbBAAoJEPFgsWC7jOeTYnYH/Rjsy/qYyTw54lXzEarFdpj0
EBOxV53ug008pO6Bn1lv2NFochKU8WV4IVpYIxTWfQVTxc/XNvkuR1O45RpcoNki
lnrvdeyKuCFyyfLqqeu5kzZ9Ed9YYufWf0IpaAnXGjvPYv0x2WAY5oLRa2A4q9Sx
jVDhkQaYurBnCnTAjotYtVT7aqppi0CBRi5ZFLbHkhs/stXW5zemh5QekSZqub6a
sCPUiyoixH7snTaWezQJQ5v19CBgEHlGa1Cl/KVyhNXob4LWhf9pqEoUXtIyrwL6
hP0S1rx+Ne5kkeBn2tG8no1bjZWK/nWVi3tiVZNnDwR6P25797uprkSoVvQCJFg=
=0f7Q
-----END PGP SIGNATURE-----

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,

I still don't understand why this change is important for
mkgmap, maybe I did not fix the real problem.
It would be great if you could reproduce it again with r312
and --output=o5m .
The problem was e.g. in file 10100011.o5m.
Next, execute splitter r312(or r311) with --output=pbf and post a
link to both files.

Gerd

> Date: Sat, 23 Nov 2013 23:05:54 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [mkgmap-dev] IO-problem with actual trunk
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Gerd,
> works as expected.
>
> Henning
>
> Am 23.11.2013 15:11, schrieb Gerd Petermann:
> > Hi Henning,
> >
> > I was not able to reproduce the problem, but I found a potential
> > error in the o5m write routine: it did not write enough reset
> > bytes. I've changed that in r313. Please try if this helps.
> >
> > Gerd
> >
> >> Date: Sat, 23 Nov 2013 10:26:07 +0100 From: [hidden email] To:
> >> [hidden email] Subject: Re: [mkgmap-dev]
> >> IO-problem with actual trunk
> >>
> >> Am 23.11.2013 09:36, schrieb Gerd Petermann:
> >>> Hi Henning,
> >>>
> >>> okay, maybe it was an error in the update process. I see that
> >>> e.g. OSM stats also had problems:
> >>> http://osmstats.altogetherlost.com/index.php?item=nodes
> >>>
> >>> Anyway, it is obvious the code in splitter is missing a check,
> >>> either in the o5m read or in the write routine (or both) :-(
> >>>
> >>> If you can reproduce the problem with the downloaded planet,
> >>> maybe try to use --output=pbf first.
> >>>
> >>> Gerd
> >>
> >> Hi Gerd, I'm not that familiar with o5m-format, but it is
> >> possible,, that only a part of the world is corrupted? Maybe you
> >> remember, that I'm splitting all my maps at ones. And all other
> >> maps are correct. If it's not possible, I think splitter have a
> >> problem with this.
> >>
> >> Also I can update the used planet-file with osmupdate.
> >>
> >> Henning
> >>
> >> _______________________________________________ mkgmap-dev
> >> mailing list [hidden email]
> >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
> >
> >
> >
> > _______________________________________________ mkgmap-dev mailing
> > list [hidden email]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.14 (GNU/Linux)
>
> iQEcBAEBAgAGBQJSkSbBAAoJEPFgsWC7jOeTYnYH/Rjsy/qYyTw54lXzEarFdpj0
> EBOxV53ug008pO6Bn1lv2NFochKU8WV4IVpYIxTWfQVTxc/XNvkuR1O45RpcoNki
> lnrvdeyKuCFyyfLqqeu5kzZ9Ed9YYufWf0IpaAnXGjvPYv0x2WAY5oLRa2A4q9Sx
> jVDhkQaYurBnCnTAjotYtVT7aqppi0CBRi5ZFLbHkhs/stXW5zemh5QekSZqub6a
> sCPUiyoixH7snTaWezQJQ5v19CBgEHlGa1Cl/KVyhNXob4LWhf9pqEoUXtIyrwL6
> hP0S1rx+Ne5kkeBn2tG8no1bjZWK/nWVi3tiVZNnDwR6P25797uprkSoVvQCJFg=
> =0f7Q
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> mkgmap-dev mailing list
> [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

osm-8
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Gerd
Here it is: http://www.aighes.de/data/mkgmap.zip

Henning

Am 24.11.2013 09:02, schrieb Gerd Petermann:

> Hi Henning,
>
> I still don't understand why this change is important for mkgmap,
> maybe I did not fix the real problem. It would be great if you
> could reproduce it again with r312 and --output=o5m . The problem
> was e.g. in file 10100011.o5m. Next, execute splitter r312(or r311)
> with --output=pbf and post a link to both files.
>
> Gerd
>
>> Date: Sat, 23 Nov 2013 23:05:54 +0100 From: [hidden email] To:
>> [hidden email] Subject: Re: [mkgmap-dev]
>> IO-problem with actual trunk
>>
> Hi Gerd, works as expected.
>
> Henning
>
> Am 23.11.2013 15:11, schrieb Gerd Petermann:
>>>> Hi Henning,
>>>>
>>>> I was not able to reproduce the problem, but I found a
>>>> potential error in the o5m write routine: it did not write
>>>> enough reset bytes. I've changed that in r313. Please try if
>>>> this helps.
>>>>
>>>> Gerd
>>>>
>>>>> Date: Sat, 23 Nov 2013 10:26:07 +0100 From: [hidden email]
>>>>> To: [hidden email] Subject: Re:
>>>>> [mkgmap-dev] IO-problem with actual trunk
>>>>>
>>>>> Am 23.11.2013 09:36, schrieb Gerd Petermann:
>>>>>> Hi Henning,
>>>>>>
>>>>>> okay, maybe it was an error in the update process. I see
>>>>>> that e.g. OSM stats also had problems:
>>>>>> http://osmstats.altogetherlost.com/index.php?item=nodes
>>>>>>
>>>>>> Anyway, it is obvious the code in splitter is missing a
>>>>>> check, either in the o5m read or in the write routine (or
>>>>>> both) :-(
>>>>>>
>>>>>> If you can reproduce the problem with the downloaded
>>>>>> planet, maybe try to use --output=pbf first.
>>>>>>
>>>>>> Gerd
>>>>>
>>>>> Hi Gerd, I'm not that familiar with o5m-format, but it is
>>>>> possible,, that only a part of the world is corrupted?
>>>>> Maybe you remember, that I'm splitting all my maps at ones.
>>>>> And all other maps are correct. If it's not possible, I
>>>>> think splitter have a problem with this.
>>>>>
>>>>> Also I can update the used planet-file with osmupdate.
>>>>>
>>>>> Henning
>>>>>
>>>>> _______________________________________________ mkgmap-dev
>>>>> mailing list [hidden email]
>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________ mkgmap-dev
>>>> mailing list [hidden email]
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>
>> _______________________________________________ mkgmap-dev
>> mailing list [hidden email]
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________ mkgmap-dev mailing
> list [hidden email]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSkisgAAoJEPFgsWC7jOeT90UH/i8z82v8ysHXuw8iOK4+OrWk
FMMz7lWY/E6ngsijFUijWxflA8NZDkavwuc08QSyg+7x6y+cSa53u31hQYTzIYhc
FZGz47NRSWc/pkSC9I2BNu6uFdRDjOQgPP/1HKVzHfcwXvTNn4C4cr1CXdOMGOD3
pKOjhSP9N1y1htGGCyg5ujTcLmQjRlbjzbL3cvg0TYqDYjYucnzxTUzRCopMn3cL
f6Ew/QcBIc/wF3pHkl9oEaKoLb60AXl/4o1apJrZF12mxNFpM/ffyI8nTn8AAJZc
fHs91VS4QrQkWppX1OidPhVpRj9HbGUNke2Ne+qBL+Jxs6CIDWy9EadfsLz0qlE=
=kkbL
-----END PGP SIGNATURE-----

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,

I am confused now. None of the o5m files contains the error that was in yesterdays
data, and the file sizes are very different (new files are smaller).
Do you have an idea why?

Gerd

Henning Scholland wrote
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Gerd
Here it is: http://www.aighes.de/data/mkgmap.zip

Henning

Am 24.11.2013 09:02, schrieb Gerd Petermann:
> Hi Henning,
>
> I still don't understand why this change is important for mkgmap,
> maybe I did not fix the real problem. It would be great if you
> could reproduce it again with r312 and --output=o5m . The problem
> was e.g. in file 10100011.o5m. Next, execute splitter r312(or r311)
> with --output=pbf and post a link to both files.
>
> Gerd
>
>> Date: Sat, 23 Nov 2013 23:05:54 +0100 From: [hidden email] To:
>> [hidden email] Subject: Re: [mkgmap-dev]
>> IO-problem with actual trunk
>>
> Hi Gerd, works as expected.
>
> Henning
>
> Am 23.11.2013 15:11, schrieb Gerd Petermann:
>>>> Hi Henning,
>>>>
>>>> I was not able to reproduce the problem, but I found a
>>>> potential error in the o5m write routine: it did not write
>>>> enough reset bytes. I've changed that in r313. Please try if
>>>> this helps.
>>>>
>>>> Gerd
>>>>
>>>>> Date: Sat, 23 Nov 2013 10:26:07 +0100 From: [hidden email]
>>>>> To: [hidden email] Subject: Re:
>>>>> [mkgmap-dev] IO-problem with actual trunk
>>>>>
>>>>> Am 23.11.2013 09:36, schrieb Gerd Petermann:
>>>>>> Hi Henning,
>>>>>>
>>>>>> okay, maybe it was an error in the update process. I see
>>>>>> that e.g. OSM stats also had problems:
>>>>>> http://osmstats.altogetherlost.com/index.php?item=nodes
>>>>>>
>>>>>> Anyway, it is obvious the code in splitter is missing a
>>>>>> check, either in the o5m read or in the write routine (or
>>>>>> both) :-(
>>>>>>
>>>>>> If you can reproduce the problem with the downloaded
>>>>>> planet, maybe try to use --output=pbf first.
>>>>>>
>>>>>> Gerd
>>>>>
>>>>> Hi Gerd, I'm not that familiar with o5m-format, but it is
>>>>> possible,, that only a part of the world is corrupted?
>>>>> Maybe you remember, that I'm splitting all my maps at ones.
>>>>> And all other maps are correct. If it's not possible, I
>>>>> think splitter have a problem with this.
>>>>>
>>>>> Also I can update the used planet-file with osmupdate.
>>>>>
>>>>> Henning
>>>>>
>>>>> _______________________________________________ mkgmap-dev
>>>>> mailing list [hidden email] 
>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________ mkgmap-dev
>>>> mailing list [hidden email] 
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>>
>>
>> _______________________________________________ mkgmap-dev
>> mailing list [hidden email] 
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________ mkgmap-dev mailing
> list [hidden email] 
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)

iQEcBAEBAgAGBQJSkisgAAoJEPFgsWC7jOeT90UH/i8z82v8ysHXuw8iOK4+OrWk
FMMz7lWY/E6ngsijFUijWxflA8NZDkavwuc08QSyg+7x6y+cSa53u31hQYTzIYhc
FZGz47NRSWc/pkSC9I2BNu6uFdRDjOQgPP/1HKVzHfcwXvTNn4C4cr1CXdOMGOD3
pKOjhSP9N1y1htGGCyg5ujTcLmQjRlbjzbL3cvg0TYqDYjYucnzxTUzRCopMn3cL
f6Ew/QcBIc/wF3pHkl9oEaKoLb60AXl/4o1apJrZF12mxNFpM/ffyI8nTn8AAJZc
fHs91VS4QrQkWppX1OidPhVpRj9HbGUNke2Ne+qBL+Jxs6CIDWy9EadfsLz0qlE=
=kkbL
-----END PGP SIGNATURE-----

_______________________________________________
mkgmap-dev mailing list
[hidden email]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: IO-problem with actual trunk

Gerd Petermann
Hi Henning,

GerdP wrote
Hi Henning,

I am confused now. None of the o5m files contains the error that was in yesterdays
data, and the file sizes are very different (new files are smaller).
Do you have an idea why?
A few more details:
Executing osmconvert with yesterdays data always shows this:
c:\temp>osmconvert 10100001.o5m -o=10100001.o5m.osm
osmconvert Warning: wrong sequence at relation 305138
osmconvert Warning: next object is node 10216945201
osmconvert Warning: osmconvert Warning: unexpected contents after logical end of
 file: 10100001.o5m

c:\temp>osmconvert 10100002.o5m -o=10100002.o5m.osm
osmconvert Warning: wrong sequence at relation 383908
osmconvert Warning: next object is node 10204620185
osmconvert Warning: osmconvert Warning: unexpected contents after logical end of
 file: 10100002.o5m


and more errors for 10100011.o5m:
c:\temp>osmconvert 10100011.o5m -o=10100011.o5m.osm
osmconvert Warning: wrong sequence at relation 152458
osmconvert Warning: next object is node 10221551320
osmconvert Warning: invalid .o5m string reference: 70->131436542
osmconvert Warning: invalid .o5m string reference: 70->88921963
osmconvert Warning: invalid .o5m string reference: 70->506


With the new data I see only the expected messages:
c:\temp\henning>osmconvert 10100001.o5m -o=10100001.o5m.osm
osmconvert Warning: wrong sequence at relation 305138
osmconvert Warning: next object is node 10216945201

c:\temp\henning>osmconvert 10100002.o5m -o=10100002.o5m.osm
osmconvert Warning: wrong sequence at relation 383908
osmconvert Warning: next object is node 10204620185

Some of the converted .osm files are equal, but the o5m files are different (garbage at the end
of the file). So, in other words, it seems that your file system was corrupted.

Gerd