Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

Bart Smienk
Hi,

I've found out my OSM Tile Server has stopped updating and osm2pgsql
(v1.2) gives off the following error:

DB writer thread failed due to ERROR: result COPY_END for
planet_osm_line failed: ERROR:  invalid input syntax for integer: "-"
CONTEXT:  COPY planet_osm_line, line 1437, column layer: "-"

Looking through its changefile, the following entry can be found:

     <way id="742619958" version="1" timestamp="2019-11-06T15:32:03Z"
uid="978786" user="padvinder" changeset="76711047">
       <nd ref="6952022619"/>
       <nd ref="6952022612"/>
       <tag k="layer" v="-"/>
       <tag k="tunnel" v="culvert"/>
       <tag k="waterway" v="drain"/>
     </way>

This seems to be malformed, as the layer should've never been "-", as
this is not a valid integer.
You can see this for yourself at:
https://planet.openstreetmap.org/replication/minute/003/747/729.osc.gz
Also more information on the layer key is over here:
https://wiki.openstreetmap.org/wiki/Key:layer

Not sure where to make the actual bug report, so I'm mailing the dev
mailgroup, does anyone of you know where this issue should be posted so
this issue won't present itself again in future replication files?

Let me know.

Kind regards,
Bart Smienk

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

Maarten Deen
On 2019-11-08 17:04, Bart Smienk wrote:

> Hi,
>
> I've found out my OSM Tile Server has stopped updating and osm2pgsql
> (v1.2) gives off the following error:
>
> DB writer thread failed due to ERROR: result COPY_END for
> planet_osm_line failed: ERROR:  invalid input syntax for integer: "-"
> CONTEXT:  COPY planet_osm_line, line 1437, column layer: "-"
>
> Looking through its changefile, the following entry can be found:
>
>     <way id="742619958" version="1" timestamp="2019-11-06T15:32:03Z"
> uid="978786" user="padvinder" changeset="76711047">
>       <nd ref="6952022619"/>
>       <nd ref="6952022612"/>
>       <tag k="layer" v="-"/>
>       <tag k="tunnel" v="culvert"/>
>       <tag k="waterway" v="drain"/>
>     </way>
>
> This seems to be malformed, as the layer should've never been "-", as
> this is not a valid integer.
> You can see this for yourself at:
> https://planet.openstreetmap.org/replication/minute/003/747/729.osc.gz
> Also more information on the layer key is over here:
> https://wiki.openstreetmap.org/wiki/Key:layer
>
> Not sure where to make the actual bug report, so I'm mailing the dev
> mailgroup, does anyone of you know where this issue should be posted
> so this issue won't present itself again in future replication files?

The changeset data gives Go Map!! as editor which can be found on
GitHub: https://github.com/bryceco/GoMap

But as you can see on taginfo there are lots of non-integer values in
OSM.
https://taginfo.openstreetmap.org/keys/?key=layer#values
So the tile server software should also do some checking and error
handling if it requires an integer. I'm surprised that osm2pgsql
apparently has a check on key/value pairs and sees this as an error.

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

Developer Discussion mailing list
In reply to this post by Bart Smienk
I corrected the layer from (-) to (-1).  Such values should probably be spotted by the API and rejected as errors.

 
Pierre


Le vendredi 8 novembre 2019 11 h 11 min 03 s UTC−5, Bart Smienk <[hidden email]> a écrit :


Hi,

I've found out my OSM Tile Server has stopped updating and osm2pgsql
(v1.2) gives off the following error:

DB writer thread failed due to ERROR: result COPY_END for
planet_osm_line failed: ERROR:  invalid input syntax for integer: "-"
CONTEXT:  COPY planet_osm_line, line 1437, column layer: "-"

Looking through its changefile, the following entry can be found:

    <way id="742619958" version="1" timestamp="2019-11-06T15:32:03Z"
uid="978786" user="padvinder" changeset="76711047">
      <nd ref="6952022619"/>
      <nd ref="6952022612"/>
      <tag k="layer" v="-"/>
      <tag k="tunnel" v="culvert"/>
      <tag k="waterway" v="drain"/>
    </way>

This seems to be malformed, as the layer should've never been "-", as
this is not a valid integer.
You can see this for yourself at:
Also more information on the layer key is over here:

Not sure where to make the actual bug report, so I'm mailing the dev
mailgroup, does anyone of you know where this issue should be posted so
this issue won't present itself again in future replication files?

Let me know.

Kind regards,
Bart Smienk

_______________________________________________
dev mailing list

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

Michael Reichert-2
In reply to this post by Bart Smienk
Hi Bart,

Am 08.11.19 um 17:04 schrieb Bart Smienk:

> I've found out my OSM Tile Server has stopped updating and osm2pgsql
> (v1.2) gives off the following error:
>
> DB writer thread failed due to ERROR: result COPY_END for
> planet_osm_line failed: ERROR:  invalid input syntax for integer: "-"
> CONTEXT:  COPY planet_osm_line, line 1437, column layer: "-"
>
> Looking through its changefile, the following entry can be found:
>
>     <way id="742619958" version="1" timestamp="2019-11-06T15:32:03Z"
> uid="978786" user="padvinder" changeset="76711047">
>       <nd ref="6952022619"/>
>       <nd ref="6952022612"/>
>       <tag k="layer" v="-"/>
>       <tag k="tunnel" v="culvert"/>
>       <tag k="waterway" v="drain"/>
>     </way>
>
> This seems to be malformed, as the layer should've never been "-", as
> this is not a valid integer.
It is valid from API point of view as any tags are just a pair of
strings (up to 256 characters). The OSM API is living freedom of
tagging, it does care how your keys and values look like except some
basic checks such as length of the strings.

> You can see this for yourself at:
> https://planet.openstreetmap.org/replication/minute/003/747/729.osc.gz
> Also more information on the layer key is over here:
> https://wiki.openstreetmap.org/wiki/Key:layer
>
> Not sure where to make the actual bug report, so I'm mailing the dev
> mailgroup, does anyone of you know where this issue should be posted so
> this issue won't present itself again in future replication files?

It is an Osm2pgsql bug which should check that valid integers are
written via PostgreSQL COPY into a integer column. The bug tracker is at
https://github.com/openstreetmap/osm2pgsql/issues

I cannot reproduce your bug. I tried the master branch (currently equal
to 1.2.0 release).

This does not lead to the bug you observed:
./osm2pgsql -d osm2pgsqldev \
--style ~/git/openstreetmap-carto/openstreetmap-carto.style \
--tag-transform ~/git/openstreetmap-carto/openstreetmap-carto.lua \
--cache 10 w742619958.osm

And even using C tag transform works:
./osm2pgsql -d osm2pgsqldev \
--style ~/git/openstreetmap-carto/openstreetmap-carto.style \
--cache 10 w742619958.osm

Using the Osm2pgsql style file (and C tag transforms) works as well:
./osm2pgsql -d osm2pgsqldev --style ../default.style --cache 10
/tmp/w742619958.osm

The input file is (without personal metadata which does not change
anything here):
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.7.5 (13835
thorn-02.openstreetmap.org)" copyright="OpenStreetMap and contributors"
attribution="http://www.openstreetmap.org/copyright"
license="http://opendatacommons.org/licenses/odbl/1-0/">
 <node id="6952022612" visible="true" version="1"
timestamp="2019-11-06T15:32:03Z" lat="52.6139789" lon="6.6775740"/>
 <node id="6952022619" visible="true" version="1"
timestamp="2019-11-06T15:32:03Z" lat="52.6139788" lon="6.6774594"/>
 <way id="742619958" visible="true" version="1"
timestamp="2019-11-06T15:32:03Z">
  <nd ref="6952022619"/>
  <nd ref="6952022612"/>
  <tag k="layer" v="-"/>
  <tag k="tunnel" v="culvert"/>
  <tag k="waterway" v="drain"/>
 </way>
</osm>

All three commands import the way into the database but the layer column
is empty as expected for the first and the second one.

How can your bug be reproduced?

Best regards

Michael



--
Michael Reichert      www.geofabrik.de
Geofabrik GmbH        Handelsregister: HRB Mannheim 703657
Amalienstr. 44        Geschaeftsfuehrung: C. Karch, F. Ramm
76133 Karlsruhe       Tel: 0721-1803560-3
[hidden email] Fax: 0721-1803560-9


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

SimonPoole

Am 08.11.2019 um 17:41 schrieb Michael Reichert:
>
> strings (up to 256 characters).
255


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

signature.asc (495 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issue with way id 742619958 / changeset 76711047 from 2019-11-06T15:32:03Z (Invalid layer value)

Sarah Hoffmann
In reply to this post by Bart Smienk
On Fri, Nov 08, 2019 at 05:04:34PM +0100, Bart Smienk wrote:

> Hi,
>
> I've found out my OSM Tile Server has stopped updating and osm2pgsql (v1.2)
> gives off the following error:
>
> DB writer thread failed due to ERROR: result COPY_END for planet_osm_line
> failed: ERROR:  invalid input syntax for integer: "-"
> CONTEXT:  COPY planet_osm_line, line 1437, column layer: "-"
>
> Looking through its changefile, the following entry can be found:
>
>     <way id="742619958" version="1" timestamp="2019-11-06T15:32:03Z"
> uid="978786" user="padvinder" changeset="76711047">
>       <nd ref="6952022619"/>
>       <nd ref="6952022612"/>
>       <tag k="layer" v="-"/>
>       <tag k="tunnel" v="culvert"/>
>       <tag k="waterway" v="drain"/>
>     </way>
>
> This seems to be malformed, as the layer should've never been "-", as this
> is not a valid integer.
> You can see this for yourself at:
> https://planet.openstreetmap.org/replication/minute/003/747/729.osc.gz
> Also more information on the layer key is over here:
> https://wiki.openstreetmap.org/wiki/Key:layer
>
> Not sure where to make the actual bug report, so I'm mailing the dev
> mailgroup, does anyone of you know where this issue should be posted so this
> issue won't present itself again in future replication files?

Looks like a potential bug in osm2pgsql's int parsing. I've opened an issue:
https://github.com/openstreetmap/osm2pgsql/issues/988

Please follow there for further updates.

Kind regards

Sarah

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev