Broken (multi)polygon cleanup

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Broken (multi)polygon cleanup

Jochen123
Hi!

There are many broken (multi)polygon geometries in OSM data leading to
all sorts of problems. I have started an effort to get this cleaned up.
You can read more about this at http://area.jochentopf.com/ .

This effort has been running for about a month now and we can already see
some good results. See https://github.com/osmlab/fixing-polygons-in-osm/issues/15
for some news about what's been happening.

I am writing to the HOT list today, because a not-so-small part of the
problems come from HOT mappers. For instance, looking at one particular
problem here (spikes in closed ways), I am seeing about 5800 errors, of
which about 1600 are in areas covered by the HOT tasking manager. (Of
couse not all of these have been done in the course of HOT mapping, but
probably most of them.)

I am reaching out to some of the regional OSM communities and also to
HOT, because I want to a) ask for the help of all the communities in OSM
to fix those errors and b) generally raise awareness, so those problems
can be minimized in the future.

Please go to the web sites linked above and read up on what the problems
are and how we are fixing them. I also want to offer my help in
preparing special data extracts or Maproulette challenges etc. to help
the HOT community participate in the cleaning up effort in a way that
makes sense to you. I could, for instance, put those 1600 errors
mentioned above in a special Maproulette challenge, if that would be a
good way for HOT mappers to work on them.

Tell me what you think. I appreciate any feedback.

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Blake Girardot HOT/OSM
Hi Jochen,

Thank you very much for writing to the HOT list!

I think you will find a lot of enthusiasm for helping get these fixed
as well as improving our training and instructions to make sure we
minimize new mappers who come to OSM through HOT and Missing Maps
making these errors going forward.

Can you send me something that would help me see those 1600 examples
you found on OSM so I can see what might be going on and how to
improve things in HOT/MM workflow to reduce them?

Cheers,
Blake

On Wed, Mar 15, 2017 at 4:06 PM, Jochen Topf <[hidden email]> wrote:

> Hi!
>
> There are many broken (multi)polygon geometries in OSM data leading to
> all sorts of problems. I have started an effort to get this cleaned up.
> You can read more about this at http://area.jochentopf.com/ .
>
> This effort has been running for about a month now and we can already see
> some good results. See https://github.com/osmlab/fixing-polygons-in-osm/issues/15
> for some news about what's been happening.
>
> I am writing to the HOT list today, because a not-so-small part of the
> problems come from HOT mappers. For instance, looking at one particular
> problem here (spikes in closed ways), I am seeing about 5800 errors, of
> which about 1600 are in areas covered by the HOT tasking manager. (Of
> couse not all of these have been done in the course of HOT mapping, but
> probably most of them.)
>
> I am reaching out to some of the regional OSM communities and also to
> HOT, because I want to a) ask for the help of all the communities in OSM
> to fix those errors and b) generally raise awareness, so those problems
> can be minimized in the future.
>
> Please go to the web sites linked above and read up on what the problems
> are and how we are fixing them. I also want to offer my help in
> preparing special data extracts or Maproulette challenges etc. to help
> the HOT community participate in the cleaning up effort in a way that
> makes sense to you. I could, for instance, put those 1600 errors
> mentioned above in a special Maproulette challenge, if that would be a
> good way for HOT mappers to work on them.
>
> Tell me what you think. I appreciate any feedback.
>
> Jochen
> --
> Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688
>
> _______________________________________________
> HOT mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/hot



--
----------------------------------------------------
Blake Girardot
Humanitarian OpenStreetMap Team, TM3 Project Manager
skype: jblakegirardot
HOT Core Team Contact: [hidden email]
Live OSM Mapper-Support channel - https://hotosm-slack.herokuapp.com/
BE A PART OF HOT'S MICRO GRANTS: https://donate.hotosm.org/

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
Hi Blake,

On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> Can you send me something that would help me see those 1600 examples
> you found on OSM so I can see what might be going on and how to
> improve things in HOT/MM workflow to reduce them?

Those 1600 are some very specific errors I am currently looking at, but
only have those on my local machine currently. But for the wider picture
you can use the OSM Inspectors "area" view to see the scope of the problem.

Just go to any place where HOT/MM was active. Here is an example:
http://tools.geofabrik.de/osmi/?view=areas&lon=33.65927&lat=-2.07600&zoom=8

My current workflow is as follows: I choose one particular well-defined
problem and extract the data from OSM that shows this problem. Then I
put the data into Maproulette and document the problem and how to fix
it. Some problems are easier, some harder to fix. Mappers can choose
what problems to work on based on their skill level. If somebody
doesn't feel comfortable with one of the harder problems, there are
simpler ones they can work on. I also have some more general
descriptions for the experts who want to go exploring/fixing on their
own. Once the challenges are done, I choose a new problem and start the
process from the beginning.

Here are the current challenges: http://area.jochentopf.com/fixing.html

Usually I am splitting up larger challenges into regions, for instance
by continent or country. I could also do this by HOT activation area or
so if that makes sense for HOT mappers. Splitting up, often makes the
problems easier to fix, because problems inside an area are often very
similar, but different to problems in other areas due to mapping
priorities and methods. For instance, some regions have lots of problems
with buildings, others with landuse areas, etc. Concentrating on one
very well-defined problem at a time makes all this work simpler.

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

john whelan-2
This is similar to my pulling in Africa a country at a time and using JOSM to check for errors.

What I have noticed is one mapper will misstag then others will follow.  So sanitising the area or checking it for errors before the new HOT mappers start would probably improve the overall data quality as they tend to follow what is already mapped.

Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings in one small area by the same mapper left as area=yes.  That's quick and easy to fix.

Recently I came across 42 highways left untagged by the same mapper in the same area.  The mapper had more than 2,000 edits so wasn't inexperienced.

Crossing highways often I'll see a town where the highway residentials are laid out in a grid but the junctions aren't nodes.

Another nice recent one was a batch of settlements that had been tagged highway=unclassified rather than landuse=residential.  Searching for highways above unclassified and crossing ways picked those out quite quickly.

These types of errors are handled more quickly directly in JOSM than involving maproulette.

Even the some other errors can be loaded up in the JOSM  to-do plugin.

I've been cleaning up in Nigeria for a while now.  Making repeated passes over the same areas over time.  I'm seeing fewer errors so my suspicion is the improved learnOSM, tutorials and simplifying what we ask mappers to map is paying off.  Validating new mappers work within 48 hours does pay off in data quality terms.  Giving feedback more than two weeks later is usually a waste of time based on my own experience and Martin's report but I don't think we have the validators to do the validation within the magic 24/48 hours.

Broken multipolyons do take a level of expertise to resolve.  I usually avoid them.  However new mappers will fearlessly tread ahead and it is to be expected they will break some.

I don't think learnOSM goes into detail about them or about relationships.  Should they?  That's another question.

For some errors we need boots on the ground.  Especially when it comes to urban areas with limited access highways such as dual carriageways​ and a highway approaching them can you turn in either direction?

Often the most motivated people to fix the errors are the locals who will be using the map.  Trouble is we first have to get something in place that is useful to get them motivated.

So yes HOT maperthons do have a reputation of the blind leading the blind, ie some are led by inexperienced mappers but we are putting effort into reducing the number of errors.  The training group has put together a very good detailed set of instructions on how best to run a maperthon and if you can identify a problem area HOT will listen and try to address the issues.

Cheerio John

On 16 Mar 2017 3:49 pm, "Jochen Topf" <[hidden email]> wrote:
Hi Blake,

On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> Can you send me something that would help me see those 1600 examples
> you found on OSM so I can see what might be going on and how to
> improve things in HOT/MM workflow to reduce them?

Those 1600 are some very specific errors I am currently looking at, but
only have those on my local machine currently. But for the wider picture
you can use the OSM Inspectors "area" view to see the scope of the problem.

Just go to any place where HOT/MM was active. Here is an example:
http://tools.geofabrik.de/osmi/?view=areas&lon=33.65927&lat=-2.07600&zoom=8

My current workflow is as follows: I choose one particular well-defined
problem and extract the data from OSM that shows this problem. Then I
put the data into Maproulette and document the problem and how to fix
it. Some problems are easier, some harder to fix. Mappers can choose
what problems to work on based on their skill level. If somebody
doesn't feel comfortable with one of the harder problems, there are
simpler ones they can work on. I also have some more general
descriptions for the experts who want to go exploring/fixing on their
own. Once the challenges are done, I choose a new problem and start the
process from the beginning.

Here are the current challenges: http://area.jochentopf.com/fixing.html

Usually I am splitting up larger challenges into regions, for instance
by continent or country. I could also do this by HOT activation area or
so if that makes sense for HOT mappers. Splitting up, often makes the
problems easier to fix, because problems inside an area are often very
similar, but different to problems in other areas due to mapping
priorities and methods. For instance, some regions have lots of problems
with buildings, others with landuse areas, etc. Concentrating on one
very well-defined problem at a time makes all this work simpler.

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

pierzen



On 16 Mar 2017 3:49 pm, "Jochen Topf" <[hidden email]> wrote:
Hi Blake,

On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> Can you send me something that would help me see those 1600 examples
> you found on OSM so I can see what might be going on and how to
> improve things in HOT/MM workflow to reduce them?

Those 1600 are some very specific errors I am currently looking at, but
only have those on my local machine currently. But for the wider picture
you can use the OSM Inspectors "area" view to see the scope of the problem.

Just go to any place where HOT/MM was active. Here is an example:
http://tools.geofabrik.de/ osmi/?view=areas&lon=33.65927& lat=-2.07600&zoom=8

My current workflow is as follows: I choose one particular well-defined
problem and extract the data from OSM that shows this problem. Then I
put the data into Maproulette and document the problem and how to fix
it. Some problems are easier, some harder to fix. Mappers can choose
what problems to work on based on their skill level. If somebody
doesn't feel comfortable with one of the harder problems, there are
simpler ones they can work on. I also have some more general
descriptions for the experts who want to go exploring/fixing on their
own. Once the challenges are done, I choose a new problem and start the
process from the beginning.

Here are the current challenges: http://area.jochentopf.com/ fixing.html

Usually I am splitting up larger challenges into regions, for instance
by continent or country. I could also do this by HOT activation area or
so if that makes sense for HOT mappers. Splitting up, often makes the
problems easier to fix, because problems inside an area are often very
similar, but different to problems in other areas due to mapping
priorities and methods. For instance, some regions have lots of problems
with buildings, others with landuse areas, etc. Concentrating on one
very well-defined problem at a time makes all this work simpler.

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

______________________________ _________________
HOT mailing list
[hidden email]
https://lists.openstreetmap. org/listinfo/hot

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot



_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
In reply to this post by john whelan-2
Hi!

I am sure there are many other problems, but can we please keep to the
issue at hand in this thread? I am not trying to solve all problems, I
am trying to solve one particular set of problem related to
(multi)polygons.

And I didn't pick that particular problem at random. This is something
that affects everybody. The short of it is: Due to upcoming software
changes, some (multi)polygons that are rendered now will disappear from
the map or look different in the future. Again, you can find all the
details about the problems and of what we are trying to achieve at
http://area.jochentopf.com/ .

We are having great success working on these problems. You can see the
stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
approach will be the best one for other problems, and it doesn't mean we
can't do even better than we are doing until now with this problem.

Btw: I am deliberately talking about "(multi)polygons". This is not
something that only affects multipolygon relations, but all polygons,
even the ones created from closed ways. They are simpler than mp
relations, but that doesn't mean they can't have problems.

Jochen

On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:

> Date: Thu, 16 Mar 2017 17:23:47 -0400
> From: john whelan <[hidden email]>
> To: Jochen Topf <[hidden email]>
> Cc: [hidden email], Blake Girardot HOT/OSM
>  <[hidden email]>
> Subject: Re: [HOT] Broken (multi)polygon cleanup
>
> This is similar to my pulling in Africa a country at a time and using JOSM
> to check for errors.
>
> What I have noticed is one mapper will misstag then others will follow.  So
> sanitising the area or checking it for errors before the new HOT mappers
> start would probably improve the overall data quality as they tend to
> follow what is already mapped.
>
> Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings in
> one small area by the same mapper left as area=yes.  That's quick and easy
> to fix.
>
> Recently I came across 42 highways left untagged by the same mapper in the
> same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
>
> Crossing highways often I'll see a town where the highway residentials are
> laid out in a grid but the junctions aren't nodes.
>
> Another nice recent one was a batch of settlements that had been tagged
> highway=unclassified rather than landuse=residential.  Searching for
> highways above unclassified and crossing ways picked those out quite
> quickly.
>
> These types of errors are handled more quickly directly in JOSM than
> involving maproulette.
>
> Even the some other errors can be loaded up in the JOSM  to-do plugin.
>
> I've been cleaning up in Nigeria for a while now.  Making repeated passes
> over the same areas over time.  I'm seeing fewer errors so my suspicion is
> the improved learnOSM, tutorials and simplifying what we ask mappers to map
> is paying off.  Validating new mappers work within 48 hours does pay off in
> data quality terms.  Giving feedback more than two weeks later is usually a
> waste of time based on my own experience and Martin's report but I don't
> think we have the validators to do the validation within the magic 24/48
> hours.
>
> Broken multipolyons do take a level of expertise to resolve.  I usually
> avoid them.  However new mappers will fearlessly tread ahead and it is to
> be expected they will break some.
>
> I don't think learnOSM goes into detail about them or about relationships.
> Should they?  That's another question.
>
> For some errors we need boots on the ground.  Especially when it comes to
> urban areas with limited access highways such as dual carriageways​ and a
> highway approaching them can you turn in either direction?
>
> Often the most motivated people to fix the errors are the locals who will
> be using the map.  Trouble is we first have to get something in place that
> is useful to get them motivated.
>
> So yes HOT maperthons do have a reputation of the blind leading the blind,
> ie some are led by inexperienced mappers but we are putting effort into
> reducing the number of errors.  The training group has put together a very
> good detailed set of instructions on how best to run a maperthon and if you
> can identify a problem area HOT will listen and try to address the issues.
>
> Cheerio John
>
> On 16 Mar 2017 3:49 pm, "Jochen Topf" <[hidden email]> wrote:
>
> > Hi Blake,
> >
> > On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> > > Can you send me something that would help me see those 1600 examples
> > > you found on OSM so I can see what might be going on and how to
> > > improve things in HOT/MM workflow to reduce them?
> >
> > Those 1600 are some very specific errors I am currently looking at, but
> > only have those on my local machine currently. But for the wider picture
> > you can use the OSM Inspectors "area" view to see the scope of the problem.
> >
> > Just go to any place where HOT/MM was active. Here is an example:
> > http://tools.geofabrik.de/osmi/?view=areas&lon=33.65927&
> > lat=-2.07600&zoom=8
> >
> > My current workflow is as follows: I choose one particular well-defined
> > problem and extract the data from OSM that shows this problem. Then I
> > put the data into Maproulette and document the problem and how to fix
> > it. Some problems are easier, some harder to fix. Mappers can choose
> > what problems to work on based on their skill level. If somebody
> > doesn't feel comfortable with one of the harder problems, there are
> > simpler ones they can work on. I also have some more general
> > descriptions for the experts who want to go exploring/fixing on their
> > own. Once the challenges are done, I choose a new problem and start the
> > process from the beginning.
> >
> > Here are the current challenges: http://area.jochentopf.com/fixing.html
> >
> > Usually I am splitting up larger challenges into regions, for instance
> > by continent or country. I could also do this by HOT activation area or
> > so if that makes sense for HOT mappers. Splitting up, often makes the
> > problems easier to fix, because problems inside an area are often very
> > similar, but different to problems in other areas due to mapping
> > priorities and methods. For instance, some regions have lots of problems
> > with buildings, others with landuse areas, etc. Concentrating on one
> > very well-defined problem at a time makes all this work simpler.
> >
> > Jochen
> > --
> > Jochen Topf  [hidden email]  https://www.jochentopf.com/
> > +49-351-31778688
> >
> > _______________________________________________
> > HOT mailing list
> > [hidden email]
> > https://lists.openstreetmap.org/listinfo/hot
> >

--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

john whelan-2
What isn't clear to me is just what you are trying to fix.  The term means nothing to me.  I am correcting duplicate segments and unclosed ways which appear to be part of your problem as I work.  Perhaps someone can explain what you are after in simpler terms because if we don't understand the problem then trying to come up with ways to avoid it in the future isn't going to happen.

Thanks John

On 17 March 2017 at 08:16, Jochen Topf <[hidden email]> wrote:
Hi!

I am sure there are many other problems, but can we please keep to the
issue at hand in this thread? I am not trying to solve all problems, I
am trying to solve one particular set of problem related to
(multi)polygons.

And I didn't pick that particular problem at random. This is something
that affects everybody. The short of it is: Due to upcoming software
changes, some (multi)polygons that are rendered now will disappear from
the map or look different in the future. Again, you can find all the
details about the problems and of what we are trying to achieve at
http://area.jochentopf.com/ .

We are having great success working on these problems. You can see the
stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
approach will be the best one for other problems, and it doesn't mean we
can't do even better than we are doing until now with this problem.

Btw: I am deliberately talking about "(multi)polygons". This is not
something that only affects multipolygon relations, but all polygons,
even the ones created from closed ways. They are simpler than mp
relations, but that doesn't mean they can't have problems.

Jochen

On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:
> Date: Thu, 16 Mar 2017 17:23:47 -0400
> From: john whelan <[hidden email]>
> To: Jochen Topf <[hidden email]>
> Cc: [hidden email], Blake Girardot HOT/OSM
>  <[hidden email]>
> Subject: Re: [HOT] Broken (multi)polygon cleanup
>
> This is similar to my pulling in Africa a country at a time and using JOSM
> to check for errors.
>
> What I have noticed is one mapper will misstag then others will follow.  So
> sanitising the area or checking it for errors before the new HOT mappers
> start would probably improve the overall data quality as they tend to
> follow what is already mapped.
>
> Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings in
> one small area by the same mapper left as area=yes.  That's quick and easy
> to fix.
>
> Recently I came across 42 highways left untagged by the same mapper in the
> same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
>
> Crossing highways often I'll see a town where the highway residentials are
> laid out in a grid but the junctions aren't nodes.
>
> Another nice recent one was a batch of settlements that had been tagged
> highway=unclassified rather than landuse=residential.  Searching for
> highways above unclassified and crossing ways picked those out quite
> quickly.
>
> These types of errors are handled more quickly directly in JOSM than
> involving maproulette.
>
> Even the some other errors can be loaded up in the JOSM  to-do plugin.
>
> I've been cleaning up in Nigeria for a while now.  Making repeated passes
> over the same areas over time.  I'm seeing fewer errors so my suspicion is
> the improved learnOSM, tutorials and simplifying what we ask mappers to map
> is paying off.  Validating new mappers work within 48 hours does pay off in
> data quality terms.  Giving feedback more than two weeks later is usually a
> waste of time based on my own experience and Martin's report but I don't
> think we have the validators to do the validation within the magic 24/48
> hours.
>
> Broken multipolyons do take a level of expertise to resolve.  I usually
> avoid them.  However new mappers will fearlessly tread ahead and it is to
> be expected they will break some.
>
> I don't think learnOSM goes into detail about them or about relationships.
> Should they?  That's another question.
>
> For some errors we need boots on the ground.  Especially when it comes to
> urban areas with limited access highways such as dual carriageways​ and a
> highway approaching them can you turn in either direction?
>
> Often the most motivated people to fix the errors are the locals who will
> be using the map.  Trouble is we first have to get something in place that
> is useful to get them motivated.
>
> So yes HOT maperthons do have a reputation of the blind leading the blind,
> ie some are led by inexperienced mappers but we are putting effort into
> reducing the number of errors.  The training group has put together a very
> good detailed set of instructions on how best to run a maperthon and if you
> can identify a problem area HOT will listen and try to address the issues.
>
> Cheerio John
>
> On 16 Mar 2017 3:49 pm, "Jochen Topf" <[hidden email]> wrote:
>
> > Hi Blake,
> >
> > On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> > > Can you send me something that would help me see those 1600 examples
> > > you found on OSM so I can see what might be going on and how to
> > > improve things in HOT/MM workflow to reduce them?
> >
> > Those 1600 are some very specific errors I am currently looking at, but
> > only have those on my local machine currently. But for the wider picture
> > you can use the OSM Inspectors "area" view to see the scope of the problem.
> >
> > Just go to any place where HOT/MM was active. Here is an example:
> > http://tools.geofabrik.de/osmi/?view=areas&lon=33.65927&
> > lat=-2.07600&zoom=8
> >
> > My current workflow is as follows: I choose one particular well-defined
> > problem and extract the data from OSM that shows this problem. Then I
> > put the data into Maproulette and document the problem and how to fix
> > it. Some problems are easier, some harder to fix. Mappers can choose
> > what problems to work on based on their skill level. If somebody
> > doesn't feel comfortable with one of the harder problems, there are
> > simpler ones they can work on. I also have some more general
> > descriptions for the experts who want to go exploring/fixing on their
> > own. Once the challenges are done, I choose a new problem and start the
> > process from the beginning.
> >
> > Here are the current challenges: http://area.jochentopf.com/fixing.html
> >
> > Usually I am splitting up larger challenges into regions, for instance
> > by continent or country. I could also do this by HOT activation area or
> > so if that makes sense for HOT mappers. Splitting up, often makes the
> > problems easier to fix, because problems inside an area are often very
> > similar, but different to problems in other areas due to mapping
> > priorities and methods. For instance, some regions have lots of problems
> > with buildings, others with landuse areas, etc. Concentrating on one
> > very well-defined problem at a time makes all this work simpler.
> >
> > Jochen
> > --
> > Jochen Topf  [hidden email]  https://www.jochentopf.com/
> > <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688
> >
> > _______________________________________________
> > HOT mailing list
> > [hidden email]
> > https://lists.openstreetmap.org/listinfo/hot
> >

--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
Hi!

did you read those documents linked from http://area.jochentopf.com/ ?

https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/background.md
https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/problems.md
https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/faq.md

If there is something unclear after reading them, please ask, and I am
happy to explain.

Jochen

On Fri, Mar 17, 2017 at 08:30:27AM -0400, john whelan wrote:

> Date: Fri, 17 Mar 2017 08:30:27 -0400
> From: john whelan <[hidden email]>
> To: Jochen Topf <[hidden email]>
> Cc: "[hidden email]" <[hidden email]>, Blake Girardot HOT/OSM
>  <[hidden email]>
> Subject: Re: [HOT] Broken (multi)polygon cleanup
>
> What isn't clear to me is just what you are trying to fix.  The term means
> nothing to me.  I am correcting duplicate segments and unclosed ways which
> appear to be part of your problem as I work.  Perhaps someone can explain
> what you are after in simpler terms because if we don't understand the
> problem then trying to come up with ways to avoid it in the future isn't
> going to happen.
>
> Thanks John
>
> On 17 March 2017 at 08:16, Jochen Topf <[hidden email]> wrote:
>
> > Hi!
> >
> > I am sure there are many other problems, but can we please keep to the
> > issue at hand in this thread? I am not trying to solve all problems, I
> > am trying to solve one particular set of problem related to
> > (multi)polygons.
> >
> > And I didn't pick that particular problem at random. This is something
> > that affects everybody. The short of it is: Due to upcoming software
> > changes, some (multi)polygons that are rendered now will disappear from
> > the map or look different in the future. Again, you can find all the
> > details about the problems and of what we are trying to achieve at
> > http://area.jochentopf.com/ .
> >
> > We are having great success working on these problems. You can see the
> > stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
> > approach will be the best one for other problems, and it doesn't mean we
> > can't do even better than we are doing until now with this problem.
> >
> > Btw: I am deliberately talking about "(multi)polygons". This is not
> > something that only affects multipolygon relations, but all polygons,
> > even the ones created from closed ways. They are simpler than mp
> > relations, but that doesn't mean they can't have problems.
> >
> > Jochen
> >
> > On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:
> > > Date: Thu, 16 Mar 2017 17:23:47 -0400
> > > From: john whelan <[hidden email]>
> > > To: Jochen Topf <[hidden email]>
> > > Cc: [hidden email], Blake Girardot HOT/OSM
> > >  <[hidden email]>
> > > Subject: Re: [HOT] Broken (multi)polygon cleanup
> > >
> > > This is similar to my pulling in Africa a country at a time and using
> > JOSM
> > > to check for errors.
> > >
> > > What I have noticed is one mapper will misstag then others will follow.
> > So
> > > sanitising the area or checking it for errors before the new HOT mappers
> > > start would probably improve the overall data quality as they tend to
> > > follow what is already mapped.
> > >
> > > Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings
> > in
> > > one small area by the same mapper left as area=yes.  That's quick and
> > easy
> > > to fix.
> > >
> > > Recently I came across 42 highways left untagged by the same mapper in
> > the
> > > same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
> > >
> > > Crossing highways often I'll see a town where the highway residentials
> > are
> > > laid out in a grid but the junctions aren't nodes.
> > >
> > > Another nice recent one was a batch of settlements that had been tagged
> > > highway=unclassified rather than landuse=residential.  Searching for
> > > highways above unclassified and crossing ways picked those out quite
> > > quickly.
> > >
> > > These types of errors are handled more quickly directly in JOSM than
> > > involving maproulette.
> > >
> > > Even the some other errors can be loaded up in the JOSM  to-do plugin.
> > >
> > > I've been cleaning up in Nigeria for a while now.  Making repeated passes
> > > over the same areas over time.  I'm seeing fewer errors so my suspicion
> > is
> > > the improved learnOSM, tutorials and simplifying what we ask mappers to
> > map
> > > is paying off.  Validating new mappers work within 48 hours does pay off
> > in
> > > data quality terms.  Giving feedback more than two weeks later is
> > usually a
> > > waste of time based on my own experience and Martin's report but I don't
> > > think we have the validators to do the validation within the magic 24/48
> > > hours.
> > >
> > > Broken multipolyons do take a level of expertise to resolve.  I usually
> > > avoid them.  However new mappers will fearlessly tread ahead and it is to
> > > be expected they will break some.
> > >
> > > I don't think learnOSM goes into detail about them or about
> > relationships.
> > > Should they?  That's another question.
> > >
> > > For some errors we need boots on the ground.  Especially when it comes to
> > > urban areas with limited access highways such as dual carriageways​ and a
> > > highway approaching them can you turn in either direction?
> > >
> > > Often the most motivated people to fix the errors are the locals who will
> > > be using the map.  Trouble is we first have to get something in place
> > that
> > > is useful to get them motivated.
> > >
> > > So yes HOT maperthons do have a reputation of the blind leading the
> > blind,
> > > ie some are led by inexperienced mappers but we are putting effort into
> > > reducing the number of errors.  The training group has put together a
> > very
> > > good detailed set of instructions on how best to run a maperthon and if
> > you
> > > can identify a problem area HOT will listen and try to address the
> > issues.
> > >
> > > Cheerio John
> > >
> > > On 16 Mar 2017 3:49 pm, "Jochen Topf" <[hidden email]> wrote:
> > >
> > > > Hi Blake,
> > > >
> > > > On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> > > > > Can you send me something that would help me see those 1600 examples
> > > > > you found on OSM so I can see what might be going on and how to
> > > > > improve things in HOT/MM workflow to reduce them?
> > > >
> > > > Those 1600 are some very specific errors I am currently looking at, but
> > > > only have those on my local machine currently. But for the wider
> > picture
> > > > you can use the OSM Inspectors "area" view to see the scope of the
> > problem.
> > > >
> > > > Just go to any place where HOT/MM was active. Here is an example:
> > > > http://tools.geofabrik.de/osmi/?view=areas&lon=33.65927&
> > > > lat=-2.07600&zoom=8
> > > >
> > > > My current workflow is as follows: I choose one particular well-defined
> > > > problem and extract the data from OSM that shows this problem. Then I
> > > > put the data into Maproulette and document the problem and how to fix
> > > > it. Some problems are easier, some harder to fix. Mappers can choose
> > > > what problems to work on based on their skill level. If somebody
> > > > doesn't feel comfortable with one of the harder problems, there are
> > > > simpler ones they can work on. I also have some more general
> > > > descriptions for the experts who want to go exploring/fixing on their
> > > > own. Once the challenges are done, I choose a new problem and start the
> > > > process from the beginning.
> > > >
> > > > Here are the current challenges: http://area.jochentopf.com/
> > fixing.html
> > > >
> > > > Usually I am splitting up larger challenges into regions, for instance
> > > > by continent or country. I could also do this by HOT activation area or
> > > > so if that makes sense for HOT mappers. Splitting up, often makes the
> > > > problems easier to fix, because problems inside an area are often very
> > > > similar, but different to problems in other areas due to mapping
> > > > priorities and methods. For instance, some regions have lots of
> > problems
> > > > with buildings, others with landuse areas, etc. Concentrating on one
> > > > very well-defined problem at a time makes all this work simpler.
> > > >
> > > > Jochen
> > > > --
> > > > Jochen Topf  [hidden email]  https://www.jochentopf.com/
> > > > +49-351-31778688
> > > >
> > > > _______________________________________________
> > > > HOT mailing list
> > > > [hidden email]
> > > > https://lists.openstreetmap.org/listinfo/hot
> > > >
> >
> > --
> > Jochen Topf  [hidden email]  https://www.jochentopf.com/
> > +49-351-31778688
> >

--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

john whelan-2
I've read it through but remember like many mappers I do not have a Ph.D. in GIS, just a degree in Chemistry so I'm usually not too bad on logic.

The first thing that stands out is the message is too complex for the intended audience.

The second "The new style tagging option is the recommended tagging option these days, but some mappers still disagree."  So we don't have complete agreement, fine.

The third is the main problem areas seem to be some rendering systems prefer one method over the other.  "Are we mapping for a specific rendering system now?"

I get the impression that the main problem is areas of landuse or defining countries.  The defining country side I leave to specialists.  Landuse=forest type things are often out of date and problematical.

At a basic level I think we are talking using JOSM to do this work.  I hate to say it but despite my efforts JOSM is still not main stream for HOT mapathons.  So essentially we will confuse the audience and confused HOT mappers may lead to other problems.

I suspect if we concentrate on unclosed ways and duplicate segments then this is something that can be done without a Ph.D. in GIS concepts and there is no disagreement.  Leave the more complex problem solving to others.

I have yet to see a HOT project that asks mappers to map country outlines. I have seen some that ask for landuse=residential but that is about as complicated as it normally gets.  There might be one or two odd ones that ask for landuse but these days they are rare.  The bulk of the mapping is buildings and highways with the occasional river thrown in and I don't think these are problem areas to you.

There have been some imports in the HOT areas of questionable data quality some are fine but that is a different matter to HOT mapping.

I would recommend if you wish to use the resources of HOT that you talk nicely to the HOT training group and see if they can sort out the message and what training needs to be given to support these efforts including what you would like mappers to avoid when mapping.

Thanks

Cheerio John



On 17 March 2017 at 09:43, Jochen Topf <[hidden email]> wrote:
Hi!

did you read those documents linked from http://area.jochentopf.com/ ?

https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/background.md
https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/problems.md
https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/faq.md

If there is something unclear after reading them, please ask, and I am
happy to explain.

Jochen

On Fri, Mar 17, 2017 at 08:30:27AM -0400, john whelan wrote:
> Date: Fri, 17 Mar 2017 08:30:27 -0400
> From: john whelan <[hidden email]>
> To: Jochen Topf <[hidden email]>
> Cc: "[hidden email]" <[hidden email]>, Blake Girardot HOT/OSM
>  <[hidden email]>
> Subject: Re: [HOT] Broken (multi)polygon cleanup
>
> What isn't clear to me is just what you are trying to fix.  The term means
> nothing to me.  I am correcting duplicate segments and unclosed ways which
> appear to be part of your problem as I work.  Perhaps someone can explain
> what you are after in simpler terms because if we don't understand the
> problem then trying to come up with ways to avoid it in the future isn't
> going to happen.
>
> Thanks John
>
> On 17 March 2017 at 08:16, Jochen Topf <[hidden email]> wrote:
>
> > Hi!
> >
> > I am sure there are many other problems, but can we please keep to the
> > issue at hand in this thread? I am not trying to solve all problems, I
> > am trying to solve one particular set of problem related to
> > (multi)polygons.
> >
> > And I didn't pick that particular problem at random. This is something
> > that affects everybody. The short of it is: Due to upcoming software
> > changes, some (multi)polygons that are rendered now will disappear from
> > the map or look different in the future. Again, you can find all the
> > details about the problems and of what we are trying to achieve at
> > http://area.jochentopf.com/ .
> >
> > We are having great success working on these problems. You can see the
> > stats at http://area.jochentopf.com/stats/ . This doesn't mean that the
> > approach will be the best one for other problems, and it doesn't mean we
> > can't do even better than we are doing until now with this problem.
> >
> > Btw: I am deliberately talking about "(multi)polygons". This is not
> > something that only affects multipolygon relations, but all polygons,
> > even the ones created from closed ways. They are simpler than mp
> > relations, but that doesn't mean they can't have problems.
> >
> > Jochen
> >
> > On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote:
> > > Date: Thu, 16 Mar 2017 17:23:47 -0400
> > > From: john whelan <[hidden email]>
> > > To: Jochen Topf <[hidden email]>
> > > Cc: [hidden email], Blake Girardot HOT/OSM
> > >  <[hidden email]>
> > > Subject: Re: [HOT] Broken (multi)polygon cleanup
> > >
> > > This is similar to my pulling in Africa a country at a time and using
> > JOSM
> > > to check for errors.
> > >
> > > What I have noticed is one mapper will misstag then others will follow.
> > So
> > > sanitising the area or checking it for errors before the new HOT mappers
> > > start would probably improve the overall data quality as they tend to
> > > follow what is already mapped.
> > >
> > > Untagged ways by one mapper tend to cluster.  I've seen a 100 buildings
> > in
> > > one small area by the same mapper left as area=yes.  That's quick and
> > easy
> > > to fix.
> > >
> > > Recently I came across 42 highways left untagged by the same mapper in
> > the
> > > same area.  The mapper had more than 2,000 edits so wasn't inexperienced.
> > >
> > > Crossing highways often I'll see a town where the highway residentials
> > are
> > > laid out in a grid but the junctions aren't nodes.
> > >
> > > Another nice recent one was a batch of settlements that had been tagged
> > > highway=unclassified rather than landuse=residential.  Searching for
> > > highways above unclassified and crossing ways picked those out quite
> > > quickly.
> > >
> > > These types of errors are handled more quickly directly in JOSM than
> > > involving maproulette.
> > >
> > > Even the some other errors can be loaded up in the JOSM  to-do plugin.
> > >
> > > I've been cleaning up in Nigeria for a while now.  Making repeated passes
> > > over the same areas over time.  I'm seeing fewer errors so my suspicion
> > is
> > > the improved learnOSM, tutorials and simplifying what we ask mappers to
> > map
> > > is paying off.  Validating new mappers work within 48 hours does pay off
> > in
> > > data quality terms.  Giving feedback more than two weeks later is
> > usually a
> > > waste of time based on my own experience and Martin's report but I don't
> > > think we have the validators to do the validation within the magic 24/48
> > > hours.
> > >
> > > Broken multipolyons do take a level of expertise to resolve.  I usually
> > > avoid them.  However new mappers will fearlessly tread ahead and it is to
> > > be expected they will break some.
> > >
> > > I don't think learnOSM goes into detail about them or about
> > relationships.
> > > Should they?  That's another question.
> > >
> > > For some errors we need boots on the ground.  Especially when it comes to
> > > urban areas with limited access highways such as dual carriageways​ and a
> > > highway approaching them can you turn in either direction?
> > >
> > > Often the most motivated people to fix the errors are the locals who will
> > > be using the map.  Trouble is we first have to get something in place
> > that
> > > is useful to get them motivated.
> > >
> > > So yes HOT maperthons do have a reputation of the blind leading the
> > blind,
> > > ie some are led by inexperienced mappers but we are putting effort into
> > > reducing the number of errors.  The training group has put together a
> > very
> > > good detailed set of instructions on how best to run a maperthon and if
> > you
> > > can identify a problem area HOT will listen and try to address the
> > issues.
> > >
> > > Cheerio John
> > >
> > > On 16 Mar 2017 3:49 pm, "Jochen Topf" <[hidden email]> wrote:
> > >
> > > > Hi Blake,
> > > >
> > > > On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote:
> > > > > Can you send me something that would help me see those 1600 examples
> > > > > you found on OSM so I can see what might be going on and how to
> > > > > improve things in HOT/MM workflow to reduce them?
> > > >
> > > > Those 1600 are some very specific errors I am currently looking at, but
> > > > only have those on my local machine currently. But for the wider
> > picture
> > > > you can use the OSM Inspectors "area" view to see the scope of the
> > problem.
> > > >
> > > > Just go to any place where HOT/MM was active. Here is an example:
> > > > http://tools.geofabrik.de/osmi/?view=areas&lon=33.65927&
> > > > lat=-2.07600&zoom=8
> > > >
> > > > My current workflow is as follows: I choose one particular well-defined
> > > > problem and extract the data from OSM that shows this problem. Then I
> > > > put the data into Maproulette and document the problem and how to fix
> > > > it. Some problems are easier, some harder to fix. Mappers can choose
> > > > what problems to work on based on their skill level. If somebody
> > > > doesn't feel comfortable with one of the harder problems, there are
> > > > simpler ones they can work on. I also have some more general
> > > > descriptions for the experts who want to go exploring/fixing on their
> > > > own. Once the challenges are done, I choose a new problem and start the
> > > > process from the beginning.
> > > >
> > > > Here are the current challenges: http://area.jochentopf.com/
> > fixing.html
> > > >
> > > > Usually I am splitting up larger challenges into regions, for instance
> > > > by continent or country. I could also do this by HOT activation area or
> > > > so if that makes sense for HOT mappers. Splitting up, often makes the
> > > > problems easier to fix, because problems inside an area are often very
> > > > similar, but different to problems in other areas due to mapping
> > > > priorities and methods. For instance, some regions have lots of
> > problems
> > > > with buildings, others with landuse areas, etc. Concentrating on one
> > > > very well-defined problem at a time makes all this work simpler.
> > > >
> > > > Jochen
> > > > --
> > > > Jochen Topf  [hidden email]  https://www.jochentopf.com/
> > > > <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688
> > > >
> > > > _______________________________________________
> > > > HOT mailing list
> > > > [hidden email]
> > > > https://lists.openstreetmap.org/listinfo/hot
> > > >
> >
> > --
> > Jochen Topf  [hidden email]  https://www.jochentopf.com/
> > <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688
> >

--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

pierzen
In reply to this post by Jochen123
Hi Jochen

Your focus on these problems more related to the softwares process is quite interesting and valuable.

While I continue to focus on the Haiti south west peninsula after the Matthew hurricane landing in october 2016I looked today at multipolygon relations misuse.

For Buildings with only outer rings, I merged the ways if necessary, removed the relation and added the building tag to the ways.
This Overpass query shows some of these multipolygons before correction.

I also found and incorrect use is multipolygon relation as a collection of buildings, removed the relation and corrected the tagging. There were 7 buildings in this relation. This does not respect the OSM data model and we cannot easily count the number of building in GIS softwares. The mapper who added this relation has more then 600 days of contribution and hence classified by Pascal Neis as Great Mapper (Very Active).

The next problem I will look at is ways with only the area tag.


Pierre







_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
In reply to this post by john whelan-2
On Fri, Mar 17, 2017 at 10:48:00AM -0400, john whelan wrote:
> I've read it through but remember like many mappers I do not have a Ph.D.
> in GIS, just a degree in Chemistry so I'm usually not too bad on logic.
>
> The first thing that stands out is the message is too complex for the
> intended audience.

Yes, and no. The documentation we have now is trying to be complete and
thorough. The intended audience are, first, power mappers. I am totally
aware that it is too much for the average mapper, but that's why I need
everybodies help not only the fix things, but also to translate the
problem into words that the different audiences can understand.

I know that HOT has capable mappers that understand OSM well and have
written documentation for new HOT mappers to use. I hope I can reach
those people here so they can help.

> The second "The *new style* tagging option is the recommended tagging
> option these days, but some mappers still disagree."  So we don't have
> complete agreement, fine.

Well, that was me being overcautious. I haven't heard any disagreements
in the last months. So I don't think there is any major disagreements
left. I have removed that sentence from the documentation.

> The third is the main problem areas seem to be some rendering systems
> prefer one method over the other.  "Are we mapping for a specific rendering
> system now?"

No, we are not. I am not sure why you got that impression from the
documentation. Can you point me to a specific sentence? I have
highlighted Osm2pgsql, because it is by far the most often used basis
for rendering maps and it is what powers the main OSM map. So changes
there will affect more people. And it also means that basically every
other system is trying to mimic what Osm2pgsql is doing (right or
wrong), because they have to keep "in sync" with the main map that OSM
mappers use to verify their work.

Currently all rendering systems have to handle old- and new-style
multipolygons and in the future they wont have to do that any more. This
makes all systems simpler.

> I suspect if we concentrate on unclosed ways and duplicate segments then
> this is something that can be done without a Ph.D. in GIS concepts and
> there is no disagreement.  Leave the more complex problem solving to others.

I totally agree. That's why I have been splitting up the work into small
Maproulette tasks, each with a very easy to understand and concrete
description of what the fixing task is. I don't expect most mappers to
be able to solve complex multipolygon cases. But there are plenty of
simple cases around.

> I would recommend if you wish to use the resources of HOT that you talk
> nicely to the HOT training group and see if they can sort out the message
> and what training needs to be given to support these efforts including what
> you would like mappers to avoid when mapping.

I am hoping these people are listening here. (Are you?) Or do I have to
go somewhere else to talk to them?

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Michael Heißmeier-2
Hi all,

Jochen Topf, 2017-03-18 11:20:
On Fri, Mar 17, 2017 at 10:48:00AM -0400, john whelan wrote:
I've read it through but remember like many mappers I do not have a Ph.D.
in GIS, just a degree in Chemistry so I'm usually not too bad on logic.

The first thing that stands out is the message is too complex for the
intended audience.
Yes, and no. The documentation we have now is trying to be complete and
thorough. The intended audience are, first, power mappers. I am totally
aware that it is too much for the average mapper, but that's why I need
everybodies help not only the fix things, but also to translate the
problem into words that the different audiences can understand.

I know that HOT has capable mappers that understand OSM well and have
written documentation for new HOT mappers to use. I hope I can reach
those people here so they can help.

The second "The *new style* tagging option is the recommended tagging
option these days, but some mappers still disagree."  So we don't have
complete agreement, fine.
Well, that was me being overcautious. I haven't heard any disagreements
in the last months. So I don't think there is any major disagreements
left. I have removed that sentence from the documentation.

The third is the main problem areas seem to be some rendering systems
prefer one method over the other.  "Are we mapping for a specific rendering
system now?"
No, we are not. I am not sure why you got that impression from the
documentation. Can you point me to a specific sentence? I have
highlighted Osm2pgsql, because it is by far the most often used basis
for rendering maps and it is what powers the main OSM map. So changes
there will affect more people. And it also means that basically every
other system is trying to mimic what Osm2pgsql is doing (right or
wrong), because they have to keep "in sync" with the main map that OSM
mappers use to verify their work.

Currently all rendering systems have to handle old- and new-style
multipolygons and in the future they wont have to do that any more. This
makes all systems simpler.

I suspect if we concentrate on unclosed ways and duplicate segments then
this is something that can be done without a Ph.D. in GIS concepts and
there is no disagreement.  Leave the more complex problem solving to others.
I totally agree. That's why I have been splitting up the work into small
Maproulette tasks, each with a very easy to understand and concrete
description of what the fixing task is. I don't expect most mappers to
be able to solve complex multipolygon cases. But there are plenty of
simple cases around.

I gave the Maproulette tasks a try yesterday evening and found them easy to tackle. The advantage here is that they are typically localized to a small area, you do not have to take lots of data into account.

For the bigger picture, OSM Inspector is the tool of choice but I agree that many errors reported here are not easy to fix.
I will try to get an overview of errors made by relatively new contributors and see how we can improve documentation and learning materials. Errors made are not always what you think of when writing documentation, cf. this "single" building which contains self-intersections:



Not to forget the biggest multipolygon of all (although not technically organized as one), the coastline, which is more often broken than not.


I would recommend if you wish to use the resources of HOT that you talk
nicely to the HOT training group and see if they can sort out the message
and what training needs to be given to support these efforts including what
you would like mappers to avoid when mapping.
I am hoping these people are listening here. (Are you?) Or do I have to
go somewhere else to talk to them?

Yes, people active in the Training Working Group are listening here as well. We are aware that new mappers need guidance and made substantial improvements over the past months. Out chair, Nick Allen, mentioned this last month in his e-mail to the hot list (http://lists.openstreetmap.org/pipermail/hot/2017-February/012977.html). We are always open to suggestions for further material to be included there.

Best Regards

Michael
(osm:michael63)


PS: John,
I have yet to see a HOT project that asks mappers to map country outlines.
I agree that these are rare but they do exist (http://tasks.hotosm.org/project/1407)



_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Coast Limit
I know this discussion is focused on cleaning up these issues, but — as a relatively new mapper — can I make a suggestion on how we might better limit mistakes in the first place?

John Whelan said earlier, "What I have noticed is one mapper will misstag then others will follow.” I think this is true, and likely goes beyond tagging — I suspect new mappers are looking for frames of reference to judge their own work by, and end up just mimicking one another’s poor mapping methods. This is one way we might be getting “experienced” mappers (i.e., thousands of edits) who are still adding a lot of problems to the map.

My suggestion:

It would be nice if HOT projects started with a single task already mapped and completed by experienced mappers, and approved by the project manager, as a “reference task” for the project. Highlight it with a unique color.

That way new mappers have an authoritative reference point for that task — they can see which imagery it should be aligned with, the level of road detail expected, the different types of tags attached to areas, which people they might message with questions, etc.

It would give motivated new mappers something trustworthy to mimic.

Thanks,

CoastLimit


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

john whelan-2
I think you mean tile but the idea has merit.

Thanks John

On 18 March 2017 at 12:43, Coast Limit <[hidden email]> wrote:
I know this discussion is focused on cleaning up these issues, but — as a relatively new mapper — can I make a suggestion on how we might better limit mistakes in the first place?

John Whelan said earlier, "What I have noticed is one mapper will misstag then others will follow.” I think this is true, and likely goes beyond tagging — I suspect new mappers are looking for frames of reference to judge their own work by, and end up just mimicking one another’s poor mapping methods. This is one way we might be getting “experienced” mappers (i.e., thousands of edits) who are still adding a lot of problems to the map.

My suggestion:

It would be nice if HOT projects started with a single task already mapped and completed by experienced mappers, and approved by the project manager, as a “reference task” for the project. Highlight it with a unique color.

That way new mappers have an authoritative reference point for that task — they can see which imagery it should be aligned with, the level of road detail expected, the different types of tags attached to areas, which people they might message with questions, etc.

It would give motivated new mappers something trustworthy to mimic.

Thanks,

CoastLimit


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
In reply to this post by Jochen123
Hi!

I just posted a new Maproulette challenge for fixing one particular case
of broken polygons: Building ways with "spikes". Details are at
http://area.jochentopf.com/fixing.html#spikes-buildings .

Because there is a large number of those in HOT areas, I have split
those off into their own challenge
(http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).
Would be great to get some help fixing those from the HOT community.

Jochen

On Wed, Mar 15, 2017 at 04:06:05PM +0100, Jochen Topf wrote:

> Date: Wed, 15 Mar 2017 16:06:05 +0100
> From: Jochen Topf <[hidden email]>
> To: [hidden email]
> Subject: [HOT] Broken (multi)polygon cleanup
>
> Hi!
>
> There are many broken (multi)polygon geometries in OSM data leading to
> all sorts of problems. I have started an effort to get this cleaned up.
> You can read more about this at http://area.jochentopf.com/ .
>
> This effort has been running for about a month now and we can already see
> some good results. See https://github.com/osmlab/fixing-polygons-in-osm/issues/15
> for some news about what's been happening.
>
> I am writing to the HOT list today, because a not-so-small part of the
> problems come from HOT mappers. For instance, looking at one particular
> problem here (spikes in closed ways), I am seeing about 5800 errors, of
> which about 1600 are in areas covered by the HOT tasking manager. (Of
> couse not all of these have been done in the course of HOT mapping, but
> probably most of them.)
>
> I am reaching out to some of the regional OSM communities and also to
> HOT, because I want to a) ask for the help of all the communities in OSM
> to fix those errors and b) generally raise awareness, so those problems
> can be minimized in the future.
>
> Please go to the web sites linked above and read up on what the problems
> are and how we are fixing them. I also want to offer my help in
> preparing special data extracts or Maproulette challenges etc. to help
> the HOT community participate in the cleaning up effort in a way that
> makes sense to you. I could, for instance, put those 1600 errors
> mentioned above in a special Maproulette challenge, if that would be a
> good way for HOT mappers to work on them.
>
> Tell me what you think. I appreciate any feedback.
>
> Jochen
> --
> Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688
>
> _______________________________________________
> HOT mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/hot

--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Donal Hunt
Re: http://area.jochentopf.com/fixing.html#spikes-buildings
I looked at one of the issues flagged (e.g. http://maproulette.org/map/2498/2186222) and don't really understand what the "spike" problem is. Are there examples of how to identify the issue? Are these issues only fixable in JOSM or can one use ID?

Re: http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499
Error: Forbidden: User [1466] does not have access to this project [379]

Thanks!

d.

On Thu, Apr 27, 2017 at 2:36 PM, Jochen Topf <[hidden email]> wrote:
Hi!

I just posted a new Maproulette challenge for fixing one particular case
of broken polygons: Building ways with "spikes". Details are at
http://area.jochentopf.com/fixing.html#spikes-buildings .

Because there is a large number of those in HOT areas, I have split
those off into their own challenge
(http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).
Would be great to get some help fixing those from the HOT community.

Jochen

On Wed, Mar 15, 2017 at 04:06:05PM +0100, Jochen Topf wrote:
> Date: Wed, 15 Mar 2017 16:06:05 +0100
> From: Jochen Topf <[hidden email]>
> To: [hidden email]
> Subject: [HOT] Broken (multi)polygon cleanup
>
> Hi!
>
> There are many broken (multi)polygon geometries in OSM data leading to
> all sorts of problems. I have started an effort to get this cleaned up.
> You can read more about this at http://area.jochentopf.com/ .
>
> This effort has been running for about a month now and we can already see
> some good results. See https://github.com/osmlab/fixing-polygons-in-osm/issues/15
> for some news about what's been happening.
>
> I am writing to the HOT list today, because a not-so-small part of the
> problems come from HOT mappers. For instance, looking at one particular
> problem here (spikes in closed ways), I am seeing about 5800 errors, of
> which about 1600 are in areas covered by the HOT tasking manager. (Of
> couse not all of these have been done in the course of HOT mapping, but
> probably most of them.)
>
> I am reaching out to some of the regional OSM communities and also to
> HOT, because I want to a) ask for the help of all the communities in OSM
> to fix those errors and b) generally raise awareness, so those problems
> can be minimized in the future.
>
> Please go to the web sites linked above and read up on what the problems
> are and how we are fixing them. I also want to offer my help in
> preparing special data extracts or Maproulette challenges etc. to help
> the HOT community participate in the cleaning up effort in a way that
> makes sense to you. I could, for instance, put those 1600 errors
> mentioned above in a special Maproulette challenge, if that would be a
> good way for HOT mappers to work on them.
>
> Tell me what you think. I appreciate any feedback.
>
> Jochen
> --
> Jochen Topf  [hidden email]  https://www.jochentopf.com/  <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688
>
> _______________________________________________
> HOT mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/hot

--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  <a href="tel:%2B49-351-31778688" value="+4935131778688">+49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Michael Heißmeier-2
Hi Donal,

Donal Hunt, 2017-04-27 16:26:
Re: http://area.jochentopf.com/fixing.html#spikes-buildings
I looked at one of the issues flagged (e.g. http://maproulette.org/map/2498/2186222) and don't really understand what the "spike" problem is. Are there examples of how to identify the issue? Are these issues only fixable in JOSM or can one use ID?


According to Jochen's description on the webpage a spike is a line which returns in itself. In the case you are asking for this line has zero length. There are three nodes in the way for this house which are at the same position. This can be visualized in JOSM using its built-in validator which detects duplicate nodes:


The house in the upper left of these nodes has the additional problem that it contains one of the twice, hence the zero-length spike:



I don't think that you can detect something like this in iD. But you can clearly see the ones with non-vanishing spikes (like http://maproulette.org/map/2499/2187301) in iD.

If you use iD my suggestion is to skip those without visible spikes and leave them to JOSM users.

Best Regards


--
Michael
(osm:michael63)

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
In reply to this post by Jochen123
On Thu, Apr 27, 2017 at 03:36:11PM +0200, Jochen Topf wrote:
> Because there is a large number of those in HOT areas, I have split
> those off into their own challenge
> (http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).

Sorry, that was the wrong URL. The correct one is:
http://maproulette.org/map/2499

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Jochen123
On Thu, Apr 27, 2017 at 05:54:39PM +0200, Jochen Topf wrote:

> Date: Thu, 27 Apr 2017 17:54:39 +0200
> From: Jochen Topf <[hidden email]>
> To: [hidden email]
> Subject: Re: [HOT] Broken (multi)polygon cleanup
>
> On Thu, Apr 27, 2017 at 03:36:11PM +0200, Jochen Topf wrote:
> > Because there is a large number of those in HOT areas, I have split
> > those off into their own challenge
> > (http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).
>
> Sorry, that was the wrong URL. The correct one is:
> http://maproulette.org/map/2499

Sorry again. I pulled the challenge and recreated a new one without the
cases where the "spike" had zero-length. That was understandably confusing.

New link is at http://maproulette.org/map/2500.

Info is still at http://area.jochentopf.com/fixing.html#spikes-buildings .

Jochen
--
Jochen Topf  [hidden email]  https://www.jochentopf.com/  +49-351-31778688

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: Broken (multi)polygon cleanup

Andrew Buck
For the ones with zero length spikes the josm validator should make
short work of them, either by fixing them automatically, or at least
flagging the duplicate node with an outline that can then manually be
fixed with the Todo plugin.  Using maproulette for these seems like a
lot more work then just loading a list of 500 or so in josm and clicking
through them in the todo plugin.

Nothing against maproulette, it is great for lots of things, but
something like this seems like you will spend more time marking things
complete on the website than actually fixing things in the editor.

-AndrewBuck


On 04/27/2017 11:08 AM, Jochen Topf wrote:

> On Thu, Apr 27, 2017 at 05:54:39PM +0200, Jochen Topf wrote:
>> Date: Thu, 27 Apr 2017 17:54:39 +0200
>> From: Jochen Topf <[hidden email]>
>> To: [hidden email]
>> Subject: Re: [HOT] Broken (multi)polygon cleanup
>>
>> On Thu, Apr 27, 2017 at 03:36:11PM +0200, Jochen Topf wrote:
>>> Because there is a large number of those in HOT areas, I have split
>>> those off into their own challenge
>>> (http://maproulette.org/ui/admin/list/379/Challenge/tasks/2499).
>>
>> Sorry, that was the wrong URL. The correct one is:
>> http://maproulette.org/map/2499
>
> Sorry again. I pulled the challenge and recreated a new one without the
> cases where the "spike" had zero-length. That was understandably confusing.
>
> New link is at http://maproulette.org/map/2500.
>
> Info is still at http://area.jochentopf.com/fixing.html#spikes-buildings .
>
> Jochen
>


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot

signature.asc (919 bytes) Download Attachment
12