AI detecting of buildings Idle thoughts

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

AI detecting of buildings Idle thoughts

john whelan-2
Could this be used to detect villages and towns which have not yet been mapped.

If something could drop some sort of marker where it thinks a cluster of buildings are then we could use overpass to pull them into JOSM and map them as places, landuse=residential, village or whatever.

Thoughts?

Thanks John

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Christoph Hormann-2
On Thursday 16 August 2018, john whelan wrote:
> Could this be used to detect villages and towns which have not yet
> been mapped.
>
> If something could drop some sort of marker where it thinks a cluster
> of buildings are then we could use overpass to pull them into JOSM
> and map them as places, landuse=residential, village or whatever.

As you might already imagine reality is way ahead of you - for example
someone has recently been dumping a whole bunch of garbage exactly like
this in eastern Congo:

https://www.openstreetmap.org/changeset/61401087
https://www.openstreetmap.org/way/614459980

From a purely technical point of view if it wasn't littering OSM this
would just be droll.

Verifiability my ass...

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Warin
On 16/08/18 17:56, Christoph Hormann wrote:

> On Thursday 16 August 2018, john whelan wrote:
>> Could this be used to detect villages and towns which have not yet
>> been mapped.
>>
>> If something could drop some sort of marker where it thinks a cluster
>> of buildings are then we could use overpass to pull them into JOSM
>> and map them as places, landuse=residential, village or whatever.
> As you might already imagine reality is way ahead of you - for example
> someone has recently been dumping a whole bunch of garbage exactly like
> this in eastern Congo:
>
> https://www.openstreetmap.org/changeset/61401087
> https://www.openstreetmap.org/way/614459980
>
>  From a purely technical point of view if it wasn't littering OSM this
> would just be droll.
>
> Verifiability my ass...
>

Sigh.

I suppose if you want to see blank areas of the map you'll have to look
for the deserts .. (including the polar caps). Oh .. and oceans too.


As that particular mapper has local knowledge on their side I'd not
challenge them.

Though a change set of close to 2,000 ways does look a bit large.


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Christoph Hormann-2
On Thursday 16 August 2018, Warin wrote:
>
> As that particular mapper has local knowledge on their side I'd not
> challenge them.

Why not?

I would love to see some ground level or aerial/satellite images
documenting the verifiability of those outlines.

The good thing about verifiability and the core of OpenStreetMap in
general is that you don't have to trust some imagined authority about
the data, you can independently verify it.

I try to give people the benefit of the doubt and assume in such cases
they are just ignorant of these principles or do not get why they are
important but with organized efforts like this i can't help but get the
impression there is a certain amount of malice to sabotage or at least
an excessive amount of carelessness.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Warin
On 16/08/18 19:26, Christoph Hormann wrote:

> On Thursday 16 August 2018, Warin wrote:
>> As that particular mapper has local knowledge on their side I'd not
>> challenge them.
> Why not?
>
> I would love to see some ground level or aerial/satellite images
> documenting the verifiability of those outlines.
>
> The good thing about verifiability and the core of OpenStreetMap in
> general is that you don't have to trust some imagined authority about
> the data, you can independently verify it.
>
> I try to give people the benefit of the doubt and assume in such cases
> they are just ignorant of these principles or do not get why they are
> important but with organized efforts like this i can't help but get the
> impression there is a certain amount of malice to sabotage or at least
> an excessive amount of carelessness.

Satellite imagery is available for the world..
But how much do you know of Africa?

A residential area in Germany or Australia is very different from one in rural Africa or Asia.
A residential house/home is also very different ..
A fuel station in Africa can be signified by PET bottles strung up in a tree.. or on a fence.
and the fuel comes in those same PET bottles ... no 'gas pumps'. But you get your fuel.
We should not be imposing our standards on them, but rather accept their interpretation of what is a house, what is residential, etc.

I would not like to go to that particular area of the world to get ground truth ...
Western CDR ok .. eastern .. ummm ... no thanks.


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Christoph Hormann-2
On Thursday 16 August 2018, Warin wrote:
>
> Satellite imagery is available for the world..
> But how much do you know of Africa?
> [...]

You maybe don't realize that but the kind of data garbage i pointed to
is the direct result of projecting ideas and experiences of settlement
structures of some part of the world onto a different one. We are
mostly talking about scattered dwellings of what are probably mostly
subsistence farmers here.  The pointless polygon geometry drawing is
the failed attempt to regard those as a typical European/North American
residential area.  

If this is due to a lack of knowledge about the actual geography or
because of a misguided belief that making it crudely look a bit like an
European/North American residential area is kind of beneficial for the
people there i don't know.

Anyway we are drifting off-topic here and this does not really help the
original question from John.  My answer to that would be:  Yes,
automated methods can help to find unmapped settlements in OSM - less
though in actually mapping them.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

ebel
What's funny is that this import was (according to the changeset
comment) based on "DigitalGlobe extracted building data". A straight up
import of the original building geometries would probably be (i) less
contentious (since a building is a building is a building), and (ii)
more accurate for calculating population figures (a use for building
data for humanitarian purposes) and (iii) better for OSM since lots of
buildings is better than landuse=residential polygons.

Sometimes people can try to be *too* clever. 🙂

On 16/08/18 13:16, Christoph Hormann wrote:

> On Thursday 16 August 2018, Warin wrote:
>>
>> Satellite imagery is available for the world..
>> But how much do you know of Africa?
>> [...]
>
> You maybe don't realize that but the kind of data garbage i pointed to
> is the direct result of projecting ideas and experiences of settlement
> structures of some part of the world onto a different one. We are
> mostly talking about scattered dwellings of what are probably mostly
> subsistence farmers here.  The pointless polygon geometry drawing is
> the failed attempt to regard those as a typical European/North American
> residential area.
>
> If this is due to a lack of knowledge about the actual geography or
> because of a misguided belief that making it crudely look a bit like an
> European/North American residential area is kind of beneficial for the
> people there i don't know.
>
> Anyway we are drifting off-topic here and this does not really help the
> original question from John.  My answer to that would be:  Yes,
> automated methods can help to find unmapped settlements in OSM - less
> though in actually mapping them.
>


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Christoph Hormann-2
On Thursday 16 August 2018, Rory McCann wrote:
> What's funny is that this import was (according to the changeset
> comment) based on "DigitalGlobe extracted building data". A straight
> up import of the original building geometries would probably be (i)
> less contentious (since a building is a building is a building), and
> (ii) more accurate for calculating population figures (a use for
> building data for humanitarian purposes) and (iii) better for OSM
> since lots of buildings is better than landuse=residential polygons.

I found this peculiar as well - the most likely explanation seems to be
that the quality of building detection and especially of building
geometry generation (if that is being done at all) is probably quite
bad and by not using the building data directly you can kind of
disguise such deficits.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Andy Townsend
In reply to this post by ebel
On 16/08/2018 12:35, Rory McCann wrote:
> What's funny is that this import was (according to the changeset
> comment) based on "DigitalGlobe extracted building data". A straight up
> import of the original building geometries would probably be (i) less
> contentious (since a building is a building is a building)

I think the problem here is that "a building is a building is a
building" isn't really true.  There are many different kinds of
structures in many different parts of the world, and many different
light conditions, and different ground surfaces.

We saw this problem with the Facebook "roads import" in Egypt - whatever
software they were using was detected many sharp edges in imagery
(walls, canals, etc.) as roads.  If you train whatever you're using to
detect stuff in one environment and try and use it in another
environment it's going to get things wrong in unpredictable ways, and
that's not really going to be obvious if you then (as here) estimate
residential areas based on extracted buildings.  I can, however, see the
sense of trying to do that (estimate residential areas based on
extracted buildings) - in some areas actual buildings are likely to be
ephemeral, but usage of areas not so much.

Best Regards,

Andy




_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Frederik Ramm
In reply to this post by Christoph Hormann-2
Hi,

On 16.08.2018 09:56, Christoph Hormann wrote:
> https://www.openstreetmap.org/changeset/61401087

Btw I have commented on this changeset. Apparently the main use case is
having a general indication of "there are people living here".

In a way, it's what we did in Western Europe when we only had Landsat
imagery: "Uh, this looks like a settlement, let's draw a grey blob" ;)

I guess it would look less silly if it had been created on a coarser
level. What we have here *suggests* precision due to the many nodes, but
in the end it's relatively random. Like if you specify the result of a
measurement as 4.35375423 when your error is +/- 1 ;)

A little buffering and simplifying would probably have been a sensible
idea. But then again, had they asked beforehand, it's possible that we
would have told them it's a bad idea to start with ;)

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

pierzen
In reply to this post by Christoph Hormann-2
Claire who added these polygons  is a resident of DR Congo. She coordinated with me the North-Kivu OSM Response in 2012, coordinating with the UN agencies and NGO's in Kinshasa.  She is coordinator of OSM-DRC and coordinator of this OSM Response for the Ebola outbreak around Beni, working closely with the DRC ministry of health and the humanitarian NGO's. I do support Claire for this coordination and other OSM projects in DRC. And we took the decision to use this info to spot rapidly the populated areas. «Take time» to look at these polygons one by one  (we did) and you will see that they reflect adequately the density of housing in these areas.

In may, has Potentiel 3.0 just started to support OSM-DRC for the OpenCities project in Kinshasa, we collectively had to reorganize quickly and respond to the Ebola Oubreak. This second outbreak in august is in a different region. Each time, OSM-DRC volunteers accept to support the responses, to go in various towns and organize activies. This is a very dynamic OSM communty that know the field.

Quality is very important for us and we started a project to use topological analysis to enhance the quality of OSM.  A first analysis based on the geometry of the buildings that I published last week on the hot lis was not commented except 1 answer. See https://lists.openstreetmap.org/pipermail/hot/2018-August/014529.html

Pursuing the analysis, I have identified buildings that cross roads or various other polygons and cleaned the data.  See https://www.openstreetmap.org/changeset/61701721#map=12/-4.3993/15.3556

While we support this response, other OSM contributors in Kinshasa are organizing a 3 days Focus group for the OpenCities project with the neighborhood representatives to evaluate infrastructures at risk in case of outbreaks or floods.
See OpenStreetMap RDC on Twitter
 






We are highly involved, volunteering for OSM and you should understand that we take some critics with a «a grain of salt».

But the contributor Christoph is going a bit far, insulting, expressing doubts about skills of OSM valuable volunteers that know the reality on the ground and respond in such difficult context. He should use less epithets, stop signing «Verifiability my ass...», clean it, realign his «idle thoughts» and make excuses to Claire.

Regard
 
Pierre


Le jeudi 16 août 2018 07 h 57 min 38 s HAE, Christoph Hormann <[hidden email]> a écrit :


On Thursday 16 August 2018, Rory McCann wrote:
> What's funny is that this import was (according to the changeset
> comment) based on "DigitalGlobe extracted building data". A straight
> up import of the original building geometries would probably be (i)
> less contentious (since a building is a building is a building), and
> (ii) more accurate for calculating population figures (a use for
> building data for humanitarian purposes) and (iii) better for OSM
> since lots of buildings is better than landuse=residential polygons.

I found this peculiar as well - the most likely explanation seems to be
that the quality of building detection and especially of building
geometry generation (if that is being done at all) is probably quite
bad and by not using the building data directly you can kind of
disguise such deficits.

--
Christoph Hormann
http://www.imagico.de/


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Christoph Hormann-2
In reply to this post by Frederik Ramm
On Thursday 16 August 2018, Frederik Ramm wrote:
>
> In a way, it's what we did in Western Europe when we only had Landsat
> imagery: "Uh, this looks like a settlement, let's draw a grey blob"

Absolutely not.

The settlement structure of Western Europe can be pretty accurately
mapped from Landsat images.

What the data import linked to contains has no similarity to this.  And
even if it did superficially this would be a pretend similarity because
the settlement structure in this part of the world looks nothing like
that of Western Europe.  This is just taking some auto-detected
buildings, throwing some random algorithms at it and labeling the
resulting abstract geometries landuse=residential.

Ironically if you did do a halfway reasonable classification of
settlement areas in Landsat data for this area the result would
probably be much more like a verifiable mapping of settlements in the
area than what we can see now in the database.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: AI detecting of buildings Idle thoughts

Christoph Hormann-2
In reply to this post by pierzen

> And we took the decision to use this info
> to spot rapidly the populated areas. «Take time» to look at these
> polygons one by one  (we did) and you will see that they reflect
> adequately the density of housing in these areas.

No, they don't (at least not for any meaningful definition of "density
of housing").

In any case even if they did - iso-lines of some model of a building
density field are quite fundamentally not something that is mappable in
OSM, especially not with landuse=residential.

It seems i need to clarify one thing:  My harsh criticizm of the data
imported (which i stand by firmly) is about the data.  I - just like
probably everyone else here - am aware that clairedelune did not
generate this data.  The kind of problem we see here is exactly the
reason why we have import guidelines and why we need a directed editing
policy so mappers do not get into a situation where they add bad data
in larger volume because they follow - usually with good intentions -
the unqualified instructions of others or wrongly believe the quality
claims of data providers.

If the import plans had been properly discussed we could have had this
discussion in advance and could have considered useful options - like
for example the idea of impoting the buildings as Rory suggested.

I also want to make sure this example is not blown out of proportion.  
There are plenty of bad quality imports and bad mapping in OSM.  If you
look at landuse=residential mapping in Eastern Africa this is not the
worst data in the database, not by a large margin.  I just pointed it
out here as an example because it was a perfect fit for the idea John
brought up.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk