Map of Population Density vs. OpenStreetMap density

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Map of Population Density vs. OpenStreetMap density

Darafei "Komяpa" Praliaskouski
Hi,

In HOT mailing list I was advised to bring a part of a thing we did to wider audience :)

We've correlated global population datasets with plain OpenStreetMap objects count. The main use case is to quickly determine how much is there to map in case of natural disaster in a smaller region, but the map itself is global - it's interesting to see what's around you and find the spots to map next, even outside of the disaster.

http://disaster.ninja/live/

What do you think?

(The HOT list thread if you are interested in disaster.ninja tool itself: https://lists.openstreetmap.org/pipermail/hot/2019-June/014908.html)

Darafei Praliaskouski
kontur.io

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Christoph Hormann-2
On Friday 05 July 2019, Darafei "Komяpa" Praliaskouski wrote:
>
> http://disaster.ninja/live/
> <http://disaster.ninja/live/#overlays=alert-shape-GDACS_EQ_1183112_12
>65046,bivariate_class;id=GDACS_EQ_1183112_1265046;layer=default-style;
>position=-13.88712117940031,30.076044779387132;zoom=2.4760319802318693
>>
>
> What do you think?

Are your densities in people/object per ground square kilometers or per
mercator square kilometers? (just to be sure - this is the number one
mistake of any kind of density analysis in the OSM context)

One warning:  All global population data sets that exist are rough
estimates with usually significant systematic biases and errors.  For
example in Switzerland the data set you used sees high population
density in mountain areas with no basis in reality.

And i am not a fan of deliberately pixelated visualizations where the
data is shown in a pixel grid at a coarser resolution than what the
display offers.

Apart from that this is an interesting analysis.  It would be kind of
nice to also do it separately for density of features that actually
correlate with population density in reality (buildings, roads,
addresses, shops etc.) and physical geography, which can be mapped just
as densely in areas with no population as in densely populated areas.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Jean-Marc Liotier
On Fri, July 5, 2019 2:40 pm, Christoph Hormann wrote:
>
> One warning:  All global population data sets that exist are rough
> estimates with usually significant systematic biases and errors.  For
> example in Switzerland the data set you used sees high population
> density in mountain areas with no basis in reality.

Same in rural Senegal. Low-resolution population data I guess.


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Darafei "Komяpa" Praliaskouski
Hi,

On Fri, Jul 5, 2019 at 4:09 PM Jean-Marc Liotier <[hidden email]> wrote:
On Fri, July 5, 2019 2:40 pm, Christoph Hormann wrote:
>
> One warning:  All global population data sets that exist are rough
> estimates with usually significant systematic biases and errors.  For
> example in Switzerland the data set you used sees high population
> density in mountain areas with no basis in reality.

Same in rural Senegal. Low-resolution population data I guess.

Can you point to a specific place please?

What would be the population dataset that you would recommend for Senegal?

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

General Discussion mailing list
Speaking more generally,
Populatation is growing fast in Africa and many african countries dont have the resources to organize regular census. See the UN Statistics Division record of last census by country https://unstats.un.org/unsd/demographic/sources/census/censusdates.htm

This means that quality vary from one country to the other and that often we lack good info about distribution of population in the various countries.  The OSM building mapping projects are often used for population estimates.
 
Pierre


Le samedi 6 juillet 2019 20 h 08 min 09 s UTC−4, Darafei "Komяpa" Praliaskouski <[hidden email]> a écrit :


Hi,

On Fri, Jul 5, 2019 at 4:09 PM Jean-Marc Liotier <[hidden email]> wrote:
On Fri, July 5, 2019 2:40 pm, Christoph Hormann wrote:
>
> One warning:  All global population data sets that exist are rough
> estimates with usually significant systematic biases and errors.  For
> example in Switzerland the data set you used sees high population
> density in mountain areas with no basis in reality.

Same in rural Senegal. Low-resolution population data I guess.


Can you point to a specific place please?

What would be the population dataset that you would recommend for Senegal?
_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

john whelan-2
There is another side to this the population counts often depend on building counts but some towns and villages are growing rapidly.  You can see the difference sometimes going from one zoom level tile to the next so the ones with buildings in often haven't the newer buildings mapped but just the old ones are so the population estimates are based on the old images not the later ones.

Cheerio John

Pierre Béland via talk wrote on 2019-07-06 8:45 PM:
Speaking more generally,
Populatation is growing fast in Africa and many african countries dont have the resources to organize regular census. See the UN Statistics Division record of last census by country https://unstats.un.org/unsd/demographic/sources/census/censusdates.htm

This means that quality vary from one country to the other and that often we lack good info about distribution of population in the various countries.  The OSM building mapping projects are often used for population estimates.
 
Pierre


Le samedi 6 juillet 2019 20 h 08 min 09 s UTC−4, Darafei "Komяpa" Praliaskouski [hidden email] a écrit :


Hi,

On Fri, Jul 5, 2019 at 4:09 PM Jean-Marc Liotier <[hidden email]> wrote:
On Fri, July 5, 2019 2:40 pm, Christoph Hormann wrote:
>
> One warning:  All global population data sets that exist are rough
> estimates with usually significant systematic biases and errors.  For
> example in Switzerland the data set you used sees high population
> density in mountain areas with no basis in reality.

Same in rural Senegal. Low-resolution population data I guess.


Can you point to a specific place please?

What would be the population dataset that you would recommend for Senegal?
_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

--
Sent from Postbox

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Oleksiy Muzalyev
In reply to this post by Darafei "Komяpa" Praliaskouski
Good morning,

As an aside, I've heard at a conference that the amount of data in the OpenStreetMap database per capita in a country is proportional to the per capita income [1] of the country.

I wonder if there is an inverse relationship. For example, if we take a country and map it exhaustively and extremely well, increasing by this the amount of the OSM data for it. Will it increase the per capita income?

I cannot be sure, but in principle in could, as a good readily available map favors economic activity. And if it were the case, the OSM mapping could be taught to millions of pupils and students as part of curriculum.

Best regards,
Oleksiy (Alex-7 @ OSM)


On 7/5/19 13:46, Darafei "Komяpa" Praliaskouski wrote:
Hi,

In HOT mailing list I was advised to bring a part of a thing we did to wider audience :)

We've correlated global population datasets with plain OpenStreetMap objects count. The main use case is to quickly determine how much is there to map in case of natural disaster in a smaller region, but the map itself is global - it's interesting to see what's around you and find the spots to map next, even outside of the disaster.

http://disaster.ninja/live/

What do you think?

(The HOT list thread if you are interested in disaster.ninja tool itself: https://lists.openstreetmap.org/pipermail/hot/2019-June/014908.html)

Darafei Praliaskouski
kontur.io

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk



_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Darafei "Komяpa" Praliaskouski
In reply to this post by Christoph Hormann-2


On Fri, Jul 5, 2019 at 3:42 PM Christoph Hormann <[hidden email]> wrote:
On Friday 05 July 2019, Darafei "Komяpa" Praliaskouski wrote:
>
> http://disaster.ninja/live/
> <http://disaster.ninja/live/#overlays=alert-shape-GDACS_EQ_1183112_12
>65046,bivariate_class;id=GDACS_EQ_1183112_1265046;layer=default-style;
>position=-13.88712117940031,30.076044779387132;zoom=2.4760319802318693
>>
>
> What do you think?

Are your densities in people/object per ground square kilometers or per
mercator square kilometers? (just to be sure - this is the number one
mistake of any kind of density analysis in the OSM context)

These are ellipsoid based square kilometers. I also dislike when visualizations fade out to the poles :)

 
One warning:  All global population data sets that exist are rough
estimates with usually significant systematic biases and errors.  For
example in Switzerland the data set you used sees high population
density in mountain areas with no basis in reality.

We're using GHS population grid in Switzerland.
https://ghsl.jrc.ec.europa.eu/data.php
Methodologically, they use radar data to find "houses". It means on their dataset people also live along roads with asphalt, and - may happen - bare rocks are also populated. You can drop them a line on [hidden email] to say thanks.

Is there a better resolution population dataset for Switzerland?

To fix it we can get "unpopulated areas" polygons from OSM. I see that forest, fields, water, quarry are likely candidates to be used to mark population as zero, if no buildings are present in OSM. What tagging would be used for unpopulated mountain?

 
And i am not a fan of deliberately pixelated visualizations where the
data is shown in a pixel grid at a coarser resolution than what the
display offers.

Can you point to a better visualization which we can learn from?
Map is supposed to be used on settlement level, where our grid is "4 pixels per screen" - to highlight a settlement without trying to predict its boundaries.
 
Apart from that this is an interesting analysis.  It would be kind of
nice to also do it separately for density of features that actually
correlate with population density in reality (buildings, roads,
addresses, shops etc.) and physical geography, which can be mapped just
as densely in areas with no population as in densely populated areas.

We've built such map initially, and it's not significantly different from this one in disaster mapping perspective. People don't map physical geography far from their home much in OSM, and large unmapped regions don't become more mapped if we lower feature counts in more mapped regions. 
Note that any large multipolygon is counted as just 1 feature here.


Darafei Praliaskouski
Kontur.io

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Darafei "Komяpa" Praliaskouski
In reply to this post by General Discussion mailing list


On Sun, Jul 7, 2019 at 3:46 AM Pierre Béland <[hidden email]> wrote:
Speaking more generally,
Populatation is growing fast in Africa and many african countries dont have the resources to organize regular census. See the UN Statistics Division record of last census by country https://unstats.un.org/unsd/demographic/sources/census/censusdates.htm

This means that quality vary from one country to the other and that often we lack good info about distribution of population in the various countries.  The OSM building mapping projects are often used for population estimates.

This is exactly the reason we threshold the visualization into just three population classes: population dataset behind it is using classified satellite imagery multiplied with census counts.
We're not much interested in exact population counts for this task, rather whether there's something on satellite that's worth inspecting and probably mapping.

Darafei Praliaskouski
Kontur.io

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Christoph Hormann-2
In reply to this post by Darafei "Komяpa" Praliaskouski
On Sunday 07 July 2019, Darafei "Komяpa" Praliaskouski wrote:
>
> We're using GHS population grid in Switzerland.
> https://ghsl.jrc.ec.europa.eu/data.php
> Methodologically, they use radar data to find "houses". It means on
> their dataset people also live along roads with asphalt, and - may
> happen - bare rocks are also populated. You can drop them a line on
> [hidden email] to say thanks.

I am familiar with that data - they use census based or otherwise
estimated population numbers per admininstrative unit and distribute
this population among areas they identified as "built-up" using rather
questionable processes (what we in German tend to describe
as "Kaffeesatzleserei").  There is no identification of houses - source
data used is way too low resolution for that.

I am not aware of any serious overall evaluation of the quality of this
or any other global population density data sets.  If you read
literature on the matter the quality/validation part is usually just
some superficial "throwing around numbers to make the results look
good" without actually looking at how the data compares to the
geographic reality it is meant to represent and where and how it fails
to do so.

I am sorry for the negativity - i just know all too well how these kind
of publicly financed research projects work in Europe and how detached
from reality they often become.

> To fix it we can get "unpopulated areas" polygons from OSM.

Not really - you would have to reproduce the population distribution
process described above based on corrected data of builtup areas.  If
you just remove populations that are obviously wrong locally you'd
underestimate the overall population.

> > And i am not a fan of deliberately pixelated visualizations where
> > the data is shown in a pixel grid at a coarser resolution than what
> > the display offers.
>
> Can you point to a better visualization which we can learn from?
> Map is supposed to be used on settlement level, where our grid is "4
> pixels per screen" - to highlight a settlement without trying to
> predict its boundaries.

You are essentially visualizing a classification map (with ten classes).  
The most common way to do this would be on a per pixel basis.  See for
example the "NLCD Land Cover" layer on https://viewer.nationalmap.gov/.
If this is too noisy (which is also very much influenced by the choice
of colors) you can denoise and geometrically generalize the
classification for the target resolution.  Just subsampling at a coarser
grid does not really work for this - you just get coarser noise and
less information.

> We've built such map initially, and it's not significantly different
> from this one in disaster mapping perspective. People don't map
> physical geography far from their home much in OSM [...].

The thing is that statement is correct to very different degrees in
different parts of the world.  Looking selectively at the mapping of
physical geography would allow evaluating those differences.

Of course you are right that just counting features would not really
work for analyzing that.  Counting features works well for things with a
fairly defined amount of information per feature (like buildings,
addresses, POIs) but not for geometrically sophisticated geometries.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Sérgio V.
In reply to this post by Darafei "Komяpa" Praliaskouski
Excelent and inteligent initiative Darafei
Measuring "density of OSM data" per "Population density" is a powerful metric to analyse where is it lacking more mapping in OSM. 
It's a nice effort to minimize "inequality" in OSM map. 
(for more info, see "World Inequality Database" at https://wid.world/world)

I've did that metric for Brazil in 2017, "Demography in Brazil to help mapping in OSM", at:
It helped me much to find places lacking mapping. 
Everytime I just take a fast look at the highlighted places int that map, I've actualy found undermapped places, lacking roads mostly.

These metrics lead to view two aspects: 
a) By one hand, the world is in a fast urbanizing process. People more and more migrate to bigger cities, looking for better jobs, services, better life. 
According to United Nations report, "68% (2/3) of the world population projected to live in urban areas by 2050" (UN 2018-05-16 - https://www.un.org/development/desa/en/news/population/2018-revision-of-world-urbanization-prospects.html).
So, it's generally expected to have a lot of irregular setlements to map in broader urban areas. Even benig irregular, millions of people live in such places.  
b) By the other hand, people remaining in country sides become more alone (many times old people), living far away from good public services and enough incomes, broadly unassisted. So also important to map their accessibility.

Don't matter with objections for too much fine precision on demography. 
That metric anyway gives much more reasonable focus than usual over&under-concentrated mapping done in OSM. 
Nice, keep going, publish it.
Regards


- - - - - - - - - - - - - - - -

Sérgio - http://www.openstreetmap.org/user/smaprs


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Darafei "Komяpa" Praliaskouski
Thank you!
Will keep publishing :)

On Mon, Jul 8, 2019 at 2:56 PM Sérgio V. <[hidden email]> wrote:
Excelent and inteligent initiative Darafei
Measuring "density of OSM data" per "Population density" is a powerful metric to analyse where is it lacking more mapping in OSM. 
It's a nice effort to minimize "inequality" in OSM map. 
(for more info, see "World Inequality Database" at https://wid.world/world)

I've did that metric for Brazil in 2017, "Demography in Brazil to help mapping in OSM", at:
It helped me much to find places lacking mapping. 
Everytime I just take a fast look at the highlighted places int that map, I've actualy found undermapped places, lacking roads mostly.

These metrics lead to view two aspects: 
a) By one hand, the world is in a fast urbanizing process. People more and more migrate to bigger cities, looking for better jobs, services, better life. 
According to United Nations report, "68% (2/3) of the world population projected to live in urban areas by 2050" (UN 2018-05-16 - https://www.un.org/development/desa/en/news/population/2018-revision-of-world-urbanization-prospects.html).
So, it's generally expected to have a lot of irregular setlements to map in broader urban areas. Even benig irregular, millions of people live in such places.  
b) By the other hand, people remaining in country sides become more alone (many times old people), living far away from good public services and enough incomes, broadly unassisted. So also important to map their accessibility.

Don't matter with objections for too much fine precision on demography. 
That metric anyway gives much more reasonable focus than usual over&under-concentrated mapping done in OSM. 
Nice, keep going, publish it.
Regards


- - - - - - - - - - - - - - - -

Sérgio - http://www.openstreetmap.org/user/smaprs

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk


--
Darafei Praliaskouski

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Map of Population Density vs. OpenStreetMap density

Darafei "Komяpa" Praliaskouski
In reply to this post by Christoph Hormann-2
Hi,

We've updated http://disaster.ninja/live/ to use fresh 2019 release of GHSL dataset.
Mountains in Switzerland aren't populated anymore. Chinese mountain regions are also better detailed.

Have a good day!

Darafei
Kontur.io

On Sun, Jul 7, 2019 at 3:23 PM Christoph Hormann <[hidden email]> wrote:
On Sunday 07 July 2019, Darafei "Komяpa" Praliaskouski wrote:
>
> We're using GHS population grid in Switzerland.
> https://ghsl.jrc.ec.europa.eu/data.php
> Methodologically, they use radar data to find "houses". It means on
> their dataset people also live along roads with asphalt, and - may
> happen - bare rocks are also populated. You can drop them a line on
> [hidden email] to say thanks.

I am familiar with that data - they use census based or otherwise
estimated population numbers per admininstrative unit and distribute
this population among areas they identified as "built-up" using rather
questionable processes (what we in German tend to describe
as "Kaffeesatzleserei").  There is no identification of houses - source
data used is way too low resolution for that. 

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk