The point on the OSM Response to the DR Congo Nord Kivu Ebola outbreak

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

The point on the OSM Response to the DR Congo Nord Kivu Ebola outbreak

pierzen
The media reported recently about the current Ebola outbreak in Nord Kivu and the difficulty for the humanitarians to operate in a difficult security context with rebels around towns plus population displacements in response to various attacks. 

After Beni, the efforts are now concentrated around Butembu. The population of this town is increasing rapidly.The DR Congo health department, the World Health Organization and MSF need good evaluations of population for vaccination campaigns. In this context, OSM-DRC has obtained recent imagery courtesy of USAID that shows many new buildings.

We would like assure a quick response but are faced with quality problems. This week, we started new a job for Bulembo, east of Butembo with new imagery.  https://tasks.hotosm.org/project/5585
With good imagery and a rural area, it seemed to us that this job would fit for newbies.  But we rapidly observed on the map more and more overlaps and imprecise building tracing.

We then used the Irregular shapes indicator that we developped a few months ago to quantify these problems. The indicator offers us a measure of the problem and let us validate the individual buildings reported as needing close evaluation of builidings quality problems (shapes + overlaps). 
Looking at project 5585, we observe :
- on dec.5 with beginners, participation, 3,025 buildings were edited but 40.2% with irregular shapes
- on dec.6 with beginners excluded,  3,471 buildings were edited with only 0.9% with irregular shapes. 

The OSM-DRC coordination team is also faced with a high ratio builidngs bad quality + overlaps for the densely town of Butembu : TM 5485, 5487 and 5507.

This is caused by various factors, including dense urban area, dark imagery, different offsets between imagery plus often very imprecise building tracing from newbies.  Observing the multiplication of problems this week for Butembu, including a high ratio of building overlaps with roads, we have decided to archive the current jobs for Butembu.  The option seems to start over with the new imagery and first realign buildings and roads before adding new buildings.  And the jobs would be reserved to experienced mappers. The Tasking manager offers options to select intermediate contributors. We should add jobs in the next days to revise Butembu.

We also need to discuss how to better integrate the participation of the newbies  in disaster mapping and at the same time assure both a quick answer and high quality.  Solutions should be proposed to assure that newbies receive an adequate training and how to better monitor their participation.  As coordinators of the response, we dont know where from the newbies come from. Are they simply coming and select a task or participate to a mapathon ? This is hard to say.

Mapathons might be the solution to better supervise newbies participation.  But this only if the mapathons organizers accept the responsability to provide high quality data for their group, to offer more training and assure more monitoring of their participants.

regard

 
Pierre

_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: The point on the OSM Response to the DR Congo Nord Kivu Ebola outbreak

Dr. Franz-Josef Behr

Very interesting findings and recommendations, Pierre!

I find it very useful to have some numbers like that, not only vague recommendations regarding education.

Best regards - Franz-Josef


Am 07.12.2018 um 06:02 schrieb Pierre Béland:
The media reported recently about the current Ebola outbreak in Nord Kivu and the difficulty for the humanitarians to operate in a difficult security context with rebels around towns plus population displacements in response to various attacks. 

After Beni, the efforts are now concentrated around Butembu. The population of this town is increasing rapidly.The DR Congo health department, the World Health Organization and MSF need good evaluations of population for vaccination campaigns. In this context, OSM-DRC has obtained recent imagery courtesy of USAID that shows many new buildings.

We would like assure a quick response but are faced with quality problems. This week, we started new a job for Bulembo, east of Butembo with new imagery.  https://tasks.hotosm.org/project/5585
With good imagery and a rural area, it seemed to us that this job would fit for newbies.  But we rapidly observed on the map more and more overlaps and imprecise building tracing.

We then used the Irregular shapes indicator that we developped a few months ago to quantify these problems. The indicator offers us a measure of the problem and let us validate the individual buildings reported as needing close evaluation of builidings quality problems (shapes + overlaps). 
Looking at project 5585, we observe :
- on dec.5 with beginners, participation, 3,025 buildings were edited but 40.2% with irregular shapes
- on dec.6 with beginners excluded,  3,471 buildings were edited with only 0.9% with irregular shapes. 

The OSM-DRC coordination team is also faced with a high ratio builidngs bad quality + overlaps for the densely town of Butembu : TM 5485, 5487 and 5507.

This is caused by various factors, including dense urban area, dark imagery, different offsets between imagery plus often very imprecise building tracing from newbies.  Observing the multiplication of problems this week for Butembu, including a high ratio of building overlaps with roads, we have decided to archive the current jobs for Butembu.  The option seems to start over with the new imagery and first realign buildings and roads before adding new buildings.  And the jobs would be reserved to experienced mappers. The Tasking manager offers options to select intermediate contributors. We should add jobs in the next days to revise Butembu.

We also need to discuss how to better integrate the participation of the newbies  in disaster mapping and at the same time assure both a quick answer and high quality.  Solutions should be proposed to assure that newbies receive an adequate training and how to better monitor their participation.  As coordinators of the response, we dont know where from the newbies come from. Are they simply coming and select a task or participate to a mapathon ? This is hard to say.

Mapathons might be the solution to better supervise newbies participation.  But this only if the mapathons organizers accept the responsability to provide high quality data for their group, to offer more training and assure more monitoring of their participants.

regard

 
Pierre


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

Re: The point on the OSM Response to the DR Congo Nord Kivu Ebola outbreak

pierzen
Thanks Franz-Josef

We started  a new project to both redress the Quality of the data for Butembo and to add new buildings with the rapid growth of this city population (more then 3% per year). Thanks to USAID who provided a recent imagery who shows a lot more buildings.  This information is important to evaluate the population of the city for vaccination campaigns. The last census was 10 years ago.
https://tasks.hotosm.org/project/5601

With the extent of the quality problems from Newbies participation, we have to reserve this project to experienced contributors preferably using JOSM and that know how to adjust offset of imageries.  Validating first contributions to project 5601 this morning, we observed that it is going very well and we just need more contributors :)  Let's keep it motivating for experienced mappers to Respond to such humanitarian emergencies while solutions are looked at to better integrate Newbies in such projects.  This first project to redress the data exclude hills where different offsets are observed for the imagery. We will add distinct projects for these areas.

Analysis of Newbies contribution

The OSM-DRC coordination team with Claire Halleux, Fred Moine and myself, we have followed closely the tasking manager projects 5485, 5487 and 5585 now archived. Looking at the map, It was easy to observe that Tracing of buildings was very inadequate, everywhere, and statistics did show a rapid growth of errors  with the participation of newbies. Restricting projects to experienced mappers, we could confirm that exclusion of Newbies was drastically reducing errors. This Butembo OSM Map extract on twitter shows Quality problems that we find everywhere around Butembo with sometimes 2-3-4 overlap buildings and overlaps over roads.
https://twitter.com/pierzen/status/1072282951210147840

For many of us, it was motivating to start to participate to OSM while mapping for major humanitarian responses. At first the ratio of newbies / Experienced mappers did let progress smoothly. But with media coverage of such OSM humanitarian responses, more and more newbies have come.  The Tasking manager and the organisation of Mapathons greatly contributed to escalate the number of participants.  Now let's say that this is time to escalate the Quality too.

The errors observed seem to confirm that Newbies have easy access to OSM editors and start to edit without any knowledge, no training and monitoring. Do Mapathons can redress this situation or do they let contributors map without any training and monitoring ? We have to assure that Mapathons organizers play a role, take the responsability to produce Quality data, to train and monitor Newbies mapping, to assure that they trace regular buildings (not fantasy forms we too often see) assure tracing corresponds to the outline of the buildings observed on the imagery. Sorry to say that the quality observed for Butembo is quite inappropriate to respond to this Ebola outbreak emergency.

Let's mention that some Mapathons organizers have good procedures to train and validate data. Other organizers should follow this trend and «Live Monitoring of quality» should also be look at by the Mapathon organizers to assure to deliver Quality data at the end of a session.

Adding to that, there are hills in various parts the city and images available have various offsets in the city as compared to the Bing reference. The newbies did not align the imagery and have simply traced buildings over the roads previously traced with Bing. Trying to correct this, some inexperienced validators simply started to realign the roads with the imagery used for projects 5485, 5487 and 5585.

In the meantime, we have no other choice for this Nord Kivu Ebola Activation to restrict mapping to experienced mappers.

regard

 
Pierre


Le vendredi 7 décembre 2018 01 h 49 min 03 s HNE, Franz-Josef Behr <[hidden email]> a écrit :


Very interesting findings and recommendations, Pierre!

I find it very useful to have some numbers like that, not only vague recommendations regarding education.

Best regards - Franz-Josef


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot
Reply | Threaded
Open this post in threaded view
|

The point on the OSM Response to the DR Congo North Kivu / Ituri Ebola outbreak

pierzen
Thanks to all the contributors for the Tasking Manager project 5601 for Butembo which is now completed with a great success. The number of buildings has doubled from 2018-12-11 and is now at 146,466 for the area covered. Also the ratio of Irregular geometries and Duplicates has dramatically decreased.

The coordination group for this Ebola Response now switch the priority to the areas along the road between Beni and Bunia (Job 5660) and we should come back later to complete Butembo. 

For job 5660, Buildings and roads need to be completed for the urban areas and the forestry isolated areas.  In the forestry area, we ask to keep the map significant and then avoid to add short segments of temporary tracks where the forestry workers did cut wood or to indicate these areas as wood cut since either these tracks or wood cut are permanent.

The project is again limited to intermediate mappers to assure quality in the context of this emergency.revising geometries and adding new buildings. 

To see jobs available and priorities for this OSM Response, see https://tasks.hotosm.org/contribute?difficulty=ALL&organisation=OSM-CD&campaign=Ebola2018

regard

 
Pierre


_______________________________________________
HOT mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/hot