Comparing quality of different geocoders?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Comparing quality of different geocoders?

Developer Discussion mailing list
Is anyone aware about some test suite comparing different geocoders?

Something with query + expected result pairs, and listing where Nominatim / Photon / etc succeeded/failed?

I want something like that, tried to find one and failed.

I want to ask whatever I missed something like that before I will make it.

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Comparing quality of different geocoders?

Sarah Hoffmann
Hi,

On Wed, Aug 26, 2020 at 09:11:44AM +0200, Mateusz Konieczny via dev wrote:
> Is anyone aware about some test suite comparing different geocoders?
>
> Something with query + expected result pairs, and listing where Nominatim / Photon / etc succeeded/failed?
>
> I want something like that, tried to find one and failed.
>
> I want to ask whatever I missed something like that before I will make it.

There are a couple of test suits out there. The two that come
immediately to mind are:
https://github.com/geocoders/geocoder-tester
https://github.com/pelias/fuzzy-tester

The issue with all these is that you have to define what you
actually want to compare. Depending on what kind of queries you
run and how you define the results you get, you'll get very different
results.

Sarah


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Comparing quality of different geocoders?

Developer Discussion mailing list



Aug 26, 2020, 11:04 by [hidden email]:
Hi,

On Wed, Aug 26, 2020 at 09:11:44AM +0200, Mateusz Konieczny via dev wrote:
Is anyone aware about some test suite comparing different geocoders?

Something with query + expected result pairs, and listing where Nominatim / Photon / etc succeeded/failed?

I want something like that, tried to find one and failed.

I want to ask whatever I missed something like that before I will make it.

There are a couple of test suits out there. The two that come
immediately to mind are:
https://github.com/geocoders/geocoder-tester
https://github.com/pelias/fuzzy-tester
Thanks!

I will need to look how tightly pelias/fuzzy-tester is
integrated with just pelias and is it viable to expand/use
it to support testing REST apis.

Maybe it will be easier to write something small from scratch.
The issue with all these is that you have to define what you
actually want to compare. Depending on what kind of queries you
run and how you define the results you get, you'll get very different
results.
Oh, I know!

For example cases such as "Elementary school number 12, Streetname 19"
where address and object are at different locations and expected result
completely depends on what user wants.

Or slang/local/vulgar names where some would prefer not finding them
and not listing them over finding them, even if naming is real.

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev