Update on recent performance tests

classic Classic list List threaded Threaded
2 messages Options
mmd
Reply | Threaded
Open this post in threaded view
|

Update on recent performance tests

mmd
Hi,

if you've been around on this mailing list a bit, you probably remember
the performance tests I did last year. As more and more performance
improvements have accumulated in my test758 branch, it was time to
re-run a similar test. So, here's a short summary on those recent
activities. I will probably write a more detailed blog post later.

Hardware was very similar to last year (32GB, Intel Xeon E3-1270V3),
with the exception of a bigger SSD, which now holds a full attic
database (480GB DC SSD). Still commodity hardware for 50-60€/month.

First test was focused on ridiculously simple queries
[out:json];node(63174280);out; (with node ids increasing), but a larger
number of them in parallel.

Due to changes in the dispatcher processing, we're now at 2000
queries/s, or some 172 mio. requests/day.

Of course, I don't want to advocate this kind of usage, but it's good to
know, we can keep up with a large number of very small requests you
would typically see with interactive maps.

Second test was a simulation of a full day in the life of
overpass-api.de. That's 534'497 real queries in total, processed with 7
parallel processes in 10.75 hours, without any waiting time, though. I
found the response time distribution quite interesting:

Response times
--------------

50%     90%     95%     99%     99.5%   99.9%   99.99%  99.999%
0.062s  0.685s  1.58s   5.5s    11.06s  33.92s  168.37s 853.6s

(quantile percentages: 50% of all queries were processed in up to 62ms)

With those changes in place, I would even go as far and reduce the
default timeout down to 60s, and set a hard query timeout to maybe 300s
- 600s, even if people set the timeout value to, say 500000s.

You can find tons of munin charts and more details over at Github:

Full day simulation: https://github.com/mmd-osm/Overpass-API/issues/3
Scalability: https://github.com/mmd-osm/Overpass-API/issues/2
Source: https://github.com/mmd-osm/Overpass-API/tree/test758_lz4hash

Cheers
mmd
mmd
Reply | Threaded
Open this post in threaded view
|

Re: Update on recent performance tests

mmd
Am 04.09.2017 um 20:25 schrieb mmd:

>
> Second test was a simulation of a full day in the life of
> overpass-api.de. That's 534'497 real queries in total, processed with 7
> parallel processes in 10.75 hours, without any waiting time, though. I
> found the response time distribution quite interesting:

As I noted here [1], you could run the same test to fit an actual 24
hours time frame (or least try to match it as closely as possible), and
see what the CPU utilization looks like.

Unlike the situation on the main instance with 85%+ CPU utilization for
most part of the day, about 2 cpu cores (or 25% cpu utilization) seemed
to be sufficient to run all queries, as well a minutely db update, an
hourly area creation delta run, and gzip compression on the web server.

In conclusion, there's some very good potential to make much better use
of available resources.

Cheers,
mmd


[1] http://www.openstreetmap.org/user/drolbr/diary/39710#comment39826
[2] https://github.com/mmd-osm/Overpass-API/issues/3#issuecomment-334138177

--