Minutely update server slow?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Minutely update server slow?

Lynn W. Deffenbaugh (Mr)

I run a personal tile server for the entire planet and have it configured with minutely updates downloaded by osmosis.  Over the past few days, I've noticed that my updates are falling behind and subsequently noticed in the osmosis.log file that it is taking a LONG time to download 4-6 hours of minutely diffs.  This usually took 5-10 minutes, but recently it takes 45 minutes to an hour.  And sometimes osmosis times out completely.

Here's my osmosis configuration.txt file if that helps.

# The URL of the directory containing change files.
baseUrl=https://planet.openstreetmap.org/replication/minute

# Defines the maximum time interval in seconds to download in a single invocation.
# Setting to 0 disables this feature.
maxInterval = 21600

Anyone else seeing slow minutely diff downloads?

Lynn (D)

PS.  Here's the munin rdnerd Data import lag daily and weekly graphs:

Data import lag

Data import lag


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Minutely update server slow?

General Discussion mailing list
what about open street cam .org
 
Sent: Saturday, April 20, 2019 1:03 PM
Subject: [OSM-talk] Minutely update server slow?
 

I run a personal tile server for the entire planet and have it configured with minutely updates downloaded by osmosis.  Over the past few days, I've noticed that my updates are falling behind and subsequently noticed in the osmosis.log file that it is taking a LONG time to download 4-6 hours of minutely diffs.  This usually took 5-10 minutes, but recently it takes 45 minutes to an hour.  And sometimes osmosis times out completely.

Here's my osmosis configuration.txt file if that helps.

# The URL of the directory containing change files.
baseUrl=https://planet.openstreetmap.org/replication/minute

# Defines the maximum time interval in seconds to download in a single invocation.
# Setting to 0 disables this feature.
maxInterval = 21600

Anyone else seeing slow minutely diff downloads?

Lynn (D)

PS.  Here's the munin rdnerd Data import lag daily and weekly graphs:

Data

Data


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Minutely update server slow?

Lynn W. Deffenbaugh (Mr)
I don't see how that relates to minutely updates on a personal tile server?

On 4/20/2019 2:13 PM, 80hnhtv4agou--- via talk wrote:
what about open street cam .org
 
Sent: Saturday, April 20, 2019 1:03 PM
Subject: [OSM-talk] Minutely update server slow?
 

I run a personal tile server for the entire planet and have it configured with minutely updates downloaded by osmosis.  Over the past few days, I've noticed that my updates are falling behind and subsequently noticed in the osmosis.log file that it is taking a LONG time to download 4-6 hours of minutely diffs.  This usually took 5-10 minutes, but recently it takes 45 minutes to an hour.  And sometimes osmosis times out completely.

Here's my osmosis configuration.txt file if that helps.

# The URL of the directory containing change files.
baseUrl=https://planet.openstreetmap.org/replication/minute

# Defines the maximum time interval in seconds to download in a single invocation.
# Setting to 0 disables this feature.
maxInterval = 21600

Anyone else seeing slow minutely diff downloads?

Lynn (D)

PS.  Here's the munin rdnerd Data import lag daily and weekly graphs:

Data

Data


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk



_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Minutely update server slow?

Lynn W. Deffenbaugh (Mr)
In reply to this post by Lynn W. Deffenbaugh (Mr)

Hm, traced the planet.openstreetmap.org back to ironbelly.openstreetmap.org and see this on their munin pages:

From https://munin.openstreetmap.org/openstreetmap.org/ironbelly.openstreetmap.org/index.html

Looks like something is ramping up in apache and TCP networking along with whatever Chrony's "Root dispersion" means.  Disk space usage is also climbing on the root file system, but that might be increasing apache logs?

Apache accesses

Disk usage in percent

Utilization per device

TCP

Chrony Tracking Stats

Lynn (D)

On 4/20/2019 2:03 PM, Lynn W. Deffenbaugh (Mr) wrote:

I run a personal tile server for the entire planet and have it configured with minutely updates downloaded by osmosis.  Over the past few days, I've noticed that my updates are falling behind and subsequently noticed in the osmosis.log file that it is taking a LONG time to download 4-6 hours of minutely diffs.  This usually took 5-10 minutes, but recently it takes 45 minutes to an hour.  And sometimes osmosis times out completely.

Here's my osmosis configuration.txt file if that helps.

# The URL of the directory containing change files.
baseUrl=https://planet.openstreetmap.org/replication/minute

# Defines the maximum time interval in seconds to download in a single invocation.
# Setting to 0 disables this feature.
maxInterval = 21600

Anyone else seeing slow minutely diff downloads?

Lynn (D)

PS.  Here's the munin rdnerd Data import lag daily and weekly graphs:

Data import lag

Data import lag


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Minutely update server slow?

Yves-2
In reply to this post by Lynn W. Deffenbaugh (Mr)
No problem on my server, I don't use Osmosis though.
Yves

Le 20 avril 2019 20:03:48 GMT+02:00, "Lynn W. Deffenbaugh (Mr)" <[hidden email]> a écrit :

I run a personal tile server for the entire planet and have it configured with minutely updates downloaded by osmosis.  Over the past few days, I've noticed that my updates are falling behind and subsequently noticed in the osmosis.log file that it is taking a LONG time to download 4-6 hours of minutely diffs.  This usually took 5-10 minutes, but recently it takes 45 minutes to an hour.  And sometimes osmosis times out completely.

Here's my osmosis configuration.txt file if that helps.

# The URL of the directory containing change files.
baseUrl=https://planet.openstreetmap.org/replication/minute

# Defines the maximum time interval in seconds to download in a single invocation.
# Setting to 0 disables this feature.
maxInterval = 21600

Anyone else seeing slow minutely diff downloads?

Lynn (D)

PS.  Here's the munin rdnerd Data import lag daily and weekly graphs:

Data import lag

Data import lag


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk