> To be fair, I am not sure this is even related to tile expiration. I
> have not tried 0.96 updates without tile expiration as a baseline.
After 3 hours, the same update has reached the same state of progress.
osm2pgsql (debian backport) 0.96 maintains < 1GB of memory usage without
expiring tiles. While expiring tiles, it ballooned to >22GB before
Seems the issue is specific to tile-expiration overhaul.
Am 30.05.19 um 00:09 schrieb j:
> I note that initial testing of this code used an extract of Europe:
> https://github.com/openstreetmap/osm2pgsql/pull/747 >
> I did not note it earlier, but I am working with the planet. I doubt
> these issues would present themselves on a modestly sized machine
> working with Europe only.
The tests were done with Europe only which is roughly 2/3 of the whole
planet. The main difference to your test is that my test applied the
diff of a single day from download.geofabrik.de. Diffs from
download.geofabrik.de are a pure diff of the old and the new file.
I will have a deeper look into this but my test machine is currently
busy doing other work.
Michael Reichert www.geofabrik.de
Geofabrik GmbH Handelsregister: HRB Mannheim 703657
Amalienstr. 44 Geschaeftsfuehrung: C. Karch, F. Ramm
76133 Karlsruhe Tel: 0721-1803560-3
[hidden email] Fax: 0721-1803560-9
> The tests were done with Europe only which is roughly 2/3 of the whole
I did not realize Europe accounted for such a large portion of the OSM
data! I did notice that your test reported a 10GB memory usage for the
I did not pay attention to the memory usage for 0.92 imports, but tile
expiration seems to be requiring significant memory as of 0.96. My own
test hit 22GB before I ^C'd. 21GB more than required w/o expiration.
> I will have a deeper look into this but my test machine is currently
> busy doing other work.
My own dev box is very busy for the next couple of weeks, but I can
spin something up for testing purposes if needed.