pgsql2osm software

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

pgsql2osm software

François Lacombe-2
Hi everyone,

DCbrain, the company I currently work for, is a software editor involved in utilities, industry and supply chains optimisers development.
As a relevant and established practice, we use OSM data and tools to make calculations and mapping plots for dataviz or dashboarding.
Here comes the time to be thankful and share some free code too.

DCbrain, Maxime Beauchene and I now release our own implementation of a pgsql2osm conversion process under GNU GPL v3 licence.
pgsql2osm allows to produce osm xml out of postgis geometries, linestrings and points.
This implementation runs sql code to transform geometries to topological postgresql tables, manage way connections with its own layers and Python to output actual OSM XML.
It is currently used in production to convert regional areas with ~1M ways and 7M nodes on a 4 CPU/16 GB server and on a monthly basis.
https://github.com/DCbrainOrg/pgsql2osm

It's was way easier for us to do so to feed OSM software with custom postgis data.
We provide it as-is without any warranty.

Some improvements will enable to process more data but currently it's mainly for regional processing.
It's certainly also possible to rework a bit the sql logic to convert geometries to topological records more quickly.

Feel free to give it a try and let us know about problems you may have on GitHub repository issues.

All the best

François


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: pgsql2osm software

Sebastiaan Couwenberg
Thanks for sharing your software.

Looking through the source I noticed some code smells like creating XML
as strings instead of using lxml and not quoting variables used in SQL.
flake8 also reports several issues with convert2osm.py.

One also wonders how this compares to ogr2osm which can be used for the
same purpose. Have you tried using that before deciding to write this?

Kind Regards,

Bas

--
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: pgsql2osm software

François Lacombe-2
Hi Sebastiaan,

Le lun. 6 avr. 2020 à 13:39, Sebastiaan Couwenberg <[hidden email]> a écrit :
Thanks for sharing your software.

Looking through the source I noticed some code smells like creating XML
as strings instead of using lxml and not quoting variables used in SQL.
flake8 also reports several issues with convert2osm.py.
 
Thank you for your time and these issues.
We'll certainly fix this in a next relase, at least for SQL.
Regarding Python, ogr2osm may replace it as discussed below.
 

One also wonders how this compares to ogr2osm which can be used for the
same purpose. Have you tried using that before deciding to write this?

At the time we have started to set up our implementation, ogr2osm didn't allow to manage how connections are done.
We had need to prevent some geometries to connect and that's why additional tags like pgsql2osm:layer exists.
Because this kind of information isn't contained in geometries actually.

As I understand how ogr2osm works (maybe mistaken), it splits geometries in small segments and then connect them depending on overlapping nodes.
We also had a need to preserve geometries shapes.

However, that would worth a try to use ogr2osm instead of our python script to convert data once topology produced according to 2 needs upside.

All the best

François




_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev