Hi,
PBF is well known for its compactness and better efficiency. No wonder it is also on the roadmap for the official API's map call (see [1]. That's reason enough to evaluate how PBF could also be integrated into Overpass API. Thanks to libosmium and Jochen's great support, I'm happy to present a first (incomplete) prototype: http://overpass-turbo.eu/s/jpK Be sure to click on Export -> raw data directly from Overpass API to avoid garbled output in overpass turbo. When trying your own examples, be sure to always use `out meta;`, all other modes are not implemented in this prototype. I tested pbf output for areas of roughly 300km by 300km, yielding about 500MB in PBF file size. (in turbo use: Load -> Examples -> Map Call) Along with PBF, libosmium also provides additional output formats like OPL (a text based representation), which is useful for command line tooling. We get this format basically for free: http://overpass-turbo.eu/s/jpL Note: This feature is available for testing on the following endpoint only: http://dev.overpass-api.de/test753/ Best, mmd [1] http://www.asklater.com/matt/blog/2015/11/18/the-road-to-api-07/ |
Great mmd For Humanitarian OSM Responses, we often have to extract a lot of data either to validate, product extracts or build statistics. Compression of the data should greatly help, especially for teams in the field with limited internet access. I tested extracting highways for south-west of Haiti. I then succeeded to import in QGIS. One great option would be to have a file name field. It could be something like [out:pbf="filename-xxx.pbf"]. It is not easy to indentify + determine extension of files named interpreter in directory file list. Great thanks to the Overpass team for such constant improvements. Pierre De : mmd <[hidden email]> À : [hidden email] Envoyé le : lundi 17 octobre 2016 12h59 Objet : [overpass] PBF output - prototype available for testing Hi, PBF is well known for its compactness and better efficiency. No wonder it is also on the roadmap for the official API's map call (see [1]. That's reason enough to evaluate how PBF could also be integrated into Overpass API. Thanks to libosmium and Jochen's great support, I'm happy to present a first (incomplete) prototype: http://overpass-turbo.eu/s/jpK Be sure to click on Export -> raw data directly from Overpass API to avoid garbled output in overpass turbo. When trying your own examples, be sure to always use `out meta;`, all other modes are not implemented in this prototype. I tested pbf output for areas of roughly 300km by 300km, yielding about 500MB in PBF file size. (in turbo use: Load -> Examples -> Map Call) Along with PBF, libosmium also provides additional output formats like OPL (a text based representation), which is useful for command line tooling. We get this format basically for free: http://overpass-turbo.eu/s/jpL Note: This feature is available for testing on the following endpoint only: http://dev.overpass-api.de/test753/ Best, mmd [1] http://www.asklater.com/matt/blog/2015/11/18/the-road-to-api-07/ |
Am 17.10.2016 um 20:51 schrieb Pierre Béland: > For Humanitarian OSM Responses, we often have to extract a lot of data > either to validate, product extracts or build statistics. Compression of > the data should greatly help, especially for teams in the field with > limited internet access. > > I tested extracting highways for south-west of Haiti. I then succeeded > to import in QGIS. Thank you for the positive feedback! I hope Roland won't object the use of libosmium as an 'include-only' library in this case, given that we can also leverage it to quickly set up a new database using a .pbf planet. Like we did for lz4 compression, this could also be an optional feature. Anyway, the main reason for making this a fairly minimum prototype has to do with ongoing restructuring of the code Roland is currently working on. Once that task is completed, we will probably revisit this PBF topic. > One great option would be to have a file name field. It could be > something like > [out:pbf="filename-xxx.pbf"]. It is not easy to indentify + determine > extension of files named interpreter in directory file list. I believe, we already have an issue for it (https://github.com/drolbr/Overpass-API/issues/153), if you remember ;) -- |
mmd <[hidden email]> wrote > I believe, we already have an issue for it > (https://github.com/drolbr/Overpass-API/issues/153), if you remember ;) One more reason if more then one extension ;) -- |
Free forum by Nabble | Edit this page |