OSM UK's first tile layer

Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

OSM UK's first tile layer

RobJN
Hi all,

Just in time for the AGM, I have just published OSM UK's first tile layer. No don't get too excited it is not a full map render. Instead I have produced a very simple tiling of the Land Registry polygon data now that this is under the OGL Open Data Licence. My view is that this is a good layer to align our mapping too - i.e. when tracing from imagery we should first align the imagery to the Land Registry polygon layer before tracing from the imagery.

The tile URL for JOSM is:
tms[13,17]:<a href="http://tiles.osmuk.org/LRpolygons/{zoom}/{x}/{y}.png">http://tiles.osmuk.org/LRpolygons/{zoom}/{x}/{y}.png

And for now this is York only as an example.

Feedback that I would appreciate:
  • Is this worthwhile?
  • Do you agree that it makes sense for us to all try to align our mapping to this (i.e. apply imagery offsets to align imagery to this before tracing)?
  • The style is very simple with just a 4 pixel red line. Is this sufficient? What changes can be made?
  • Any tips on how to keep the PNG file sizes as small as possible? For now I am using the Mapnik rule "png8:c=2:t=1:m=o". Is there anything that can yield smaller file sizes?
  • What max zoom is worthwhile? Currently it goes to 17, is this enough?
Our plan would be to pre-render all the tiles and host them on our site. The data doesn't change much so we would only re-render on request or once a year. My estimate is that we'd need 35GB for tiles to zoom level 17, and 133 GB to get everything to zoom 18. Our current server is on the small side with just 512MB memory and a 100GB disk allowance. It is unsuitable for on the fly rendering and we'd need more disk space to get the level 18 zoom. A beefier server is of course possible but any bump in specs comes with an equal bump in costs so worth checking this is worthwhile before proceeding.

P.S. The Land Registry themselves host this data on a WMS service rather than a TMS (tile) service. This makes it possible to zoom much further in. If you want to have a look at that detail you can use their website or (temporarily) use the following URL in JOSM. Please don't use this for mapping as we don't have permission to use their WMS service

wms:<a href="http://inspire.landregistry.gov.uk/inspire/ows?SERVICE=WMS&amp;FORMATS=image%2Fpng&amp;LAYERS=inspire%3ACP.CadastralParcel&amp;FORMAT=image%2Fpng&amp;TRANSPARENT=true&amp;VERSION=1.1.1&amp;REQUEST=GetMap&amp;STYLES=&amp;INFO_FORMAT=application%2Fvnd.ogc.gml&amp;EXCEPTIONS=XML&amp;_OLSALT=0.789927776902914&amp;SRS={proj}&amp;BBOX={bbox}&amp;WIDTH={width}&amp;HEIGHT={height}">http://inspire.landregistry.gov.uk/inspire/ows?SERVICE=WMS&FORMATS=image%2Fpng&LAYERS=inspire%3ACP.CadastralParcel&FORMAT=image%2Fpng&TRANSPARENT=true&VERSION=1.1.1&REQUEST=GetMap&STYLES=&INFO_FORMAT=application%2Fvnd.ogc.gml&EXCEPTIONS=XML&_OLSALT=0.789927776902914&SRS={proj}&BBOX={bbox}&WIDTH={width}&HEIGHT={height}

Best regards,
Rob

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

Jez Nicholson
Nice one. I've been wanting to do this for ages.

Re: file size. Can JOSM and iD display Mapbox .pbf vector tiles? These would be smaller.

On Sat, 17 Oct 2020, 00:22 Rob Nickerson, <[hidden email]> wrote:
Hi all,

Just in time for the AGM, I have just published OSM UK's first tile layer. No don't get too excited it is not a full map render. Instead I have produced a very simple tiling of the Land Registry polygon data now that this is under the OGL Open Data Licence. My view is that this is a good layer to align our mapping too - i.e. when tracing from imagery we should first align the imagery to the Land Registry polygon layer before tracing from the imagery.

The tile URL for JOSM is:

And for now this is York only as an example.

Feedback that I would appreciate:
  • Is this worthwhile?
  • Do you agree that it makes sense for us to all try to align our mapping to this (i.e. apply imagery offsets to align imagery to this before tracing)?
  • The style is very simple with just a 4 pixel red line. Is this sufficient? What changes can be made?
  • Any tips on how to keep the PNG file sizes as small as possible? For now I am using the Mapnik rule "png8:c=2:t=1:m=o". Is there anything that can yield smaller file sizes?
  • What max zoom is worthwhile? Currently it goes to 17, is this enough?
Our plan would be to pre-render all the tiles and host them on our site. The data doesn't change much so we would only re-render on request or once a year. My estimate is that we'd need 35GB for tiles to zoom level 17, and 133 GB to get everything to zoom 18. Our current server is on the small side with just 512MB memory and a 100GB disk allowance. It is unsuitable for on the fly rendering and we'd need more disk space to get the level 18 zoom. A beefier server is of course possible but any bump in specs comes with an equal bump in costs so worth checking this is worthwhile before proceeding.

P.S. The Land Registry themselves host this data on a WMS service rather than a TMS (tile) service. This makes it possible to zoom much further in. If you want to have a look at that detail you can use their website or (temporarily) use the following URL in JOSM. Please don't use this for mapping as we don't have permission to use their WMS service


Best regards,
Rob
_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

Robert Whittaker (OSM lists)
In reply to this post by RobJN
On Sat, 17 Oct 2020 at 00:22, Rob Nickerson <[hidden email]> wrote:
> Just in time for the AGM, I have just published OSM UK's first tile layer. No don't get too excited it is not a full map render. Instead I have produced a very simple tiling of the Land Registry polygon data now that this is under the OGL Open Data Licence. My view is that this is a good layer to align our mapping too - i.e. when tracing from imagery we should first align the imagery to the Land Registry polygon layer before tracing from the imagery.
>
> The tile URL for JOSM is:
> tms[13,17]:http://tiles.osmuk.org/LRpolygons/{zoom}/{x}/{y}.png

Excellent. Ever since the new Bing imagery landed, I've been after a
good source to align things to. I've been having to rely on my own GPS
traces and/or existing mapping so far.

By the way, when there was some previous discussion on this list about
using OS data for imagery alignment, an issue was raised about needing
to ensure any transformation from OSGB grid coordinates to WGS84 is
accurate enough:
https://lists.openstreetmap.org/pipermail/talk-gb/2020-August/025077.html
. Popular transforms may be out by a few meters (which would be
noticeable in our detailed mapping.) Are you doing such a
transformation, and are you sure what you're doing is sufficiently
accurate?

Robert.

--
Robert Whittaker

--
Robert Whittaker

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

ndrw6
In reply to this post by RobJN
Hi Rob,

Good stuff, it's definitely worthwhile. Thinner lines could work better
(for me 1px would be perfect), especially that the max zoom stops at 17.
You could perhaps consider increasing the max zoom a notch as well.

Are the numbers in wms tiles UPRNs? If so, you could consider displaying
them as well.

Best regards,

ndrw6

On 17/10/2020 00:20, Rob Nickerson wrote:

> Hi all,
>
> Just in time for the AGM, I have just published OSM UK's first tile
> layer. No don't get too excited it is not a full map render. Instead I
> have produced a very simple tiling of the Land Registry polygon data
> now that this is under the OGL Open Data Licence. My view is that this
> is a good layer to align our mapping too - i.e. when tracing from
> imagery we should first align the imagery to the Land Registry polygon
> layer before tracing from the imagery.
>
> The tile URL for JOSM is:
> tms[13,17]:http://tiles.osmuk.org/LRpolygons/{zoom}/{x}/{y}.png
>
> And for now this is _York only_ as an example.
>
> Feedback that I would appreciate:
>
>   * Is this worthwhile?
>   * Do you agree that it makes sense for us to all try to align our
>     mapping to this (i.e. apply imagery offsets to align imagery to
>     this before tracing)?
>   * The style is very simple with just a 4 pixel red line. Is this
>     sufficient? What changes can be made?
>   * Any tips on how to keep the PNG file sizes as small as possible?
>     For now I am using the Mapnik rule "png8:c=2:t=1:m=o". Is there
>     anything that can yield smaller file sizes?
>   * What max zoom is worthwhile? Currently it goes to 17, is this enough?
>
> Our plan would be to pre-render all the tiles and host them on our
> site. The data doesn't change much so we would only re-render on
> request or once a year. My estimate is that we'd need 35GB for tiles
> to zoom level 17, and 133 GB to get everything to zoom 18. Our current
> server is on the small side with just 512MB memory and a 100GB disk
> allowance. It is unsuitable for on the fly rendering and we'd need
> more disk space to get the level 18 zoom. A beefier server is of
> course possible but any bump in specs comes with an equal bump in
> costs so worth checking this is worthwhile before proceeding.
>
> P.S. The Land Registry themselves host this data on a WMS service
> rather than a TMS (tile) service. This makes it possible to zoom much
> further in. If you want to have a look at that detail you can use
> their website or (temporarily) use the following URL in JOSM. Please
> don't use this for mapping as we don't have permission to use their
> WMS service
>
> wms:http://inspire.landregistry.gov.uk/inspire/ows?SERVICE=WMS&FORMATS=image%2Fpng&LAYERS=inspire%3ACP.CadastralParcel&FORMAT=image%2Fpng&TRANSPARENT=true&VERSION=1.1.1&REQUEST=GetMap&STYLES=&INFO_FORMAT=application%2Fvnd.ogc.gml&EXCEPTIONS=XML&_OLSALT=0.789927776902914&SRS={proj}&BBOX={bbox}&WIDTH={width}&HEIGHT={height}
>
> Best regards,
> *Rob*
>
> _______________________________________________
> Talk-GB mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/talk-gb

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

RobJN
In reply to this post by Jez Nicholson
Hi all,

To address this and some of the other questions:

>Can JOSM and iD display Mapbox .pbf vector tiles?
I don't think JOSM can which is a big shame as it would make it much easier to host these sort of layers with minimal overhead.

>ensure any transformation from OSGB grid coordinates to WGS84 is accurate enough
Yes indeed. Currently I used QGIS to transform the data for York however I see that Mapnik can do the transform itself. For a complete GB dataset I intend to download all files and bring them together to pass to Mapnik. It probably then makes sense to have Mapnik do the transform. Any details on what transformation I should be using would be much appreciated. Likewise if you know a location that gets a big error if done wrong, that would help me check.

>Thinner lines could work better
Yes, I think so too.

>consider increasing the max zoom a notch as well
Hopefully the new OSM UK board can investigate that as it will need a beefier server so would push costs up.

>Are the numbers in wms tiles UPRNs?
My understanding is that they are not. That level of coordination would be amazing but we're not there with Open Data yet!

P.S. If anyone is aware of the equivalent data for Northern Ireland, Isle of Man or the Channel Islands, please let me know.

Best regards,
Rob


On Sat, 17 Oct 2020 at 08:16, Jez Nicholson <[hidden email]> wrote:
Nice one. I've been wanting to do this for ages.

Re: file size. Can JOSM and iD display Mapbox .pbf vector tiles? These would be smaller.

On Sat, 17 Oct 2020, 00:22 Rob Nickerson, <[hidden email]> wrote:
Hi all,

Just in time for the AGM, I have just published OSM UK's first tile layer. No don't get too excited it is not a full map render. Instead I have produced a very simple tiling of the Land Registry polygon data now that this is under the OGL Open Data Licence. My view is that this is a good layer to align our mapping too - i.e. when tracing from imagery we should first align the imagery to the Land Registry polygon layer before tracing from the imagery.

The tile URL for JOSM is:

And for now this is York only as an example.

Feedback that I would appreciate:
  • Is this worthwhile?
  • Do you agree that it makes sense for us to all try to align our mapping to this (i.e. apply imagery offsets to align imagery to this before tracing)?
  • The style is very simple with just a 4 pixel red line. Is this sufficient? What changes can be made?
  • Any tips on how to keep the PNG file sizes as small as possible? For now I am using the Mapnik rule "png8:c=2:t=1:m=o". Is there anything that can yield smaller file sizes?
  • What max zoom is worthwhile? Currently it goes to 17, is this enough?
Our plan would be to pre-render all the tiles and host them on our site. The data doesn't change much so we would only re-render on request or once a year. My estimate is that we'd need 35GB for tiles to zoom level 17, and 133 GB to get everything to zoom 18. Our current server is on the small side with just 512MB memory and a 100GB disk allowance. It is unsuitable for on the fly rendering and we'd need more disk space to get the level 18 zoom. A beefier server is of course possible but any bump in specs comes with an equal bump in costs so worth checking this is worthwhile before proceeding.

P.S. The Land Registry themselves host this data on a WMS service rather than a TMS (tile) service. This makes it possible to zoom much further in. If you want to have a look at that detail you can use their website or (temporarily) use the following URL in JOSM. Please don't use this for mapping as we don't have permission to use their WMS service


Best regards,
Rob
_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

Great Britain mailing list
The Ordnance Survey provides a transformation between OSGB36 and ETRS. It is described on this page https://www.ordnancesurvey.co.uk/gps/transformation/ and on the pages linked from there. The transformation is definitive. In other words, OSGB36 is redefined as being what you get when you apply the transformation to sets of ETRS co-ordinates. This must mean that if you compare an OS 1:1250 National Grid plan, with an older version of the plan from the era of OSTN02, features may have shifted slightly.

The transformation involves a mathematical transformation, and an adjustment based on a look-up table, to make the result match the errors in the old triangulation system. The OS provides applications to do the transformation, both ways, for a range of platforms. It also provides source code, the look-up table, and details of the mathematical transformation.

JOSM handles projections using proj. If you want to know what JOSM does with EPSG:27700, you need to know how it is defined in proj. The source code of JOSM includes the OSTN02 look-up table (15MB), but it can't be in the jar (also 15MB), so I don't know how that works.

Rob asked about position errors from the Helmert transformation without a look-up table. Here are some examples.
Larger errors
Place             error, m
St Kilda           4.9
Scilly             4.7
Lizard Point       4.1
Butt of Lewis      3.2
King's Lynn        2.7
Mallaig            2.6
Flamborough Head   2.4
Colchester         2.4
Plymouth           2.4
Nottingham         2.3
Anglesey           2.1
Northampton        2.0
North Foreland     1.9
Isle of Man S      1.9
Carmarthen         1.9
Smaller errors
St Catherine's Pt  1.4
Carlisle           0.8
Edinburgh          0.6
Aberdeen           1.8
Thurso             1.6
Orkney             1.0
Foula (Shetland)   1.2
The errors are particularly small near Bristol, Edinburgh and Fair Isle. They exceed 2m in South Devon, Cornwall, East Anglia, Nottinghamshire, Lincolnshire, the East Riding of Yorkshire, Pembrokeshire, Anglesey and Western Scotland.

+1 for referencing GB to ETRS.

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

Gareth Boyes
In reply to this post by RobJN
With regard to the discussion on the best transformation, OSM recommends that all transformations from OSGB36 to WGS84 should be carried out using a 3-parameter transformation which is accurate to 21 metres.  (https://wiki.openstreetmap.org/wiki/Converting_to_WGS84#Great_Britain). Is this still the recommended transformation?

There are 4 different transformations commonly used to transform from OSGB36 to WGS84//ETRS 89

The first method uses a 3-parameter transformation and is accurate up to 21m. (See https://epsg.org/transformation_1195/OSGB-1936-to-WGS-84-1.html)  This is the transformation recommended by OSM (https://wiki.openstreetmap.org/wiki/Converting_to_WGS84#Great_Britain)

The second uses a 7-parameter transformation (the Helmert Transformation) and is accurate up to 2m. (See  https://epsg.org/transformation_1314/OSGB-1936-to-WGS-84-6.html)

The third is OSTN02 which uses a NTv2 grid file and was accurate to centimetres but has now been superseded by OSTN15.

The fourth is OSTN15 which also uses a NTv2 grid file (OSTN15_NTv2_OSGBtoETRS.gsb) and is also accurate to centimetres.(See  https://epsg.org/transformation_7953/ETRS89-to-OSGB-1936-British-National-Grid-3.html)  

QGIS and other GIS software will generally use the OSTN15 (or OSTN02) transformation by default but only if they can find the NTv2 file. They will otherwise revert to the second 7-parameter transformation.

Using the correct transformation has implications for everyone wishing to import open data sets such as UPRN and NAPTAN data.  These data sets have their location data sourced in BNG (and are probably positioned with reference to OS Mastermap).  When these data sets are openly published a convenient Lat/Long is also provided that has been transformed/reprojected from the original BNG.   It should be noted that there is no consistency in the Lat/Longs that are provided by data providers, as they are free to use whichever transformation they choose.

It should also be noted that OSM is mapped in WGS84 because OSM is a global map and WGS84 is a global CRS.  The Ordnance Survey transformation on the other hand converts British National Grid (BNG) to ETRS89.  However, ETRS89 is a continental CRS which moves 2.5 cm every year with respect to WGS 84. That is a different can of worms best left unopened. 

Do OSM contributors use the 3-parameter transformation recommended on the OSM wiki page?  If they don’t,  should the wiki page be updated to reflect current practice. 

Gareth



_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

RobJN
In reply to this post by RobJN
Thanks for this.

I'm still confused as to what exactly I need but at least I can run some tests using these places. If it helps anyone, below is the xml we are using in Mapnik. The Map srs is from the OpenStreetMap default render and relates to the web projection. The Layer SRS is the standard for WGS84.

The intent going forward is to not transform from the british national grid to WGS84 before feeding in to Mapnik. Instead I will give Mapnik the raw data and the relevant Layer srs. I believe this is "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs" based on the link at https://spatialreference.org/ref/epsg/osgb-1936-british-national-grid/ I will test this but if you or others have feedback at this stage, please let me know.

<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over">

  <Style name="My Style">
    <Rule>
      <LineSymbolizer stroke="rgb(100%,0%,0%)" stroke-width="4" />
    </Rule>
  </Style>

  <Layer name="world" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>My Style</StyleName>
    <Datasource>
      <Parameter name="type">shape</Parameter>
      <Parameter name="file">york.shp</Parameter>
    </Datasource>
  </Layer>

</Map>

Rob


On Mon, 19 Oct 2020 at 20:02, Adrian <[hidden email]> wrote:
The Ordnance Survey provides a transformation between OSGB36 and ETRS. It is described on this page https://www.ordnancesurvey.co.uk/gps/transformation/ and on the pages linked from there. The transformation is definitive. In other words, OSGB36 is redefined as being what you get when you apply the transformation to sets of ETRS co-ordinates. This must mean that if you compare an OS 1:1250 National Grid plan, with an older version of the plan from the era of OSTN02, features may have shifted slightly.

The transformation involves a mathematical transformation, and an adjustment based on a look-up table, to make the result match the errors in the old triangulation system. The OS provides applications to do the transformation, both ways, for a range of platforms. It also provides source code, the look-up table, and details of the mathematical transformation.

JOSM handles projections using proj. If you want to know what JOSM does with EPSG:27700, you need to know how it is defined in proj. The source code of JOSM includes the OSTN02 look-up table (15MB), but it can't be in the jar (also 15MB), so I don't know how that works.

Rob asked about position errors from the Helmert transformation without a look-up table. Here are some examples.
Larger errors
Place             error, m
St Kilda           4.9
Scilly             4.7
Lizard Point       4.1
Butt of Lewis      3.2
King's Lynn        2.7
Mallaig            2.6
Flamborough Head   2.4
Colchester         2.4
Plymouth           2.4
Nottingham         2.3
Anglesey           2.1
Northampton        2.0
North Foreland     1.9
Isle of Man S      1.9
Carmarthen         1.9
Smaller errors
St Catherine's Pt  1.4
Carlisle           0.8
Edinburgh          0.6
Aberdeen           1.8
Thurso             1.6
Orkney             1.0
Foula (Shetland)   1.2
The errors are particularly small near Bristol, Edinburgh and Fair Isle. They exceed 2m in South Devon, Cornwall, East Anglia, Nottinghamshire, Lincolnshire, the East Riding of Yorkshire, Pembrokeshire, Anglesey and Western Scotland.

+1 for referencing GB to ETRS.

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

RobJN
Hi all,

A few more tiles added to help test the alignment. I've added the following locations:

    # Colchester  
    bbox = (0.8895, 51.8831, 0.9190, 51.8974)
    render_tiles(bbox, mapfile, tile_dir, 17, 17 , "Colchester")

    # Flamborough  
    bbox = (-0.132106, 54.102325, -0.111807, 54.121253)
    render_tiles(bbox, mapfile, tile_dir, 17, 17 , "Flamborough")

    # Stornoway  
    bbox = (-6.414124, 58.198201, -6.250257, 58.238161)
    render_tiles(bbox, mapfile, tile_dir, 17, 17 , "Stornoway")

    # Scilly  
    bbox = (-6.331023, 49.909491, -6.287335, 49.928125)
    render_tiles(bbox, mapfile, tile_dir, 17, 17 , "Scilly")

Just zoom level 17 for these areas for now.

These tiles were created without first converting from British National Grid to WGS 84 (lat/lon). Instead I pass Mapnik data in British National Grid projection and Mapnik transforms it to the web projection. The good news is that it hasn't altered the final result and that this still matches the result from the Land Registry's WMS service. So we can say that we are transforming the same way that their WMS service is.

The bad news however is that if I used the link Adrian provided [1] to transform to ETRS89 and compare to the coordinates that I can read out in JOSM, it does indeed look like we have an error. For example in Scilly, checking just one point, I get an adjustment of 4.8 meters which is almost identical to the 4.7m error that Adrian highlighted.

So in summary, more work is required. I will keep chipping away at this. Fortunately the Scottish data is published in both ETRS89 and British National Grid so I can check those in detail (assuming the Scottish data has been transformed correctly).

It does beg the question, what do we want to map to as our reference point? My knowledge on these matters is very limited and based on just one article [2]. My reading of this is that ETRS is a reference system for Europe but that if you want to adjust to the International version then you get the problem of the continental plate shifting 2.5cm per year. ETRS89 dates from 1989 so 31 years later we have a (31 x 2.5 = ) 77.5cm shift. Noticeable but nowhere near as bad as the 4.7m issue with the crude transformation from the british national grid.


Best regards,
Rob


On Fri, 23 Oct 2020 at 21:53, Rob Nickerson <[hidden email]> wrote:
Thanks for this.

I'm still confused as to what exactly I need but at least I can run some tests using these places. If it helps anyone, below is the xml we are using in Mapnik. The Map srs is from the OpenStreetMap default render and relates to the web projection. The Layer SRS is the standard for WGS84.

The intent going forward is to not transform from the british national grid to WGS84 before feeding in to Mapnik. Instead I will give Mapnik the raw data and the relevant Layer srs. I believe this is "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs" based on the link at https://spatialreference.org/ref/epsg/osgb-1936-british-national-grid/ I will test this but if you or others have feedback at this stage, please let me know.

<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over">

  <Style name="My Style">
    <Rule>
      <LineSymbolizer stroke="rgb(100%,0%,0%)" stroke-width="4" />
    </Rule>
  </Style>

  <Layer name="world" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>My Style</StyleName>
    <Datasource>
      <Parameter name="type">shape</Parameter>
      <Parameter name="file">york.shp</Parameter>
    </Datasource>
  </Layer>

</Map>

Rob


On Mon, 19 Oct 2020 at 20:02, Adrian <[hidden email]> wrote:
The Ordnance Survey provides a transformation between OSGB36 and ETRS. It is described on this page https://www.ordnancesurvey.co.uk/gps/transformation/ and on the pages linked from there. The transformation is definitive. In other words, OSGB36 is redefined as being what you get when you apply the transformation to sets of ETRS co-ordinates. This must mean that if you compare an OS 1:1250 National Grid plan, with an older version of the plan from the era of OSTN02, features may have shifted slightly.

The transformation involves a mathematical transformation, and an adjustment based on a look-up table, to make the result match the errors in the old triangulation system. The OS provides applications to do the transformation, both ways, for a range of platforms. It also provides source code, the look-up table, and details of the mathematical transformation.

JOSM handles projections using proj. If you want to know what JOSM does with EPSG:27700, you need to know how it is defined in proj. The source code of JOSM includes the OSTN02 look-up table (15MB), but it can't be in the jar (also 15MB), so I don't know how that works.

Rob asked about position errors from the Helmert transformation without a look-up table. Here are some examples.
Larger errors
Place             error, m
St Kilda           4.9
Scilly             4.7
Lizard Point       4.1
Butt of Lewis      3.2
King's Lynn        2.7
Mallaig            2.6
Flamborough Head   2.4
Colchester         2.4
Plymouth           2.4
Nottingham         2.3
Anglesey           2.1
Northampton        2.0
North Foreland     1.9
Isle of Man S      1.9
Carmarthen         1.9
Smaller errors
St Catherine's Pt  1.4
Carlisle           0.8
Edinburgh          0.6
Aberdeen           1.8
Thurso             1.6
Orkney             1.0
Foula (Shetland)   1.2
The errors are particularly small near Bristol, Edinburgh and Fair Isle. They exceed 2m in South Devon, Cornwall, East Anglia, Nottinghamshire, Lincolnshire, the East Riding of Yorkshire, Pembrokeshire, Anglesey and Western Scotland.

+1 for referencing GB to ETRS.

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

Great Britain mailing list
 I can confirm that the Land Registry wms parcels appear to have been converted with the Helmert 7-element transformation (no look-up table). This gives a misalignment of up to 5 metres. It's ironic that the Land Registry don't seem to know where their parcels are to better than 5m.

Now we know what EPSG:27700 does. It does the above transformation.

I agree with Rob that the misalignment of 5m is obvious if you look at Hugh Town (Scilly). Both if you compare with the OSM data and if you compare with the tracklogs that have been uploaded to OSM. So this transformation won't do. I think we need to go for the look-up table.

I've done some testing with JOSM. The look-up table transformation is not in JOSM's list of (thousands) of projections. But this custom projection does it:

+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +units=m +nadgrids=OSTN02_NTv2.gsb +bounds=-9,49,2,61 +no_defs

I expect something very similar would work in Mapnik.

When you set up this custom projection, JOSM downloads the grid file from the JOSM server, and puts it in the JOSM cache folder under a modified name. There is then a wait of several seconds while JOSM configures the custom projection. You can also get JOSM to do the latest, OSTN15 transformation. The only change needed, is to the grid file. This needs some simple hacking because it's not supported. You don't change the custom projection, but you alter the file in the cache folder. So, find the file, copy its name, and then delete it. Download the OSTN15 grid file from the OS website. As Gareth says, you need OSTN15_NTv2_OSGBtoETRS.gsb, (and not the other way round). Put the file in the cache folder and rename it to the name you just copied. You then need to quit and relaunch JOSM for this change to 'take'.

The difference between OSTN02 and OSTN15 is a shift, mostly in longitude, and in a similar direction throughout GB, of 1-2cm.

With the look-up table transformation, there will still be a misalignment of 0.75m relative to WGS84, but this is a lot better than 5m.

If there is consensus, then the wiki needs to be updated to recommend the OSTN15 transformation.

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb
Reply | Threaded
Open this post in threaded view
|

Re: OSM UK's first tile layer

James Derrick-2
Hi,

On 27/10/2020 06:18, Adrian via Talk-GB wrote:
I agree with Rob that the misalignment of 5m is obvious if you look at Hugh Town (Scilly). Both if you compare with the OSM data and if you compare with the tracklogs that have been uploaded to OSM. So this transformation won't do. I think we need to go for the look-up table.

Whilst the details of your geodesy is impressive but way beyond my expertise, over ten years of OSM survey traces suggests another factor to be wary of when comparing sub-10m position sources.

Using a Garmin Oregon 550 as a baseline, Oregon 650 and 750 consistently give location between 4-8m North North West in Northumberland - the tool may influence the measurement beyond your accuracy.

For resilience, I map with at least two GPSr on my bike handlebars and regularly upload both tracks to OSM and use both to better position mapping and any layers such as imagery. Over the years I've used five or six Garmin GPSr. None are even close to a 'proper' differential total station, however with datum/ spheroid set to WGS 84, the same tools and JOSM workflow show the offset. Changing GPS/ GLONASS or WAAS/ EGNOS seems to have less impact than the choice of Garmin unit (same settings across devices). Firmware updates have changed motion compensation when changing direction fast, but the offset remains.

The trouble will be is without device data in tracklogs there's no way to separate random from systematic offsets (even if you had them...) - you can only average all data.


Thanks for your interesting work - I remember tales from Registers of Scotland of an OS baseline survey error that 'moved' the East coast by many meters proving 'You Are Here' is hard to quantify!


James
-- 
James Derrick    
    [hidden email], Cramlington, England
    I wouldn't be a volunteer if you paid me...
    https://www.openstreetmap.org/user/James%20Derrick

_______________________________________________
Talk-GB mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk-gb