Importing NaPTAN Data [Thread 2]

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Importing NaPTAN Data [Thread 2]

Silent Spike
Since the other thread has become a tagging discussion on the pros and cons of PTv2, let's try this again.

Would there be any objections to an import of the following scope:
  1. Import data specifically for the Aberdeen admin area (ATCO code 639)
  2. Import stops of type BCT ("On-street Bus / Coach / Trolley Stop.")
  3. Import BCT stops of type MKD ("Marked (pole, shelter etc)")
  4. Manually conflate and review the data before upload using JOSM
  5. Split the edits up according to the "LocalityName" data field (basically districts)  [or is it better to be one changeset for the whole area?]
    Using the following established tags:
      • `highway=bus_stop`
      • `public_transport=platform`
      • `bus=yes`
      • `name=*` [Imported]
      • `naptan:AtcoCode=*` [Imported]
      • `naptan:NaptanCode=*` [Imported]
      • `source=naptan`
      • `naptan:verified=no` 
    Alternatively, if you support the proposal then it's also useful to know 😊

    Cheers

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Mateusz Konieczny-3

    5 Jul 2019, 09:04 by [hidden email]:
      • `naptan:AtcoCode=*` [Imported]
      • `naptan:NaptanCode=*` [Imported]
    I reformatted
    (added templates making them machine readable, descriptions will appear
    for example at Taginfo)

    Is it useful to use both? What is the difference between them?
    Is NaptanCode actually used as planned ("referring to the stop in public facing systems")?

    Alternatively, if you support the proposal then it's also useful to know 😊

    It may help to include example (of full) .osm data file to make easy for mappers
    to judge data quality in their area.

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Gareth L
    I’m certain I’ve seen “text this bus stop code to see next departures” use the naptan code. Whether or not that service is still live, I dunno.

    On 5 Jul 2019, at 10:17, Mateusz Konieczny <[hidden email]> wrote:


    5 Jul 2019, 09:04 by [hidden email]:
      • `naptan:AtcoCode=*` [Imported]
      • `naptan:NaptanCode=*` [Imported]
    I reformatted
    (added templates making them machine readable, descriptions will appear
    for example at Taginfo)

    Is it useful to use both? What is the difference between them?
    Is NaptanCode actually used as planned ("referring to the stop in public facing systems")?

    Alternatively, if you support the proposal then it's also useful to know 😊

    It may help to include example (of full) .osm data file to make easy for mappers
    to judge data quality in their area.
    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Stuart Reynolds
    Yes, both are still in use

    AtcoCode is the unique “ID” of the bus stop and is the key that is used in databases to identify the stop. It is therefore also the key that gets used in an AnnotatedStopRef within the TransXChange (timetable) data.

    NaptanCode is, in my view, unhelpfully named because in a NaPTAN database you would tend to think of it as the key. But it isn’t. It is certainly unique, but it has some constraints around which characters can follow other characters so that on (very old style, now) phones you didn’t have to push the same button twice in succession (and actually using a combination of letters that use the same key sequence also works). By and large it is text (except in Scotland, if I recall, where it is numeric) and is the code that you would send to the (charged for) 84628 “next departures” SMS service (Nextbuses). You can also use it online at nextbuses.mobi for free, although there you can also use the AtcoCode, confusingly. 

    For example, Southend Travel Centre stop A has the AtcoCode 15800720 (158 is the Southend prefix) and the NaptanCode soadjdaw (where soa is the Southend prefix). The next departures can be seen using  http://nextbuses.mobi/WebView/BusStopSearch/BusStopSearchResults/soadjdaw

    Both the SMS and Nexbuses services are still active, and not planned to be turned off any time soon.

    Regards,
    Stuart Reynolds
    for traveline south east and anglia

    On 5 Jul 2019, at 10:27, Gareth L <[hidden email]> wrote:

    I’m certain I’ve seen “text this bus stop code to see next departures” use the naptan code. Whether or not that service is still live, I dunno.

    On 5 Jul 2019, at 10:17, Mateusz Konieczny <[hidden email]> wrote:


    5 Jul 2019, 09:04 by [hidden email]:
      • `naptan:AtcoCode=*` [Imported]
      • `naptan:NaptanCode=*` [Imported]
    I reformatted
    (added templates making them machine readable, descriptions will appear
    for example at Taginfo)

    Is it useful to use both? What is the difference between them?
    Is NaptanCode actually used as planned ("referring to the stop in public facing systems")?

    Alternatively, if you support the proposal then it's also useful to know 😊

    It may help to include example (of full) .osm data file to make easy for mappers
    to judge data quality in their area.
    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb


    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    In reply to this post by Mateusz Konieczny-3
    On Fri, Jul 5, 2019 at 10:17 AM Mateusz Konieczny <[hidden email]> wrote:
    I reformatted
    (added templates making them machine readable, descriptions will appear
    for example at Taginfo)

    Thank you, documentation of the tags imported previously has definitely been lacking
     
    Is it useful to use both? What is the difference between them?
    Is NaptanCode actually used as planned ("referring to the stop in public facing systems")? 
     
    Of this I'm personally unsure, however there's not much harm in it and it seems standard enough practise
     
    It may help to include example (of full) .osm data file to make easy for mappers
    to judge data quality in their area.

    Here is a single file with all stops which I would be importing for the Aberdeen area (https://drive.google.com/open?id=1_QWn02Gi0YG8GCza0CeNQXtIKM_J82Y0), if split into subdivisions there are 63 areas in total within this.
     
    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    In reply to this post by Stuart Reynolds
    On Fri, Jul 5, 2019 at 10:52 AM Stuart Reynolds <[hidden email]> wrote:
    Yes, both are still in use

    -snip-

    Regards,
    Stuart Reynolds
    for traveline south east and anglia
     
    As someone more in the know, are there any fields of the NaPTAN data which you think should be imported which I haven't included?

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Stuart Reynolds
    From your original post, I didn’t think so, especially. 

    However, the fields in NaPTAN are designed to be granular, and while many may seem to be irrelevant to OSM (e.g. why have a street name when the street is named on the map) different local authorities will have different approaches to using that granularity for flag display. So the stop in NaPTAN might have a common name of “Stuart Close” with an indicator of “adj”, showing that the stop was adjacent to Stuart Close. However, street name becomes relevant if the local authority shows e.g. "High Street / Stuart Close” on the flag (with High Street being the street that the stop is physically located on).

    So it is a bit of a “suck it and see”, but in general I would suggest that, of the descriptive fields, you definitely want Common Name and Indicator, and you should consider including Streetname, Crossing and Landmark. In Wales and Gaelic-speaking parts of Scotland (including Ayr, apparently, where they are now erecting dual language signage) you should also include any non-English alternative names too. I’m less fussed about Locality, because the context of the map should give it, but sometimes child (lowest level) localities are not named on the map, or are named differently to the map.

    I certainly wouldn’t include the short names (which are usually just the abbreviations on on-street real time displays or fronts of buses) or the plate / cleardown etc codes.

    Note in passing that NaPTAN status often gets mis-used. Some authorities have a tendency to mark a stop as deleted when it no longer has services at it, even thought the infrastructure is still in place on the ground. These are supposed to be ACT but unused in NaPTAN, not deleted. If they are covered up, or physically removed, then they can be deleted. But not otherwise. You may come across some of these (or you may not).

    Hope that helps.

    Regards,
    Stuart Reynolds
    for traveline south east and anglia

    On 5 Jul 2019, at 10:57, Silent Spike <[hidden email]> wrote:

    On Fri, Jul 5, 2019 at 10:52 AM Stuart Reynolds <[hidden email]> wrote:
    Yes, both are still in use

    -snip-

    Regards,
    Stuart Reynolds
    for traveline south east and anglia
     
    As someone more in the know, are there any fields of the NaPTAN data which you think should be imported which I haven't included?

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb


    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    On Fri, Jul 5, 2019 at 11:35 AM Stuart Reynolds <[hidden email]> wrote:
    -snip-

    Hope that helps.

    Pretty much the perfect response, thank you!

    Good point about alternate names, will add support for those to my script now (maybe unnecessary for my area, but I'd like to share it in future). I see that many of the English alternate names are nearby landmarks or features (e.g. "The Crossroads"). Should I include these too as `loc_name` or similar (open question to all)?

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Gareth L
    Forgive me if this is silly question/statement, but the adj/alt names etc are in the naptan dataset. Wouldn’t it be better to have the link made between the stop in OSM and the record in naptan (using the codes prior mentioned).
    My thinking is data consumers could link and retrieve values using that. Merging the extra data values again might potentially develop discrepancies over time.

    I’d think the atco code is unique and (hopefully) not reused, but the alt names etc could be modified over time.


    Gareth

    On 5 Jul 2019, at 12:34, Silent Spike <[hidden email]> wrote:

    On Fri, Jul 5, 2019 at 11:35 AM Stuart Reynolds <[hidden email]> wrote:
    -snip-

    Hope that helps.

    Pretty much the perfect response, thank you!

    Good point about alternate names, will add support for those to my script now (maybe unnecessary for my area, but I'd like to share it in future). I see that many of the English alternate names are nearby landmarks or features (e.g. "The Crossroads"). Should I include these too as `loc_name` or similar (open question to all)?
    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    On Fri, Jul 5, 2019 at 1:02 PM Gareth L <[hidden email]> wrote:
    Forgive me if this is silly question/statement, but the adj/alt names etc are in the naptan dataset. Wouldn’t it be better to have the link made between the stop in OSM and the record in naptan (using the codes prior mentioned).
    My thinking is data consumers could link and retrieve values using that. Merging the extra data values again might potentially develop discrepancies over time.

    I’d think the atco code is unique and (hopefully) not reused, but the alt names etc could be modified over time.

    A perfectly valid question and something I wonder also.

    For example, the indicator field is somewhat meaningless to a data consumer unless they know what it represents as per the NaPTAN schema - so perhaps it's best left out of the OSM data?

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Stuart Reynolds
    When you say that

    the indicator field is somewhat meaningless to a data consumer 

    what did you have in mind? To the end-user of the data (i.e. someone wanting to know about where to catch the bus) this is absolutely critical.

    For example, in my previous post I face a suggestion of “Stuart Close” with an indicator of “adj”. There would usually be a second stop of the same common name, Stuart Close, perhaps with the indicator “opp”. Without understanding the NaPTAN schema, “app Stuart Close” and “adj Stuart Close” are understandable and are service direction dependent.

    Even more so in bus stations. The name Derby Bus Station (actually just "Bus Station” in the locality of Derby) applies equally to all 29 bays in the bus station. “Bay 1” through “Bay 29” are the indicators.

    Regards,
    Stuart Reynolds
    for traveline south east and anglia


    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Andy Townsend
    In reply to this post by Silent Spike
    On 05/07/2019 13:19, Silent Spike wrote:
    On Fri, Jul 5, 2019 at 1:02 PM Gareth L <[hidden email]> wrote:
    Forgive me if this is silly question/statement, but the adj/alt names etc are in the naptan dataset. Wouldn’t it be better to have the link made between the stop in OSM and the record in naptan (using the codes prior mentioned).
    My thinking is data consumers could link and retrieve values using that. Merging the extra data values again might potentially develop discrepancies over time.

    I’d think the atco code is unique and (hopefully) not reused, but the alt names etc could be modified over time.

    A perfectly valid question and something I wonder also.

    For example, the indicator field is somewhat meaningless to a data consumer unless they know what it represents as per the NaPTAN schema - so perhaps it's best left out of the OSM data?

    FWIW I did actually append that to name when displaying those because it "looked useful" (to append to the name and distinguish between other identically named stops):

    https://github.com/SomeoneElseOSM/SomeoneElse-style/blob/master/style.lua#L5010

    As to the original question "Would there be any objections to an import of the following scope" - certainly not from me.  I wouldn't personally use the PTv2 "platform" tag but its presence doesn't break anything.

    The most interesting bit would be the "Manually conflate and review the data before upload using JOSM" - any JOSM CSS style that you end up using to highlight duplicates would be really useful, as would a basic OSM diary entry describing the process and the end result.

    Best Regards,

    Andy





    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Ed Loach-2
    In reply to this post by Stuart Reynolds
    Stuart wrote:

    > Even more so in bus stations. The name Derby Bus Station (actually
    > just "Bus Station” in the locality of Derby) applies equally to all 29
    > bays in the bus station. “Bay 1” through “Bay 29” are the indicators.

    Agreed this is useful when adding the stops for the mapper, but unless we are going to keep updating the naptan information fields as well as the OSM fields, isn't the naptancode (or atcocode) sufficient to check the latest information? Take for example this bus stop, imported ages ago:
    https://www.openstreetmap.org/node/474893182#map=19/51.88949/0.89687&layers=T
    It has "naptan:Indicator"="Stop T" but "local_ref"="Fa" which might well be in the naptan indicator field now, but for a local mapper they are more likely to update local_ref to what is signposted when it changes than look at the naptan fields. And I don't think "opp" or "adj" belong in local ref.

    What has been useful when adding routes is stop bearing, but again I can check this in NaPTAN data from the reference - it should be obvious in OSM from looking at the map. However I did find a few sections of very roughly traced roads that were the wrong side of the stops when I was creating the route relations, which I used the stop bearing and aerial imagery to sort out.

    Also, I suspect some latitudes and longitudes have been refined in NaPTAN since stops were first imported. I was creating one route where the stops showed as in the middle, or behind, the buildings fronting the road.

    Ed


    ---
    This email has been checked for viruses by Avast antivirus software.
    https://www.avast.com/antivirus


    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    In reply to this post by Stuart Reynolds
    On Fri, Jul 5, 2019 at 1:33 PM Stuart Reynolds <[hidden email]> wrote:
    what did you have in mind? To the end-user of the data (i.e. someone wanting to know about where to catch the bus) this is absolutely critical.

    For example, in my previous post I face a suggestion of “Stuart Close” with an indicator of “adj”. There would usually be a second stop of the same common name, Stuart Close, perhaps with the indicator “opp”. Without understanding the NaPTAN schema, “app Stuart Close” and “adj Stuart Close” are understandable and are service direction dependent.

    Well I was just thinking they're often descriptive abbreviations (and admittedly aren't hard to deduce) with limited possible values as per the NaPTAN schema. However, if it's just following the NaPTAN schema is there a point in including it in OSM if that data is available and more up to date there? While tags such as the stop name have corresponding tags in OSM which have use to consumers who are not strictly following the NaPTAN schema.
     
    I'm certainly not against it's inclusion and would defer to your opinion that it's useful data.

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Stuart Reynolds
    In reply to this post by Andy Townsend
    If you wanted to conflate indicator and common name into the OSM “name” field, then I can provide a list of what we call “prefix indicators” as opposed to “suffix indicators”. It’s fairly self-explanatory, as prefix indicators are generally those that are positional e.g. o/s, opp, adj, nr etc. For example, Stuart Close / opp -> “opp Stuart Close", whereas the suffix ones are not e.g. Bus Station / Bay 8 -> "Bus Station (Bay 8)”

    Regards,
    Stuart Reynolds
    for traveline south east and anglia

    On 5 Jul 2019, at 13:40, Andy Townsend <[hidden email]> wrote:

    On 05/07/2019 13:19, Silent Spike wrote:
    On Fri, Jul 5, 2019 at 1:02 PM Gareth L <[hidden email]> wrote:
    Forgive me if this is silly question/statement, but the adj/alt names etc are in the naptan dataset. Wouldn’t it be better to have the link made between the stop in OSM and the record in naptan (using the codes prior mentioned).
    My thinking is data consumers could link and retrieve values using that. Merging the extra data values again might potentially develop discrepancies over time.

    I’d think the atco code is unique and (hopefully) not reused, but the alt names etc could be modified over time.

    A perfectly valid question and something I wonder also.

    For example, the indicator field is somewhat meaningless to a data consumer unless they know what it represents as per the NaPTAN schema - so perhaps it's best left out of the OSM data?

    FWIW I did actually append that to name when displaying those because it "looked useful" (to append to the name and distinguish between other identically named stops):

    https://github.com/SomeoneElseOSM/SomeoneElse-style/blob/master/style.lua#L5010

    As to the original question "Would there be any objections to an import of the following scope" - certainly not from me.  I wouldn't personally use the PTv2 "platform" tag but its presence doesn't break anything.

    The most interesting bit would be the "Manually conflate and review the data before upload using JOSM" - any JOSM CSS style that you end up using to highlight duplicates would be really useful, as would a basic OSM diary entry describing the process and the end result.

    Best Regards,

    Andy




    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb


    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Ed Loach-2
    In reply to this post by Stuart Reynolds
    I wrote:

    > https://www.openstreetmap.org/node/474893182#map=19/51.889
    > 49/0.89687&layers=T
    > It has "naptan:Indicator"="Stop T" but "local_ref"="Fa" which might
    > well be in the naptan indicator field now

    Perhaps a better example would be further along the road at
    https://www.openstreetmap.org/node/474893183
    where neither the current local_ref or name are in the imported but dated naptan fields.

    Ed


    ---
    This email has been checked for viruses by Avast antivirus software.
    https://www.avast.com/antivirus


    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    In reply to this post by Andy Townsend
    On Fri, Jul 5, 2019 at 1:41 PM Andy Townsend <[hidden email]> wrote:
    The most interesting bit would be the "Manually conflate and review the data before upload using JOSM" - any JOSM CSS style that you end up using to highlight duplicates would be really useful, as would a basic OSM diary entry describing the process and the end result.

    Would be happy to write an entry about my process if it comes to fruition. I suspect it won't be as interesting as you might imagine since there's really not a huge existing coverage of stops in the area. However, anything to help future mappers also interested in this available data.

    By downloading only bus stops in JOSM from the overpass API can easily see where duplicates exist just by proximity to the NaPTAN stops. Have considered expanding my script to also assist in conflation (marking possible duplicates for manual review before upload), but for the Aberdeen area it's probably not worth it.

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb
    Reply | Threaded
    Open this post in threaded view
    |

    Re: Importing NaPTAN Data [Thread 2]

    Silent Spike
    In reply to this post by Silent Spike
    On Fri, Jul 5, 2019 at 8:04 AM Silent Spike <[hidden email]> wrote:
    Using the following established tags:

    After all discussion so far I would update the list of tags to import as follows:
    • `highway=bus_stop`
    • `public_transport=platform`
    • `bus=yes`
    • `name` [Imported - defer to existing tagging during conflation]
    • `naptan:AtcoCode` [Imported]
    • `naptan:NaptanCode` [Imported]
    • `naptan:CommonName` [Imported - should match `name` but may differ, is what the indicator is relative to]
    • `naptan:Indicator` [Imported - useful to distinguish stops and sometimes can be `loc_ref` data]
    • `naptan:verified=no` [Gets deleted upon verification according to the wiki]

    I've checked and none of the stops here have Gaelic names (`name:gd`) - though my script now supports them for import anyway. I also feel like `source=naptan` is unnecessary since `naptan:verified` is basically the same thing and the changeset(s) can have a source tag.

    I'm trying to keep the amount of data somewhat minimal in line with the following quote from the imports guidelines wiki page:
    However, don't go overboard with metadata. OSM is only interested in what is verifiable. This doesn't include (for example) foreign keys from another database, unless those are absolutely necessary for maintaining the data in future. Your data source may have many many fields, but OSM data elements with many many tags can be difficult to work with. Strike a balance. Figure out (discuss!) what fields the OSM community are interested in.  

    _______________________________________________
    Talk-GB mailing list
    [hidden email]
    https://lists.openstreetmap.org/listinfo/talk-gb