Re: dev Digest, Vol 188, Issue 1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: dev Digest, Vol 188, Issue 1

Seth Deegan
Oh wow. So Data Items basically is what I'm proposing. I also didn't know it also had a role in the Wikibase Registry (probably wouldn't want to get rid of that).

I guess my only issues with the current system would be:
1. Lack of quality control (it said in the Wiki page for Data items that it has not been implemented yet). Quality control should probably apply to the corresponding Wiki pages themselves too.
2. Lack of synchronization between the Data items and their Wiki pages.
3. JOSM doesn't use Data items yet. 
4. The MediaWiki interface is still hard for new users to grasp.
5. Layouts for Wiki pages aren't standardized. Something else my proposal could address would be the conversion of Proposed Feature pages to actual Wiki pages (would require a standardization) but honestly it is not the most important thing.
6. Devs still have to manually add presets. One of my first issues on the iD repo was actually concerning this (btw uneducated at how approved features worked at the time): https://github.com/openstreetmap/iD/issues/7552 

On Sun, Nov 15, 2020 at 4:01 PM <[hidden email]> wrote:
Send dev mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.openstreetmap.org/listinfo/dev
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of dev digest..."


Today's Topics:

   1. Standardized feature/tag database idea & proposal (Seth Deegan)
   2. Re: Standardized feature/tag database idea & proposal
      (Mateusz Konieczny)


----------------------------------------------------------------------

Message: 1
Date: Sun, 15 Nov 2020 12:14:19 -0600
From: Seth Deegan <[hidden email]>
To: [hidden email]
Subject: [OSM-dev] Standardized feature/tag database idea & proposal
Message-ID:
        <CAAk9NOJqNPhb1OvrRrXLB-gEXPH9BxQbSxeM33ed=[hidden email]>
Content-Type: text/plain; charset="utf-8"

*Idea*
Has anyone ever thought of creating an official database that stores all of
the approved and in-use tags/features in OSM?

*Reasoning*
This could allow the editors (iD and JOSM, StreetComplete, GoMap!) and data
consumers (osm-carto,. Mapbox etc.), to easily stay up-to-date with new
features, without requiring their developers to browse the Wiki etc.

*Examples*
Both iD and JOSM have their own preset file/repo and are independent of
each other. Creating a database that could be used as a dependency of both
that would store these feature presets and their fields means:
 - Both are up-to-date and in-sync
 - Adding new presets could be done automatically by retrieving and parsing
data from the DB.

Creating applications that use OSM data can be hard and time-lengthened by
requiring developers to browse the Wiki to find all of the features and
their keys and values. Having a database that they could easily get the
keys they want, their values, etc. would *significantly* allow greater OSM
developer potential.

*Specifics*
The specifics as to how this database would be arranged such as to where
presets/fields/tags/features go has not been thought of yet. I just wanted
to ask if this has ever been proposed before. If someone would like me to
make a DB layout to help them better understand what I'm proposing, I'd be
happy to do so.

*My pre-DB construction proposal *
Before any type of database is made, one would would *construct a dedicated
page structure on openstreetmap.org <http://openstreetmap.org> *that would
display and be in-sync with all of the Features from the DB in a format
similar to the Wiki, and then *remove all of the features from the Wiki
altogether.*

*Why?* If you see in *Compiling and distributing the DB* below, a Wiki bot
is going to be required to get all of the pages for all of the
already-standard features. Most of the features' pages do have a similar
structure as to how they are arranged (template that shows what elements
they use, Keys' values and their descriptions are stored in a table, etc.),
however these pages would be *impossible* to parse thoroughly with a
computer and are going to require team of humans to get all of their data.

New features that are proposed and approved after the database would be
created would require them to be hand-added. Making a standard way to
propose and approve new features on a new part of the website with
formatting constraints and database-syncing abilities that MediaWiki cannot
offer, would mean that the DB would never have to be physically touched
again and ensure greater long-term DB management efficiency.

*So why basically delete one of the primary functions of the Wiki and
create a new system on openstreetmap.org <http://openstreetmap.org>?*
I recently have got into the OSM community head-on in the past two months.
I have never really contributed to Wikipedia or any other site that uses
the MediaWiki website format so learning about how to contribute and get
around the OSM Wiki took time (learning about the proposal process,
Templates, talk pages, formatting, the list goes on and on...). It also
made me realize that this could discourage new users from ever looking into
the depths of OSM or even finding the Wiki at all. The WikiMedia interface
is not the prettiest either. It can take time for new users to explore and
find what they are looking for.
(Also, this would mean moving the proposal process over to osm.org too)

Therefore, I think creating a easily accessible, pretty, and
easily-contributable interface on osm.org would strengthen the OSM
community significantly. For example, a heading called "Features" could be
added to the left "GPS traces". It would greet the user with the primary
features of OSM (kind of like the "Map features" on the Wiki already does)
and Feature pages would have a standard format that is consistent
throughout the website (unlike Wiki pages where tables for values can be
different, proposals can be different, etc.). Pages could also be "locked",
or require a proposal before ever changing any of the contents of their
page/feature. This would ensure the DB is secure and uniform with the
community's agreement on Features (the database is directly synced and
edited through changes on the website) and no "random edits' by users like
on the current Wiki would have to tracked (since anyone can directly edit).
There are other possible benefits that you could probably think of.

Also, other pages on the Wiki would not be deleted. There are plenty of
great pages on it that have nothing to do with the DB and work well in the
open environment the Wiki has to offer.

*Compiling and distributing the DB*

   1. One would probably use a Wiki bot like
   https://en.wikipedia.org/wiki/Wikipedia:AutoWikiBrowser to get all of
   the features with Categories "approved" and "in-use" (and "depreciated" as
   well just to let possible future editors know what to get rid of) and add
   their Keys and Values, descriptions, what elements they are allowed on
   (nodes, ways, areas, relations), etc. to the DB.
   2. A team would have to go through all of their Key Values that don't
   have Wiki pages and add them to the DB along with their descriptions, etc.
   3. A team would compare the iD and JOSM present repo and xml file and
   create a "standard" matching list.
   4. When the DB is finished, iD and JOSM would use it as a dependency and
   an announcement would be made to data consumers that it is available for
   use.

*Feasibility*
This would probably be a huge undertaking and require a funding grant. A
plan and dedicated team would be required to ensure all of the tasks are
complete and put in-place. I personally think the benefits outweigh the
costs, specifically from a developer point-of view. Also, I am not an OSMF
member so I'm not sure how much say I would have in making this possible.

*Questions?*
Please reply. I like big OSM ideas and am not afraid to get shut-down as I
have before with previous ideas. I am clearly a newer contributor, but I
hope my ideas and work can foster progress for OSM.

--
Thanks,
Seth Deegan (lectrician1)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20201115/a53d3e3b/attachment-0001.htm>

------------------------------

Message: 2
Date: Sun, 15 Nov 2020 22:59:47 +0100 (CET)
From: Mateusz Konieczny <[hidden email]>
Cc: Dev <[hidden email]>
Subject: Re: [OSM-dev] Standardized feature/tag database idea &
        proposal
Message-ID: <[hidden email]>
Content-Type: text/plain; charset="utf-8"




Nov 15, 2020, 19:14 by [hidden email]:

> Idea
> Has anyone ever thought of creating an official database that stores all of the approved and in-use tags/features in OSM? 
>
Yes.

Depending on what one means by "database" OSM Wiki can be considered as one.

Wikidata copy ( https://wiki.openstreetmap.org/wiki/Data_items ) is definitely one.
How your proposal differs from data items?

>
> Reasoning
> This could allow the editors (iD and JOSM, StreetComplete, GoMap!) and data consumers (osm-carto,. Mapbox etc.), to easily stay up-to-date with new features, without requiring their developers to browse the Wiki etc.
>
I see no benefit whatsoever of data items (from perspective of someone involved in making
StreetComplete and was active in osm-carto development).

Browsing Wiki, Taginfo, reviewing how tags are actually used and so on would be definitely needed.

>
> Examples
> Both iD and JOSM have their own preset file/repo and are independent of each other. Creating a database that could be used as a dependency of both that would store these feature presets and their fields means:
>  - Both are up-to-date and in-sync
>  - Adding new presets could be done automatically by retrieving and parsing data from the DB. 
>
There is a good reason why presets are created manually and not pulled from unreviewed
dataset (note that JOSM and iD have separate presets despite that pulling from another preset
would be technically possinle)

> Creating applications that use OSM data can be hard and time-lengthened by requiring developers to browse the Wiki to find all of the features and their keys and values. Having a database that they could easily get the keys they want, their values, etc. would > significantly>  allow greater OSM developer potential.
>
I am unsure what would be difference.

> Specifics
> The specifics as to how this database would be arranged such as to where presets/fields/tags/features go has not been thought of yet. I just wanted to ask if this has ever been proposed before. If someone would like me to make a DB layout to help them better understand what I'm proposing, I'd be happy to do so. 
>
> My pre-DB construction proposal 
>
> Before any type of database is made, one would would > construct a dedicated page structure on > openstreetmap.org <http://openstreetmap.org>>  > that would display and be in-sync with all of the Features from the DB in a format similar to the Wiki, and then > remove all of the features from the Wiki altogether.
>
> Why?>  If you see in > Compiling and distributing the DB>  below, a Wiki bot is going to be required to get all of the pages for all of the already-standard features. Most of the features' pages do have a similar structure as to how they are arranged (template that shows what elements they use, Keys' values and their descriptions are stored in a table, etc.), however these pages would be > impossible>  to parse thoroughly with a computer and are going to require team of humans to get all of their data. 
>
> New features that are proposed and approved after the database would be created would require them to be hand-added. Making a standard way to propose and approve new features on a new part of the website with formatting constraints and database-syncing abilities that MediaWiki cannot offer, would mean that the DB would never have to be physically touched again and ensure greater long-term DB management efficiency.
>
> So why basically delete one of the primary functions of the Wiki and create a new system on > openstreetmap.org <http://openstreetmap.org>> ?
> I recently have got into the OSM community head-on in the past two months. I have never really contributed to Wikipedia or any other site that uses the MediaWiki website format so learning about how to contribute and get around the OSM Wiki took time (learning about the proposal process, Templates, talk pages, formatting, the list goes on and on...). It also made me realize that this could discourage new users from ever looking into the depths of OSM or even finding the Wiki at all. The WikiMedia interface is not the prettiest either. It can take time for new users to explore and find what they are looking for.
> (Also, this would mean moving the proposal process over to > osm.org <http://osm.org>>  too)
>
> Therefore, I think creating a easily accessible, pretty, and easily-contributable interface on > osm.org <http://osm.org>>  would strengthen the OSM community significantly. For example, a heading called "Features" could be added to the left "GPS traces". It would greet the user with the primary features of OSM (kind of like the "Map features" on the Wiki already does) and Feature pages would have a standard format that is consistent throughout the website (unlike Wiki pages where tables for values can be different, proposals can be different, etc.). Pages could also be "locked", or require a proposal before ever changing any of the contents of their page/feature. This would ensure the DB is secure and uniform with the community's agreement on Features (the database is directly synced and edited through changes on the website) and no "random edits' by users like on the current Wiki would have to tracked (since anyone can directly edit). There are other possible benefits that you could probably think of.
>
> Also, other pages on the Wiki would not be deleted. There are plenty of great pages on it that have nothing to do with the DB and work well in the open environment the Wiki has to offer.
>
> Compiling and distributing the DB
> One would probably use a Wiki bot like > https://en.wikipedia.org/wiki/Wikipedia:AutoWikiBrowser>  to get all of the features with Categories "approved" and "in-use" (and "depreciated" as well just to let possible future editors know what to get rid of) and add their Keys and Values, descriptions, what elements they are allowed on (nodes, ways, areas, relations), etc. to the DB.
> A team would have to go through all of their Key Values that don't have Wiki pages and add them to the DB along with their descriptions, etc.
> A team would compare the iD and JOSM present repo and xml file and create a "standard" matching list.
> When the DB is finished, iD and JOSM would use it as a dependency and an announcement would be made to data consumers that it is available for use.
> Feasibility
> This would probably be a huge undertaking and require a funding grant. A plan and dedicated team would be required to ensure all of the tasks are complete and put in-place. I personally think the benefits outweigh the costs, specifically from a developer point-of view. Also, I am not an OSMF member so I'm not sure how much say I would have in making this possible.
>
> Questions?>  
> Please reply. I like big OSM ideas and am not afraid to get shut-down as I have before with previous ideas. I am clearly a newer contributor, but I hope my ideas and work can foster progress for OSM.
>
> --
> Thanks,
> Seth Deegan (lectrician1)
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20201115/f317602e/attachment.htm>

------------------------------

Subject: Digest Footer

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev


------------------------------

End of dev Digest, Vol 188, Issue 1
***********************************


--
Thanks,
Seth

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: dev Digest, Vol 188, Issue 1

Developer Discussion mailing list
(0) if you want to reply it would be better to disable digest mode

Nov 16, 2020, 01:21 by [hidden email]:
Oh wow. So Data Items basically is what I'm proposing. I also didn't know it also had a role in the Wikibase Registry (probably wouldn't want to get rid of that).

I guess my only issues with the current system would be:
1. Lack of quality control (it said in the Wiki page for Data items that it has not been implemented yet). Quality control should probably apply to the corresponding Wiki pages themselves too.
Wiki pages are relatively well watched (warning: I am biased here,
as I am one of people quite active in that field).

Data items are not, as watchlisting them is dysfunctional (no way to skip
translations that given person cannot review, no way of grouping edits, standard edit
leave no description of edit and so on)
2. Lack of synchronization between the Data items and their Wiki pages.
This sort of works and is causing issues due to lack of control in Data Items
3. JOSM doesn't use Data items yet. 
I am dubious is it useful or going to happen, but it is up to JOSM developers
4. The MediaWiki interface is still hard for new users to grasp.
And note, that even if data items interface would be very easy - you still need
wiki pages for freeform description/comments
5. Layouts for Wiki pages aren't standardized. Something else my proposal could address would be the conversion of Proposed Feature pages to actual Wiki pages (would require a standardization) but honestly it is not the most important thing.
6. Devs still have to manually add presets. One of my first issues on the iD repo was actually concerning this (btw uneducated at how approved features worked at the time): https://github.com/openstreetmap/iD/issues/7552 
I am dubious whatever authors of editors would abandon maintaining presets and switch to
simply pulling from Data Items

Note that, for example, JOSM and iD are using icons in different styles.



_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev