Migrating osm.org to vectors/Kartotherian

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Migrating osm.org to vectors/Kartotherian

Yuri Astrakhan
At the State of the Map conference, some OSM developers were discussing the possibility of  migrating osm.org to the vector-based backend. I would like to get some feedback on feasibility and desirability of this effort.

Kartotherian, the Mapnik+Mapbox-based vector service has been implemented and trial-launched at Wikipedia. The service itself is fairly stable, but the styles can use some improvements - both the sql->vtile and vtile->image. Hopefully this work can be used as the basis for the osm.org style. Once the vtiles are ready, we can easily move to client side WebGL rendering.

P.S. Paul Norman has applied for a grant to help us with style work. Please add your support and comments to his proposal.

Thanks!

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Tom Hughes-3
On 30/10/15 01:11, Yuri Astrakhan wrote:

> At the State of the Map conference, some OSM developers were discussing
> the possibility of  migrating osm.org <http://osm.org> to the
> vector-based backend. I would like to get some feedback on feasibility
> and desirability of this effort.

Well I think everybody thinks it's desirable and will happen at some
point now that there are various options starting to mature as far as
underlying technology stacks go.

Really though it's up to the stylesheet team - they're the ones that are
going to have to do a lot of work and only once we have a style would we
need to consider what stack to run it on.

Tom

--
Tom Hughes ([hidden email])
http://compton.nu/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Christoph Hormann
In reply to this post by Yuri Astrakhan
On Friday 30 October 2015, Yuri Astrakhan wrote:

>
> Kartotherian <https://github.com/kartotherian/kartotherian>, the
> Mapnik+Mapbox-based vector service has been implemented and
> trial-launched <https://maps.wikimedia.org/> at Wikipedia. The
> service itself is fairly stable, but the styles can use some
> improvements - both the sql->vtile
> <https://github.com/kartotherian/osm-bright.tm2source> and
> vtile->image <https://github.com/kartotherian/osm-bright.tm2>.
> Hopefully this work can be used as the basis for the osm.org style.
> Once the vtiles are ready, we can easily move to client side WebGL
> rendering.

From my perspective this, i.e. imposing a certain technological
framework on designers based on technological considerations, is the
wrong approach.  I wrote about this on my blog recently from a slightly
different angle[1].  For a high quality style, design development has
to mandate the technological framework, not the other way round.

If you look at design problems recently discussed in the osm-carto style
development[2] you will see most of them have nothing to do with vector
tiles, they would not be made any easier to address with such an
approach.  On the other hand there are a multitude of things the
current style handles fairly gracefully, especially the problem of
reducing geometric complexity, that would be much harder to deal with
in a vector tiles system[3].

In general it seems to me vector tiles are today often carried as some
kind of religious mantra promising to be the solution of all problems
while in reality they certainly are not.  It is better to look for and
identify actual design problems and see what technological means are
available to solve them.  So far use of vector tiles seem to primarily
have lead to the following effects:

- improved tile serving efficiency
- a larger bandwidth of style variations
- tighter contraints in basic styling decisions beyond what is already
imposed by the OSM data model

In short: from a design perspective vector tiles so far brought more
variety in map styles but they ultimately all look very similar beyond
superficial aspects.  Nearly all of the more unusual maps that
currently exist are not vector tile based.

[1] http://blog.imagico.de/map-design-economics/
[2] https://github.com/gravitystorm/openstreetmap-carto
[3] For another example of where wikimedia maps fails miserably here
see:
https://maps.wikimedia.org/#18/47.99579/7.85194
http://www.openstreetmap.org/#map=18/47.99579/7.85194

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

dieterdreist

2015-10-30 10:52 GMT+01:00 Christoph Hormann <[hidden email]>:
So far use of vector tiles seem to primarily
have lead to the following effects:

- improved tile serving efficiency
- a larger bandwidth of style variations
- tighter contraints in basic styling decisions beyond what is already
imposed by the OSM data model


some additional points that come into mind:

- vector maps are consuming more energy to display (because have to be calculated/rendered) -> problem on mobile devices, but also generally a problem because every client has to spend energy on calculating the "same" image (admittedly depends how many different styles there are, and how many people are looking at them until the underlying data changes), so globally (i.e. wikipedia use and not some "niche" usecases) this means really a lot of wasted energy.

- are likely slower to display (for the same reason), although this is depending on different factors (e.g. if you have internet connection and how fast it is (vector maps likely scale better for offline use), how complex the style sheet is, etc.) -> generally vector maps require more ressources, newer more capable client devices.

Cheers,
Martin

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Christoph Hormann
On Friday 30 October 2015, Martin Koppenhoefer wrote:
> >
> > - improved tile serving efficiency
> > - a larger bandwidth of style variations
> > - tighter contraints in basic styling decisions beyond what is
> > already imposed by the OSM data model
>
> some additional points that come into mind:
>
> [...]

Note these are issues with client side rendering, not with vector
tiles - those are ultimately independent things.  That client side
rendering is often not very efficient if you consider overall costs is
a given.

Especially when serving different styles from the same infrastructure
(as most vector tiles map providers do) there is certainly potential
for energy saving by caching the vector representations on the server
side.  If the way this is done with current vector tiles stacks is the
most efficient approach for that is a different matter of course.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

SimonPoole
In reply to this post by dieterdreist


Am 30.10.2015 um 11:10 schrieb Martin Koppenhoefer:

2015-10-30 10:52 GMT+01:00 Christoph Hormann <[hidden email]>:
So far use of vector tiles seem to primarily
have lead to the following effects:

- improved tile serving efficiency
- a larger bandwidth of style variations
- tighter contraints in basic styling decisions beyond what is already
imposed by the OSM data model


some additional points that come into mind:

- vector maps are consuming more energy to display (because have to be calculated/rendered) -> problem on mobile devices, but also generally a problem because every client has to spend energy on calculating the "same" image (admittedly depends how many different styles there are, and how many people are looking at them until the underlying data changes), so globally (i.e. wikipedia use and not some "niche" usecases) this means really a lot of wasted energy.

- are likely slower to display (for the same reason), although this is depending on different factors (e.g. if you have internet connection and how fast it is (vector maps likely scale better for offline use), how complex the style sheet is, etc.) -> generally vector maps require more ressources, newer more capable client devices.

I don't think anybody is contemplating moving completely away from server-side tile rendering osm.org at this point in time,  using vector tiles pre-rendering will likely simply be a further option.

I kind of half agree with Christoph, but I think the perspective that the vector tiles are a complete replacement for a style specific rendering database is likely simply wrong. So while vector tiles allow to produce a group of related styles where you previously simply had one, if you are doing something completely different you will not be able to use the same tile set. Naturally, in a wide sense of the word, there is likely a substantial amount of economic pressure to reuse what is already there which might lead to really different styles becoming rarer than they are now . Obviously Andy, Christoph, Richard et al know a lot more about this than I do. 

Simon

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Jochen123
In reply to this post by Christoph Hormann
On Fr, Okt 30, 2015 at 10:52:40 +0100, Christoph Hormann wrote:
> If you look at design problems recently discussed in the osm-carto style
> development[2] you will see most of them have nothing to do with vector
> tiles, they would not be made any easier to address with such an
> approach.  On the other hand there are a multitude of things the
> current style handles fairly gracefully, especially the problem of
> reducing geometric complexity, that would be much harder to deal with
> in a vector tiles system[3].

This is not a failure of the vector tile approach in general. It might be a
problem of how the vector tiles are generated today, but not a general
problem of the vector tile approach. A vector tile can contain any data
that you need for rendering any kind of style. The question is how the
data will get into that vector tile. Currently the tool chain that does
this is not mature enough, I agree. It doesn't take into account different
sources of data (needed for special maps) and doesn't do generalization
well enough. But this is something we can and should work on.

And one more thing: I suggest we start thinking about not one set of vector
tiles, but many. Each one can contain different data (from OSM or not) derived
in different ways. Vector tiles from different tile sets can then be merged
(either in an extra step before delivering to client, or during rendering).
This will give us enormous flexiblity and decouple different parts of the
tool chain to allow for easier experimentation.

As an example: Somebody can have a very specialized process for generalizing,
say, railway infrastructure for low zoom levels. This can then be combined with
the standard tiles for rendering specialized maps without requiring the
maker of the specialized map to have a full OSM tool chain working.

Jochen
--
Jochen Topf  [hidden email]  http://www.jochentopf.com/  +49-351-31778688

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Oleksiy Muzalyev
In reply to this post by dieterdreist
One of the advantages of the the vector-based map would be the
multilingualism.

For instance at the moment the OSM map of the Middle East is basically
useless for me as I do not know the Arabic alphabet yet. But as far as I
understand and as I heard at the conference the vector-based map would
allow the choice of a language of the map itself.

There are so many disputes about languages, so an ability to just select
whatever language one wants makes it not just a new feature but speaking
figuratively an advent of a new brave world.

brgds
Oleksiy




_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Maarten Deen
On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
> One of the advantages of the the vector-based map would be the
> multilingualism.
>
> For instance at the moment the OSM map of the Middle East is basically
> useless for me as I do not know the Arabic alphabet yet. But as far as
> I understand and as I heard at the conference the vector-based map
> would allow the choice of a language of the map itself.

I do not see how that can not be solved with png-based tiles. You only
have to render the tiles.
The method for detecting which tileset/language to show is the same.

BTW: it is still not as simple as rendering "in a different language".
Then you start rendering a map in English and see names like "Cologne"
or "Brussels"  show up on the map.

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Oleksiy Muzalyev
I've heard from the OSM operations team's member at the conference that it is the question of the servers' infrastructure cost.

But if we have a vector-based map itself language selection, that we could just add tags name:fr or name: ru and have the OSM map of say New York in French or in Russian for tourists. Or map of Siberia in German also for tourists, Middle East in English, etc. without any additions cost on the server side. It is not that difficult to add such multi-language tags. There could be also the map in Basque, Catalan, Kurdish, Scots and other smaller (by number of speakers) languages without any additional cost and without a civil conflict.

I realize that mobile hardware is not enough advanced for that yet and vector-based technology is only in an development stage.

brgds
Oleksiy

On 30.10.2015 12:06, Maarten Deen wrote:
On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
One of the advantages of the the vector-based map would be the multilingualism.

For instance at the moment the OSM map of the Middle East is basically
useless for me as I do not know the Arabic alphabet yet. But as far as
I understand and as I heard at the conference the vector-based map
would allow the choice of a language of the map itself.

I do not see how that can not be solved with png-based tiles. You only have to render the tiles.
The method for detecting which tileset/language to show is the same.

BTW: it is still not as simple as rendering "in a different language". Then you start rendering a map in English and see names like "Cologne" or "Brussels"  show up on the map.

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Christoph Hormann
In reply to this post by Jochen123
On Friday 30 October 2015, Jochen Topf wrote:

>
> This is not a failure of the vector tile approach in general. It
> might be a problem of how the vector tiles are generated today, but
> not a general problem of the vector tile approach. A vector tile can
> contain any data that you need for rendering any kind of style. The
> question is how the data will get into that vector tile. Currently
> the tool chain that does this is not mature enough, I agree. It
> doesn't take into account different sources of data (needed for
> special maps) and doesn't do generalization well enough. But this is
> something we can and should work on.

We need to distinguish a number of different things here:

1. the idea of storing and transporting geometries in a tiled form in
general
2. the concept of current vector tile stacks
3. the idea of processing geometry on a level that is not generally
possible to do on the fly during rendering

I was exclusively talking about 2. here, in particular because in
contrast to the classical non-vector-tile stacks theys are not suited
to efficiently deal with the geometric complexity issues without 3.  As
i have mentioned many times i think 3. is important and grossly
neglected.  Vector tiles might bring an additional incentive to address
this of course.

The one important thing speaking against tiled geometry storage and
transport in general is handling of different map projections.  Any
such system will require you to select one projection from the very
beginning and you stay bound to it or need fully separate setups for
every projection you want to offer.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Colin Smale
In reply to this post by Oleksiy Muzalyev

 

 

Can't we have a multi-lingual map by overlaying a base tile with a transparent text layer in the chosen language? We wouldn't *need* vector tiles for that, just a bit more storage (bitmap-based text layers should compress nicely) and clients which can handle the selection and display of the extra layer, which is pretty commonplace these days anyway).

--colin

On 2015-10-30 12:29, Oleksiy Muzalyev wrote:

I've heard from the OSM operations team's member at the conference that it is the question of the servers' infrastructure cost.

But if we have a vector-based map itself language selection, that we could just add tags name:fr or name: ru and have the OSM map of say New York in French or in Russian for tourists. Or map of Siberia in German also for tourists, Middle East in English, etc. without any additions cost on the server side. It is not that difficult to add such multi-language tags. There could be also the map in Basque, Catalan, Kurdish, Scots and other smaller (by number of speakers) languages without any additional cost and without a civil conflict.

I realize that mobile hardware is not enough advanced for that yet and vector-based technology is only in an development stage.

brgds
Oleksiy

On 30.10.2015 12:06, Maarten Deen wrote:
On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
One of the advantages of the the vector-based map would be the multilingualism.

For instance at the moment the OSM map of the Middle East is basically
useless for me as I do not know the Arabic alphabet yet. But as far as
I understand and as I heard at the conference the vector-based map
would allow the choice of a language of the map itself.

I do not see how that can not be solved with png-based tiles. You only have to render the tiles.
The method for detecting which tileset/language to show is the same.

BTW: it is still not as simple as rendering "in a different language". Then you start rendering a map in English and see names like "Cologne" or "Brussels"  show up on the map.

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Daniel Koć
In reply to this post by dieterdreist
W dniu 30.10.2015 11:10, Martin Koppenhoefer napisał(a):

> - vector maps are consuming more energy to display (because have to be
> calculated/rendered) -> problem on mobile devices, but also generally

Mobile applications like OsmAnd show that while it really is slower,
it's still usable.

I would not like to "migrate" osm.org to anything - including vector
tiles - right now. I would rather like to have vector playground to test
it and draw some conclusions myself. Yet at this moment I'm not aware of
such experimental service.

--
"The train is always on time / The trick is to be ready to put your bags
down" [A. Cohen]

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Stadin, Benjamin
One such service would be MapBox [1] itself. And in my opinion
mapbox-gl-js would be a natural choice for the client part. MapBox is
doing a really good job, and this is getting really solid.

I wrote a tiny script to create a mapbox-gl-js package for own deployment
[2]. It doesn't do much useful at the moment, other than doing a basic
host config of the build and generating and packaging some of the required
assets. No custom style integration, and no font generation (I wanted to
add this to the script, so they'd be generated from freetype fonts [3]
using fontnik [4]).

~Ben

[1] https://www.mapbox.com/mapbox-gl-js/examples/
[2] https://github.com/benstadin/mapbox-gl-js-packager
[3] https://fedorahosted.org/liberation-fonts/
[4] https://github.com/mapbox/node-fontnik



Am 30.10.15 12:50 schrieb "Daniel Koć" unter <[hidden email]>:

>W dniu 30.10.2015 11:10, Martin Koppenhoefer napisał(a):
>
>> - vector maps are consuming more energy to display (because have to be
>> calculated/rendered) -> problem on mobile devices, but also generally
>
>Mobile applications like OsmAnd show that while it really is slower,
>it's still usable.
>
>I would not like to "migrate" osm.org to anything - including vector
>tiles - right now. I would rather like to have vector playground to test
>it and draw some conclusions myself. Yet at this moment I'm not aware of
>such experimental service.
>
>--
>"The train is always on time / The trick is to be ready to put your bags
>down" [A. Cohen]
>
>_______________________________________________
>dev mailing list
>[hidden email]
>https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Maarten Deen
In reply to this post by Oleksiy Muzalyev
On 2015-10-30 12:29, Oleksiy Muzalyev wrote:

> But if we have a vector-based map itself language selection, that we
> could just add tags _name:fr_ or _name: ru_ and have the OSM map of
> say New York in French or in Russian for tourists. Or map of Siberia
> in German also for tourists, Middle East in English, etc. without any
> additions cost on the server side. It is not that difficult to add
> such multi-language tags. There could be also the map in Basque,
> Catalan, Kurdish, Scots and other smaller (by number of speakers)
> languages without any additional cost and without a civil conflict.

Again, that does not work. I am Dutch. There are a lot of name:nl tags
in the database that really are too obscure to put on a map.
There is a place "Grevelingen" which we associate with a lake in the
southwest of the country. But it is also the old dutch name for the
french town of Gravelines which nobody uses.
But similarly, I don't want to see the name:en tag for of cities
because, as in my example, I will see Cologne or Brussels or Hook of
Holland, which are all not what I want to see.
So you have to have some kind of mechanisme to decide when to show
name:en (which I want to see for e.g. Afghanistan) and when to show
name.


> I realize that mobile hardware is not enough advanced for that yet and
> vector-based technology is only in an development stage.
>
> brgds
> Oleksiy
>
> On 30.10.2015 12:06, Maarten Deen wrote:
>
>> On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
>>
>>> One of the advantages of the the vector-based map would be the
>>> multilingualism.
>>>
>>> For instance at the moment the OSM map of the Middle East is
>>> basically
>>> useless for me as I do not know the Arabic alphabet yet. But as
>>> far as
>>> I understand and as I heard at the conference the vector-based map
>>>
>>> would allow the choice of a language of the map itself.
>>
>> I do not see how that can not be solved with png-based tiles. You
>> only have to render the tiles.
>> The method for detecting which tileset/language to show is the same.
>>
>>
>> BTW: it is still not as simple as rendering "in a different
>> language". Then you start rendering a map in English and see names
>> like "Cologne" or "Brussels"  show up on the map.
>>
>> Regards,
>> Maarten
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

SimonPoole


Am 30.10.2015 um 13:31 schrieb Maarten Deen:
>
> So you have to have some kind of mechanisme to decide when to show
> name:en (which I want to see for e.g. Afghanistan) and when to show name.
>
That is simple, we live with some people not being completely happy :-) *

Seriously the whole point is exposing questionable tag values then they
get fixed, so if Gravelines has name:nl=Grevelingen tagged but in
reality that should be old_name:nl=Grevelingen the best way to get it
fixed is to show the wrong value on the map.

Simon

* even an imperfect language specific map is likely going to make more
people happy than now



_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Oleksiy Muzalyev
In reply to this post by Maarten Deen
In Dutch, in English and other languages based on the Latin alphabet Afghanistan is spelled about the same. But for example in Greek it is: Αφγανιστάν, in Ukrainian: Афганістан, in Kazakh: Ауғанстан, etc. Even though people study foreign languages in school, the absolute majority speak, read and write realistically only in a native language. The name of the country Afghanistan even in Latin letters is not understandable to a lot of people.

Certainly, there is a lot of imperfection in a language tagging yet, as it has not been that important. But if there is a robust comprehensive multilingual map solution similar to say the Gettext for text websites, then mappers will pay more attention to this type of tags.

brgds
Oleksiy

On 30.10.2015 13:31, Maarten Deen wrote:
So you have to have some kind of mechanisme to decide when to show name:en (which I want to see for e.g. Afghanistan) and when to show name.


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Peter Wendorff
In reply to this post by Colin Smale
Hi Colin,

just stitching a labels-layer and a base layer together would work, but
most often would not look well.

Map rendering involves deciding which labels to show considering more
than just the labels, but icons and shields etc. as well to avoid
overlapping between all of them.

You don't want to get a map where the label overlaps icons.

As names in different languages differ in their need for geometrical
space on the map, these placement decisions have to be taken for each
language differently, thus the labels layer would include icons and
stuff like that as well - even oneway and so on.

Of course it would be possible to do, but the result would look worse
than our current maps.

regards
Peter

Am 30.10.2015 um 12:41 schrieb Colin Smale:

> Can't we have a multi-lingual map by overlaying a base tile with a
> transparent text layer in the chosen language? We wouldn't *need* vector
> tiles for that, just a bit more storage (bitmap-based text layers should
> compress nicely) and clients which can handle the selection and display
> of the extra layer, which is pretty commonplace these days anyway).
>
> --colin
>
> On 2015-10-30 12:29, Oleksiy Muzalyev wrote:
>
>> I've heard from the OSM operations team's member at the conference
>> that it is the question of the servers' infrastructure cost.
>>
>> But if we have a vector-based map itself language selection, that we
>> could just add tags /name:fr/ or /name: ru/ and have the OSM map of
>> say New York in French or in Russian for tourists. Or map of Siberia
>> in German also for tourists, Middle East in English, etc. without any
>> additions cost on the server side. It is not that difficult to add
>> such multi-language tags. There could be also the map in Basque,
>> Catalan, Kurdish, Scots and other smaller (by number of speakers)
>> languages without any additional cost and without a civil conflict.
>>
>> I realize that mobile hardware is not enough advanced for that yet and
>> vector-based technology is only in an development stage.
>>
>> brgds
>> Oleksiy
>>
>> On 30.10.2015 12:06, Maarten Deen wrote:
>>> On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
>>>> One of the advantages of the the vector-based map would be the
>>>> multilingualism.
>>>>
>>>> For instance at the moment the OSM map of the Middle East is basically
>>>> useless for me as I do not know the Arabic alphabet yet. But as far as
>>>> I understand and as I heard at the conference the vector-based map
>>>> would allow the choice of a language of the map itself.
>>>
>>> I do not see how that can not be solved with png-based tiles. You
>>> only have to render the tiles.
>>> The method for detecting which tileset/language to show is the same.
>>>
>>> BTW: it is still not as simple as rendering "in a different
>>> language". Then you start rendering a map in English and see names
>>> like "Cologne" or "Brussels"  show up on the map.
>>>
>>> Regards,
>>> Maarten
>>>
>>> _______________________________________________
>>> dev mailing list
>>> [hidden email]
>>> https://lists.openstreetmap.org/listinfo/dev
>>
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email] <mailto:[hidden email]>
>> https://lists.openstreetmap.org/listinfo/dev
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/dev
>


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Stadin, Benjamin
Hi Peter,

Most what you said is already solved by data driven style possibilities
and clustering that you have already with some existing WebGL libraries.
E.g. which city label takes priority when zooming out over another city¹s
label (by default the one with highest population), or whether city labels
take priority over icons.

I do not see why there would be any difference doing this on client or
server side, besides you¹d have to adapt a few things at first of course.
For an example, if you have an algorithm that places your labels depending
on language and geometrical shape of labels, then this algorithm can
likely be applied in realtime on the client side.
If you have different anchor points for labels depending on language, then
this is even less a problem: Decide by a filter rule in realtime which
labels to render, each label may have a different position. This can be as
simple as a single line in your style rule.

Regards
Ben


Am 31.10.15 09:32 schrieb "Peter Wendorff" unter
<[hidden email]>:

>Hi Colin,
>
>just stitching a labels-layer and a base layer together would work, but
>most often would not look well.
>
>Map rendering involves deciding which labels to show considering more
>than just the labels, but icons and shields etc. as well to avoid
>overlapping between all of them.
>
>You don't want to get a map where the label overlaps icons.
>
>As names in different languages differ in their need for geometrical
>space on the map, these placement decisions have to be taken for each
>language differently, thus the labels layer would include icons and
>stuff like that as well - even oneway and so on.
>
>Of course it would be possible to do, but the result would look worse
>than our current maps.
>
>regards
>Peter
>
>Am 30.10.2015 um 12:41 schrieb Colin Smale:
>> Can't we have a multi-lingual map by overlaying a base tile with a
>> transparent text layer in the chosen language? We wouldn't *need* vector
>> tiles for that, just a bit more storage (bitmap-based text layers should
>> compress nicely) and clients which can handle the selection and display
>> of the extra layer, which is pretty commonplace these days anyway).
>>
>> --colin
>>
>> On 2015-10-30 12:29, Oleksiy Muzalyev wrote:
>>
>>> I've heard from the OSM operations team's member at the conference
>>> that it is the question of the servers' infrastructure cost.
>>>
>>> But if we have a vector-based map itself language selection, that we
>>> could just add tags /name:fr/ or /name: ru/ and have the OSM map of
>>> say New York in French or in Russian for tourists. Or map of Siberia
>>> in German also for tourists, Middle East in English, etc. without any
>>> additions cost on the server side. It is not that difficult to add
>>> such multi-language tags. There could be also the map in Basque,
>>> Catalan, Kurdish, Scots and other smaller (by number of speakers)
>>> languages without any additional cost and without a civil conflict.
>>>
>>> I realize that mobile hardware is not enough advanced for that yet and
>>> vector-based technology is only in an development stage.
>>>
>>> brgds
>>> Oleksiy
>>>
>>> On 30.10.2015 12:06, Maarten Deen wrote:
>>>> On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
>>>>> One of the advantages of the the vector-based map would be the
>>>>> multilingualism.
>>>>>
>>>>> For instance at the moment the OSM map of the Middle East is
>>>>>basically
>>>>> useless for me as I do not know the Arabic alphabet yet. But as far
>>>>>as
>>>>> I understand and as I heard at the conference the vector-based map
>>>>> would allow the choice of a language of the map itself.
>>>>
>>>> I do not see how that can not be solved with png-based tiles. You
>>>> only have to render the tiles.
>>>> The method for detecting which tileset/language to show is the same.
>>>>
>>>> BTW: it is still not as simple as rendering "in a different
>>>> language". Then you start rendering a map in English and see names
>>>> like "Cologne" or "Brussels"  show up on the map.
>>>>
>>>> Regards,
>>>> Maarten
>>>>
>>>> _______________________________________________
>>>> dev mailing list
>>>> [hidden email]
>>>> https://lists.openstreetmap.org/listinfo/dev
>>>
>>>
>>> _______________________________________________
>>> dev mailing list
>>> [hidden email] <mailto:[hidden email]>
>>> https://lists.openstreetmap.org/listinfo/dev
>>
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.openstreetmap.org/listinfo/dev
>>
>
>
>_______________________________________________
>dev mailing list
>[hidden email]
>https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating osm.org to vectors/Kartotherian

Yuri Astrakhan
Thanks for all the responses! Are there any people on the stylesheet team interested in working on this?  It might be possible to adapt osm-carto work directly, since it is based on the same underlying technology.

Jochen, Kartotherian can dynamically alter/recombine tiles, e.g. adding layers from multiple sources (e.g. language), or adding raster layers (both raster and vector data can be stored in the same vtile). So we could create a set of well-defined layers, stored either together or separate, and combined on the fly.

Oleksiy, the true multilingual support requires Mapnik hstore support, see #1113 [1]. Wikimedia DE might be able to help us with it, but it's still in the air.  Another issue is Mapbox's WebGL cannot shape connected fonts yet (e.g. Arabic), so they will still rely on the server-side renderig.

Martin:  If WebGL is a problem for mobile, they can fallback to PNGs generated by Kartotherian, but we should consider the energy+time needed to download high DPI PNG tiles vs smaller vector tiles + local rendering, plus per-client customization aspects (multilingual, user-centric icons, etc)

Christoph, I agree that our current map is nowhere near as feature rich as default OSM - we spent most of our time building the tile serving platform, and spent very little time on the map itself.  We have even considered using Natural Earth for the low zooms, possibly with an extra labels layer.

Benjamin, we had to alter Mapbox's Bright to use the free Google Noto fonts. Can you help with automating WebGL setup & font generation for Kartotherian? See [2]

Colin, we considered raster-layer combining for languages, but it might be bad for caching, and would still require a lot of storage due to per-item overhead. Storing all available languages in one vtile, and deciding at render time seems more efficient.

Christoph, re multiple projections - if Mapnik implements vector tile merging, we could in theory dynamicaly recombine tiles and change projections, would't we? At a performance cost of course.  And we could store some low-complexity data in the original format, converting it to the needed tile on the fly, in any projection.

Thanks!!



On Sat, Oct 31, 2015 at 6:45 PM, Stadin, Benjamin <[hidden email]> wrote:
Hi Peter,

Most what you said is already solved by data driven style possibilities
and clustering that you have already with some existing WebGL libraries.
E.g. which city label takes priority when zooming out over another city¹s
label (by default the one with highest population), or whether city labels
take priority over icons.

I do not see why there would be any difference doing this on client or
server side, besides you¹d have to adapt a few things at first of course.
For an example, if you have an algorithm that places your labels depending
on language and geometrical shape of labels, then this algorithm can
likely be applied in realtime on the client side.
If you have different anchor points for labels depending on language, then
this is even less a problem: Decide by a filter rule in realtime which
labels to render, each label may have a different position. This can be as
simple as a single line in your style rule.

Regards
Ben


Am 31.10.15 09:32 schrieb "Peter Wendorff" unter
<[hidden email]>:

>Hi Colin,
>
>just stitching a labels-layer and a base layer together would work, but
>most often would not look well.
>
>Map rendering involves deciding which labels to show considering more
>than just the labels, but icons and shields etc. as well to avoid
>overlapping between all of them.
>
>You don't want to get a map where the label overlaps icons.
>
>As names in different languages differ in their need for geometrical
>space on the map, these placement decisions have to be taken for each
>language differently, thus the labels layer would include icons and
>stuff like that as well - even oneway and so on.
>
>Of course it would be possible to do, but the result would look worse
>than our current maps.
>
>regards
>Peter
>
>Am 30.10.2015 um 12:41 schrieb Colin Smale:
>> Can't we have a multi-lingual map by overlaying a base tile with a
>> transparent text layer in the chosen language? We wouldn't *need* vector
>> tiles for that, just a bit more storage (bitmap-based text layers should
>> compress nicely) and clients which can handle the selection and display
>> of the extra layer, which is pretty commonplace these days anyway).
>>
>> --colin
>>
>> On 2015-10-30 12:29, Oleksiy Muzalyev wrote:
>>
>>> I've heard from the OSM operations team's member at the conference
>>> that it is the question of the servers' infrastructure cost.
>>>
>>> But if we have a vector-based map itself language selection, that we
>>> could just add tags /name:fr/ or /name: ru/ and have the OSM map of
>>> say New York in French or in Russian for tourists. Or map of Siberia
>>> in German also for tourists, Middle East in English, etc. without any
>>> additions cost on the server side. It is not that difficult to add
>>> such multi-language tags. There could be also the map in Basque,
>>> Catalan, Kurdish, Scots and other smaller (by number of speakers)
>>> languages without any additional cost and without a civil conflict.
>>>
>>> I realize that mobile hardware is not enough advanced for that yet and
>>> vector-based technology is only in an development stage.
>>>
>>> brgds
>>> Oleksiy
>>>
>>> On 30.10.2015 12:06, Maarten Deen wrote:
>>>> On 2015-10-30 11:53, Oleksiy Muzalyev wrote:
>>>>> One of the advantages of the the vector-based map would be the
>>>>> multilingualism.
>>>>>
>>>>> For instance at the moment the OSM map of the Middle East is
>>>>>basically
>>>>> useless for me as I do not know the Arabic alphabet yet. But as far
>>>>>as
>>>>> I understand and as I heard at the conference the vector-based map
>>>>> would allow the choice of a language of the map itself.
>>>>
>>>> I do not see how that can not be solved with png-based tiles. You
>>>> only have to render the tiles.
>>>> The method for detecting which tileset/language to show is the same.
>>>>
>>>> BTW: it is still not as simple as rendering "in a different
>>>> language". Then you start rendering a map in English and see names
>>>> like "Cologne" or "Brussels"  show up on the map.
>>>>
>>>> Regards,
>>>> Maarten
>>>>
>>>> _______________________________________________
>>>> dev mailing list
>>>> [hidden email]
>>>> https://lists.openstreetmap.org/listinfo/dev
>>>
>>>
>>> _______________________________________________
>>> dev mailing list
>>> [hidden email] <mailto:[hidden email]>
>>> https://lists.openstreetmap.org/listinfo/dev
>>
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.openstreetmap.org/listinfo/dev
>>
>
>
>_______________________________________________
>dev mailing list
>[hidden email]
>https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
12