OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Developer Discussion mailing list
I've been working on a new project, OpenStreetMap Cartographic. This is
a client-side rendering based on OpenStreetMap Carto. This is an
ambitious project, as OpenStreetMap Carto is an extremely complex style
which shows a large number of features. The technical choices I'm making
are designed so the style is capable of handling the load of osm.org
with minutely updates.

I've put up a world-wide demo at
https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html,
using data from 2020-03-16, and you can view the code at
https://github.com/pnorman/openstreetmap-cartographic.

Incomplete parts
================

Only zoom 0 to 8 has been implemented so far. I started at zoom 0 and am
working my way down.

Admin boundaries are not implemented. OpenStreetMap Carto uses
Mapnik-specific tricks to deduplicate the rendering of these. I know how
I can do this, but it requires the changes I intend to make with the flex
backend.

Landuse, vegetation, and other natural features are not rendered until
zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and these
features first appear at zoom 5. There are numerous problems with
unprocessed OpenStreetMap data at these scales. OpenStreetMap Carto gets
a result that looks acceptable but is poor at conveying information by
tweaking Mapnik image rasterizing options. I'm looking for better options
here involving preprocessed data, but haven't found any.

I'm still investigating how to best distribute sprites.

Technology
==========

The technology choices are designed to be suitable for a replacement for
tile.osm.org. This means minutely updates, high traffic, high reliability,
and multiple servers. Tilekiln, the vector tile generator, supports all
of these. It's designed to better share the rendering results among
multiple servers, a significant flaw with renderd + mod_tile and the
standard filesystem storage. It uses PostGIS' ST_AsMVT, which is very
fast with PostGIS 3.0. On my home system it generates z0-z8 in under
40 minutes.

Often forgotten is the development requirements. The style needs to
support multiple developers working on similar areas, git merge conflicts
while maintaining an easy development workflow. I'm still figuring this
out. Mapbox GL styles are written in JSON and most of the tools
overwrite any formatting. This means there's no way to add comments to
lines of codes. Comments are a requirement for a style like this, so
I'm investigating minimal pre-processing options. The downside to this
will make it harder to use with existing GUI editors like Fresco or
Maputnik.

Cartography
===========

The goal of this project isn't to do big cartography changes yet, but
client-side rendering opens up new tools. The biggest immediate change
is zoom is continuous, no longer an integer or fixed value. This means
parameters like sizes can smoothly change as you zoom in and out,
specified by their start and end size instead of having to specify each
zoom.

Want to help?
=============

Have a look at https://github.com/pnorman/openstreetmap-cartographic and
have a go at setting it up and generating your own map. If you have
issues, open an issue or pull request. Or, because OpenStreetMap
Cartographic uses Tilekiln have a look at
https://github.com/pnorman/tilekiln/issues.


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Joseph Eisenberg
Thank you for making this, it looks like a lot of work!

These client-side vector tiles at z8? (https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html)

My laptop (2015 Macbook Pro, 16 GB RAM, 2.2 GHz Intel Core i7) appears to use some effort to show areas with a large amount of data,
For example if I view England + Wales on z8 and then move over to the Netherlands and then to Germany, at z7 and z8, 
CPU usage goes up to >100% for a time, and there is a noticeable delay. Perhaps part of this is a delay in serving the tiles?

It would be nice to see a demonstration of this applied server-side to produce raster tiles. Are there any samples of that yet?

– Joseph Eisenberg

On Sun, May 24, 2020 at 5:37 PM Paul Norman via dev <[hidden email]> wrote:
I've been working on a new project, OpenStreetMap Cartographic. This is
a client-side rendering based on OpenStreetMap Carto. This is an
ambitious project, as OpenStreetMap Carto is an extremely complex style
which shows a large number of features. The technical choices I'm making
are designed so the style is capable of handling the load of osm.org
with minutely updates.

I've put up a world-wide demo at
https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html,
using data from 2020-03-16, and you can view the code at
https://github.com/pnorman/openstreetmap-cartographic.

Incomplete parts
================

Only zoom 0 to 8 has been implemented so far. I started at zoom 0 and am
working my way down.

Admin boundaries are not implemented. OpenStreetMap Carto uses
Mapnik-specific tricks to deduplicate the rendering of these. I know how
I can do this, but it requires the changes I intend to make with the flex
backend.

Landuse, vegetation, and other natural features are not rendered until
zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and these
features first appear at zoom 5. There are numerous problems with
unprocessed OpenStreetMap data at these scales. OpenStreetMap Carto gets
a result that looks acceptable but is poor at conveying information by
tweaking Mapnik image rasterizing options. I'm looking for better options
here involving preprocessed data, but haven't found any.

I'm still investigating how to best distribute sprites.

Technology
==========

The technology choices are designed to be suitable for a replacement for
tile.osm.org. This means minutely updates, high traffic, high reliability,
and multiple servers. Tilekiln, the vector tile generator, supports all
of these. It's designed to better share the rendering results among
multiple servers, a significant flaw with renderd + mod_tile and the
standard filesystem storage. It uses PostGIS' ST_AsMVT, which is very
fast with PostGIS 3.0. On my home system it generates z0-z8 in under
40 minutes.

Often forgotten is the development requirements. The style needs to
support multiple developers working on similar areas, git merge conflicts
while maintaining an easy development workflow. I'm still figuring this
out. Mapbox GL styles are written in JSON and most of the tools
overwrite any formatting. This means there's no way to add comments to
lines of codes. Comments are a requirement for a style like this, so
I'm investigating minimal pre-processing options. The downside to this
will make it harder to use with existing GUI editors like Fresco or
Maputnik.

Cartography
===========

The goal of this project isn't to do big cartography changes yet, but
client-side rendering opens up new tools. The biggest immediate change
is zoom is continuous, no longer an integer or fixed value. This means
parameters like sizes can smoothly change as you zoom in and out,
specified by their start and end size instead of having to specify each
zoom.

Want to help?
=============

Have a look at https://github.com/pnorman/openstreetmap-cartographic and
have a go at setting it up and generating your own map. If you have
issues, open an issue or pull request. Or, because OpenStreetMap
Cartographic uses Tilekiln have a look at
https://github.com/pnorman/tilekiln/issues.


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Developer Discussion mailing list
On 2020-05-24 10:26 p.m., Joseph Eisenberg wrote:

> Thank you for making this, it looks like a lot of work!
>
> These client-side vector tiles at z8?
> (https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html)
>
> My laptop (2015 Macbook Pro, 16 GB RAM, 2.2 GHz Intel Core i7) appears
> to use some effort to show areas with a large amount of data,
> For example if I view England + Wales on z8 and then move over to the
> Netherlands and then to Germany, at z7 and z8,
> CPU usage goes up to >100% for a time, and there is a
> noticeable delay. Perhaps part of this is a delay in serving the tiles?

Some of the tiles are decently large so could be slow to download, but
based on your CPU it's not that. More likely it's the high number of
vegetation and landuse features at those zooms. There are some things I
might be able to do reduce the feature count.

> It would be nice to see a demonstration of this applied server-side to
> produce raster tiles. Are there any samples of that yet?

There are a few examples of producing raster tiles from vector tiles and
a Mapbox GL style. Mapbox does this with Mapbox GL Native on the server,
but I don't believe they've released the exact software they use.
https://github.com/maptiler/tileserver-gl/ is open-source software that
generates raster tiles on demand.


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Maarten Deen
On 2020-05-25 09:59, Paul Norman via dev wrote:

> On 2020-05-24 10:26 p.m., Joseph Eisenberg wrote:
>> Thank you for making this, it looks like a lot of work!
>>
>> These client-side vector tiles at z8?
>> (https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html)
>>
>> My laptop (2015 Macbook Pro, 16 GB RAM, 2.2 GHz Intel Core i7) appears
>> to use some effort to show areas with a large amount of data,
>> For example if I view England + Wales on z8 and then move over to the
>> Netherlands and then to Germany, at z7 and z8,
>> CPU usage goes up to >100% for a time, and there is a
>> noticeable delay. Perhaps part of this is a delay in serving the
>> tiles?
>
> Some of the tiles are decently large so could be slow to download, but
> based on your CPU it's not that. More likely it's the high number of
> vegetation and landuse features at those zooms. There are some things
> I might be able to do reduce the feature count.

I'm not noticing much slowness on my computer, but having a lot of cores
does help here I think. I won't notice it when my browser is using one
core (out of six) at 100%.
I did get some delay when zooming in, at first I thought "oh, only
motorways, that's not much" but after some 10 seconds I got landuse and
other roads.
Still, it looks to be a very simplified subset of the complete data. So
I'd be interested to see how this works on a full dataset.

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Christoph Hormann-2
In reply to this post by Developer Discussion mailing list
On Monday 25 May 2020, Paul Norman via dev wrote:
> https://github.com/pnorman/openstreetmap-cartographic.

In general this looks like a good approach to evaluate where the
practical issues are without being bogged down by bloated legacy tools
(well - except for the client side of course).

> Landuse, vegetation, and other natural features are not rendered
> until zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and
> these features first appear at zoom 5. There are numerous problems
> with unprocessed OpenStreetMap data at these scales. OpenStreetMap
> Carto gets a result that looks acceptable but is poor at conveying
> information by tweaking Mapnik image rasterizing options. I'm looking
> for better options here involving preprocessed data, but haven't
> found any.

Yes, i think this style as is is a good demonstration that the current
approach used universally in the industry (and that is used in
ST_AsMVT() as well) of performing per feature lossy vector geometry
compression (usually in combination with way_area filtering) is a dead
end.  You can see that nicely if you overzoom the demo map:

http://imagico.de/files/screenshot_cartographic.png
http://imagico.de/files/screenshot_carto.png

This creates a lot more artefacts and noise in the rendering than the
AGG rendering issues in Mapnik typically do even at the resolution it
is intended for - including counterproductively adding a lot of high
frequency noise in many cases.

> Often forgotten is the development requirements. The style needs to
> support multiple developers working on similar areas, git merge
> conflicts while maintaining an easy development workflow. I'm still
> figuring this out. Mapbox GL styles are written in JSON and most of
> the tools overwrite any formatting. This means there's no way to add
> comments to lines of codes. Comments are a requirement for a style
> like this, so I'm investigating minimal pre-processing options. The
> downside to this will make it harder to use with existing GUI editors
> like Fresco or Maputnik.

I think this is a point that cannot be overstated.  Mapbox GL JSON (or
FWIW JSON in general) in contrast to CartoCSS is a software developer
centered format and not a cartographer centered format.  Many
mainstream low sophistication users of client side vector tile
frameworks focus exclusively on interactive editors and not working on
the code level - which as you explained is not an option for a
cooperative community project or for styles with a higher level of
sophistication.

Just re-casting the JSON data structures in a different format easier to
edit by hand, suitable for comments and leading to well readable diffs
is IMO not enough for a truly cartographer centered approach.  But this
is definitely a hard problem because there are so many in parts
opposite requirements.

Having a reference renderer that does not build on an extremely complex
and platform dependent framework (a.k.a. web browser) would be
important for style development - i think Joseph also hinted in that
direction.  It would also be important for things like automated tests
etc.

One other thing to keep in mind - OSM-Carto uses polygon fill patterns a
lot for differentiating different types of polygons.  While it is in
principle possible to show fill patterns in continuous zooming in an
ergonomic fashion i have so far never seen maps in the wild doing that.  
This would be something to consider if you want to create a style
similar in the richness of landcover rendering to OSM-Carto.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

ndrw6
In reply to this post by Developer Discussion mailing list
On 25/05/2020 01:34, Paul Norman via dev wrote:
>
> Landuse, vegetation, and other natural features are not rendered until
> zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and these
> features first appear at zoom 5. There are numerous problems with
> unprocessed OpenStreetMap data at these scales. OpenStreetMap Carto gets
> a result that looks acceptable but is poor at conveying information by
> tweaking Mapnik image rasterizing options. I'm looking for better options
> here involving preprocessed data, but haven't found any.


I think we will need fairly heavy pre-processing (filtering and
simplification/generalization) of data before they hit the tile and the
client. Carto is doing some of it already, even though data stay on the
server side at all times. With the client-side rendering this is much
more important for three reason - real time rendering, network traffic,
limited resources on client side.

Some ideas:

- Tiles should contains only objects to be rendered, so all the
filtering currently done in Carto should be applied at the tile
construction time.

- Currently Carto produces a lot of data that are later filtered out in
Mapnik (via spatial occlusion) - this also should be brought back to a
tiling script.

- Tiles should be split into layers (similar to layers in Carto). Then
the user would have an option to select layers they are interested in
(think language versions, POIs). With more, smaller tiles there is also
a potential for distributing the server side load more uniformly.

- For z5-z12 we will need some kind of polygon
generalization/simplification. For example, at z7 I can see all landuses
in England, with some towns occupying only a couple of px but but still
being comprised of hundreds of landuse polygons. I'm not talking about
anything fancy, but merging anything sub-1px into "pixel rectangles"
could go a long way. (Edit: I see you are already doing it - I can see
some simplified landuse polygons when zooming in further).


> The technology choices are designed to be suitable for a replacement for
> tile.osm.org. This means minutely updates, high traffic, high
> reliability,
> and multiple servers.


Do we need minutely updates for vector tiles? Sure, that's a nice
feature to have but with client-side rendering any performance issues
will be directly exposed to the users. And performance is a big issue in
most client-side rendering solutions. If we can keep the current raster
backend with minutely updates for mapping feedback, I have no problem
with hourly or daily updates for the vector version. We would need a
raster option for browsers/devices not supporting WebGL anyway.

ndrw



_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Jason Remillard
In reply to this post by Developer Discussion mailing list
Hi Paul,
 
The Mapbox GL styles are very low level, and are like the mapnik XML input files. The Mapbox GL editing tools available don't seem to provide abstractions over the underlying specification.  

Also, rewriting from scratch the main OpenStreetMap Carto style sheet is not appealing. There has been a lot of work put into it, and whatever happens next, everybody is going to be living with it for a long time.

I have done some projects with Mapbox GL with client side rendering, and the performance on my tiny style sheet was unacceptable on mobile. Having the main style sheet use client side rendering will need to wait for newer and faster phones.
 
Going back to the problems. Long term it is clear that it would be a good thing if OSM had a client side rendering option. Also, right now, the existing tile servers are overloaded. A victim of the projects ongoing success and CartoCSS baked in performance issues. 

My ambitious thought is that the equivalent of the CartoCSS needs to be invented to provide a human friendly interface to both the Mapbox GL and mapnik rendering engines.

The language should be at a higher level of abstraction. For example, it should just handle the "layer" key. To broaden the pool of contributors, it should directly use the OSM key and values, and not a foreign intermediate SQL database or tile schema. Lastly, accept that the language is just for rendering normal maps, and assume things like labels are always on top. 

I am not a fan of CartoCSS, but in theory, we could write a new CartoCSS processor to target Mapbox GL to save the main style sheet.

Jason

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Developer Discussion mailing list
On 2020-05-25 9:25 a.m., Jason Remillard wrote:
> Hi Paul,
>
> The Mapbox GL styles are very low level, and are like the mapnik XML
> input files. The Mapbox GL editing tools available don't seem to
> provide abstractions over the underlying specification.

Yes, the specification and tools are quite clearly not designed for
human readability. There has been some work at languages that compile to
Mapbox GL styles, but nothing that supports expressions yet.

> Also, rewriting from scratch the main OpenStreetMap Carto style sheet
> is not appealing. There has been a lot of work put into it, and
> whatever happens next, everybody is going to be living with it for a
> long time.

I'm a maintainer of OpenStreetMap Carto so I'm familiar with its
development. I don't think this is a problem. It's not from scratch,
it's implementing existing cartography in a new language.

> I have done some projects with Mapbox GL with client side rendering,
> and the performance on my tiny style sheet was unacceptable on mobile.
> Having the main style sheet use client side rendering will need to
> wait for newer and faster phones.

Performance with normal basemaps and small stylesheets should be
acceptable. The question is, how do you get normal sized tiles at low
zooms from OSM data, how big will the tiles be at high zooms, and will
the stylesheet be too complex.


> Going back to the problems. Long term it is clear that it would be a
> good thing if OSM had a client side rendering option. Also, right now,
> the existing tile servers are overloaded. A victim of the projects
> ongoing success and CartoCSS baked in performance issues.

No, the issues have been with the tile CDN, not the tile servers.
CartoCSS has never been a performance issue.


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Yves-2


Le 26 mai 2020 03:58:50 GMT+02:00, Paul Norman via dev <[hidden email]> a écrit :

>Performance with normal basemaps and small stylesheets should be
>acceptable.

Reading this, I don't think I understand. However it makes me think of a raster basemap plus a vector overlay for labels to solve internationalization issue. How lightweight such an overlay could be?
Yves

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Joseph Eisenberg
It seems like the biggest problem with the vector rendering, for performance and cartography, is rendering areas such as water and landcover, while the big benefit is with text labels, SVG icons, and linear features.

Is it possible to continue rendering the landcover and water areas from a pre-rendered raster file with this toolchain?

– Joseph Eisenberg

On Mon, May 25, 2020 at 9:27 PM Yves <[hidden email]> wrote:


Le 26 mai 2020 03:58:50 GMT+02:00, Paul Norman via dev <[hidden email]> a écrit :

>Performance with normal basemaps and small stylesheets should be
>acceptable.

Reading this, I don't think I understand. However it makes me think of a raster basemap plus a vector overlay for labels to solve internationalization issue. How lightweight such an overlay could be?
Yves

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Maarten Deen
In reply to this post by Yves-2
On 2020-05-26 06:24, Yves wrote:
> Le 26 mai 2020 03:58:50 GMT+02:00, Paul Norman via dev
> <[hidden email]> a écrit :
>
>> Performance with normal basemaps and small stylesheets should be
>> acceptable.
>
> Reading this, I don't think I understand. However it makes me think of
> a raster basemap plus a vector overlay for labels to solve
> internationalization issue. How lightweight such an overlay could be?

You might run into problems when you have to place labels very exactly.
For low zoom that is not an issue, if a place name is a bit left or
right than that's no problem, but when you get to high zoom levels and
want to place lets say street names, than you have to know where to
place it and that means you have to know how the raster tile behind it
is rendered.

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Developer Discussion mailing list
In reply to this post by Maarten Deen
On 2020-05-25 1:15 a.m., Maarten Deen wrote:
> Still, it looks to be a very simplified subset of the complete data.
> So I'd be interested to see how this works on a full dataset.

No, it has the full data except for admin boundaries and bay/straight
names on zoom 7 and 8. When you compare it to OpenStreetMap Carto
they're pretty similar. There are some labeling differences with sizes
of labels and when labels come in, but those don't have anything to do
with performance or size.


_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Maarten Deen
On 2020-05-26 10:15, Paul Norman via dev wrote:
> On 2020-05-25 1:15 a.m., Maarten Deen wrote:
>> Still, it looks to be a very simplified subset of the complete data.
>> So I'd be interested to see how this works on a full dataset.
>
> No, it has the full data except for admin boundaries and bay/straight
> names on zoom 7 and 8. When you compare it to OpenStreetMap Carto
> they're pretty similar. There are some labeling differences with sizes
> of labels and when labels come in, but those don't have anything to do
> with performance or size.

No, that's not what I'm seeing. I am seeing very basic and rectangular
landuses, obviously simplified, along with waterways, railways,
motorways, trunk, primary secondary roads and they too have been
simplified, to the extent that parallel ways on motorways are now
crossing eachother.

This is in my area:
https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html#12.14/51.36196/6.08322
https://imgur.com/a/WcYlVUc

Regards,
Maarten

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

SimonPoole
I think you are missing

"Only zoom 0 to 8 has been implemented so far. I started at zoom 0 and
am working my way down."

Simon

Am 26.05.2020 um 10:42 schrieb Maarten Deen:

> On 2020-05-26 10:15, Paul Norman via dev wrote:
>> On 2020-05-25 1:15 a.m., Maarten Deen wrote:
>>> Still, it looks to be a very simplified subset of the complete data.
>>> So I'd be interested to see how this works on a full dataset.
>>
>> No, it has the full data except for admin boundaries and bay/straight
>> names on zoom 7 and 8. When you compare it to OpenStreetMap Carto
>> they're pretty similar. There are some labeling differences with sizes
>> of labels and when labels come in, but those don't have anything to do
>> with performance or size.
>
> No, that's not what I'm seeing. I am seeing very basic and rectangular
> landuses, obviously simplified, along with waterways, railways,
> motorways, trunk, primary secondary roads and they too have been
> simplified, to the extent that parallel ways on motorways are now
> crossing eachother.
>
> This is in my area:
> https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html#12.14/51.36196/6.08322
> https://imgur.com/a/WcYlVUc
>
> Regards,
> Maarten
>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Developer Discussion mailing list
More specifically what you are seeing is that it will let you zoom in
beyond zoom 8 but you are seeing data that was simplified on the basis
that it would be shown at zoom 8 where those artefacts would not be
visible.

Tom

On 26/05/2020 09:59, Simon Poole wrote:

> I think you are missing
>
> "Only zoom 0 to 8 has been implemented so far. I started at zoom 0 and
> am working my way down."
>
> Simon
>
> Am 26.05.2020 um 10:42 schrieb Maarten Deen:
>> On 2020-05-26 10:15, Paul Norman via dev wrote:
>>> On 2020-05-25 1:15 a.m., Maarten Deen wrote:
>>>> Still, it looks to be a very simplified subset of the complete data.
>>>> So I'd be interested to see how this works on a full dataset.
>>>
>>> No, it has the full data except for admin boundaries and bay/straight
>>> names on zoom 7 and 8. When you compare it to OpenStreetMap Carto
>>> they're pretty similar. There are some labeling differences with sizes
>>> of labels and when labels come in, but those don't have anything to do
>>> with performance or size.
>>
>> No, that's not what I'm seeing. I am seeing very basic and rectangular
>> landuses, obviously simplified, along with waterways, railways,
>> motorways, trunk, primary secondary roads and they too have been
>> simplified, to the extent that parallel ways on motorways are now
>> crossing eachother.
>>
>> This is in my area:
>> https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html#12.14/51.36196/6.08322
>> https://imgur.com/a/WcYlVUc
>>
>> Regards,
>> Maarten
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.openstreetmap.org/listinfo/dev
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/dev
>


--
Tom Hughes ([hidden email])
http://compton.nu/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Maarten Deen
In reply to this post by SimonPoole
On 2020-05-26 10:59, Simon Poole wrote:
> I think you are missing
>
> "Only zoom 0 to 8 has been implemented so far. I started at zoom 0 and
> am working my way down."

Oh zoom 8 is way further out than I was, my bad. The simplifications are
not visible at that zoomlevel.

Maarten


>
> Simon
>
> Am 26.05.2020 um 10:42 schrieb Maarten Deen:
>> On 2020-05-26 10:15, Paul Norman via dev wrote:
>>> On 2020-05-25 1:15 a.m., Maarten Deen wrote:
>>>> Still, it looks to be a very simplified subset of the complete data.
>>>> So I'd be interested to see how this works on a full dataset.
>>>
>>> No, it has the full data except for admin boundaries and bay/straight
>>> names on zoom 7 and 8. When you compare it to OpenStreetMap Carto
>>> they're pretty similar. There are some labeling differences with
>>> sizes
>>> of labels and when labels come in, but those don't have anything to
>>> do
>>> with performance or size.
>>
>> No, that's not what I'm seeing. I am seeing very basic and rectangular
>> landuses, obviously simplified, along with waterways, railways,
>> motorways, trunk, primary secondary roads and they too have been
>> simplified, to the extent that parallel ways on motorways are now
>> crossing eachother.
>>
>> This is in my area:
>> https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html#12.14/51.36196/6.08322
>> https://imgur.com/a/WcYlVUc
>>
>> Regards,
>> Maarten
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.openstreetmap.org/listinfo/dev
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.openstreetmap.org/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Christoph Hormann-2
In reply to this post by Developer Discussion mailing list
On Tuesday 26 May 2020, Tom Hughes via dev wrote:
> More specifically what you are seeing is that it will let you zoom in
> beyond zoom 8 but you are seeing data that was simplified on the
> basis that it would be shown at zoom 8 where those artefacts would
> not be visible.

As said that is not quite true - but of course attributing artefacts,
noise and rendering distortions that are visible to specific data
processing steps is not easy.

As Joseph has hinted when it comes to accurate rendering for precise
mapper feedback without unintentional incentives at the lower zoom
levels there are other options based on pre-rasterizing the data which
would still allow using the benefits of client side styling.  I have
demonstrated that in the past.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

ndrw6
On 26/05/2020 13:26, Christoph Hormann wrote:
> As Joseph has hinted when it comes to accurate rendering for precise
> mapper feedback without unintentional incentives at the lower zoom
> levels there are other options based on pre-rasterizing the data which
> would still allow using the benefits of client side styling.  I have
> demonstrated that in the past.

Rasterization is a data generalization technique too. Yes, in some cases
it leaves no artifacts at all but that's not true in a general case and
almost never true on mobile platforms. It is a viable solution but to
make it work well the style should not use borders or bake in
antialiasing (i.e. leave that to the client code).

I am concerned about something else - even with polygon generalization
in place, the performance is not great. Zooming/panning is rather slow
on my not so weak PC - the map would be unusable on a mobile device or
at best it would be draining the battery quickly.

I've been looking into the code and I couldn't find anything related to
polygon generalization. Where is it done? If it happens automatically in
the Mapbox GL renderer that's too late - all the cost of preparing
tiles, transmitting them, processing them in the client has already been
incurred. Displaying data via WebGL is probably the cheapest part of the
processing chain.

ndrw



_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Christoph Hormann-2
On Tuesday 26 May 2020, [hidden email] wrote:
>
> Rasterization is a data generalization technique too.

No it isn't.  Generalization is the most frequently abused term in
cartography.  Calling ST_Simplify and way_area filtering on OSM
landcover polygon data generalization is a bit like calling JPEG
compression noise reduction.

But we are drifting off topic here.

> Yes, in some
> cases it leaves no artifacts at all

Who would say that?  In fact sampling artefacts (a.k.a. aliasing) are
the mother of all artefacts in computer graphics.

The point is that sampling artefacts are very well understood and it is
known how to minimize them.

> I've been looking into the code and I couldn't find anything related
> to polygon generalization. Where is it done?

Lossy geometry compression is happening in ST_AsMVTGeom - which is not
just the inevitable coordinate rounding to the vector tile coordinate
grid but also the subjective choice of per feature Douglas-Peucker line
compression.  See

https://github.com/postgis/postgis/pull/463

for the well argued suggestion not to do that.  The way_area filtering
is explicit in the SQL code.

--
Christoph Hormann
http://www.imagico.de/

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Tom Browder
In reply to this post by Developer Discussion mailing list
On Sun, May 24, 2020 at 19:37 Paul Norman via dev <[hidden email]> wrote:
...

out. Mapbox GL styles are written in JSON and most of the tools
overwrite any formatting. This means there's no way to add comments to

Have you investigated HJSON? See:


The Raku language (https://raku.org) has a module that handles it.

Blessings,

-Tom

_______________________________________________
dev mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/dev
12