mod_tile stable version ?

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

mod_tile stable version ?

Bernard Fouché
Hi All,

I'm new to openstreetmap and I've nearly succeeded in having a working
tile server on a Fedora 18 box.

However I experience problems with mod_tile/renderd:

- all wiki pages I find simply state for instance:

Get and install mod_tile itself:
cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile
cd mod_tile
./autogen.sh
./configure

- but I have crashes with renderd with what's currently available on the
SVN repo:

*** glibc detected *** /data/mod_tile/.libs/lt-renderd: free(): invalid
next size (fast): 0x00007f602c36af00 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3e5887ca8e]
/data/mod_tile/.libs/lt-renderd[0x407920]
/data/mod_tile/.libs/lt-renderd[0x406bf1]
/lib64/libpthread.so.0[0x3e59007d15]
/lib64/libc.so.6(clone+0x6d)[0x3e588f246d]

- Revision 29404, the current revision also has a bug in store_file.c:

...
     tmp = malloc(sizeof(char) * strlen(meta_path) + 12);
     sprintf(tmp, "%s.%lu", meta_path, pthread_self());

     if (mkdirp(tmp))
         free(tmp);
         return -1;

     fd = open(tmp, O_WRONLY | O_TRUNC | O_CREAT, 0666);
...

'{}' are missing around the free/return following a failed call to
mkdirp().
https://github.com/openstreetmap/mod_tile/blob/master/store_file.c shows
the same problem.

https://github.com/openstreetmap/mod_tile/commit/398310fdf662d56a283801a8274a072c07983a8b#store_file.c 
shows that the file was initially added with the bug.

Such a bug makes me think that the current revision isn't a stable one
since it breaks to whole functioning of renderd and not a single tile
can be generated.

So I'm looking for a SVN revision known to be stable (or a repo that
holds stable revisions) and wasn't able to locate one yet.

Any hint welcome!

Thanks,

   Bernard

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Sven Geggus
Bernard Fouché <[hidden email]> wrote:

> - but I have crashes with renderd with what's currently available on the
> SVN repo

Frustration about renderd was the main reason why tirex was born a few years
ago.

So you will probably give it a try instead of renderd.

https://trac.openstreetmap.org/browser/applications/utils/tirex

Regards

Sven

--
"Das Einzige wovor wir Angst haben müssen ist die Angst selbst"
                                                (Franklin D. Roosevelt)

/me is giggls@ircnet, http://sven.gegg.us/ on the Web

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
Le 26/03/2013 13:19, Sven Geggus a écrit :

> Bernard Fouché <[hidden email]> wrote:
>
>> - but I have crashes with renderd with what's currently available on the
>> SVN repo
> Frustration about renderd was the main reason why tirex was born a few years
> ago.
>
> So you will probably give it a try instead of renderd.
>
> https://trac.openstreetmap.org/browser/applications/utils/tirex
>
> Regards
>
> Sven
>
Speaking of frustration:

- I downloaded Tirex and failed to compile it because some include file
wasn't provided in the mapnik F18 RPM
- I tried to recompile mapnik and failed because SConstruct wasn't able
to find boost thread lib while the package is installed
- I looked at the missing include file in Tirex code
(mapnik/expression.hpp included by backend-mapnik/metafilehandler.cc). I
discovered that I can compile without including this file.
- Then I ran into `mapnik-config --dep-libs lists` that returns
'-l[agg]' which breaks ld. I added a call to sed(1) to remove this from
mapnik-config output .

Tirex compilation completed, now I'm battling to understand where to fix
a mysterious "renderer internal error"  generated by
tirex-backend-manager: I have mod_tile talking to tirex-master, the
back-end manager sees the requests and then takes some time before
failing with this message for each tile to generate.

     Bernard

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Kai Krueger
In reply to this post by Bernard Fouché
On 03/26/2013 04:52 AM, Bernard Fouché wrote:
Hi All,

I'm new to openstreetmap and I've nearly succeeded in having a working tile server on a Fedora 18 box.

However I experience problems with mod_tile/renderd:

- all wiki pages I find simply state for instance:

Get and install mod_tile itself:
cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile
cd mod_tile
./autogen.sh
./configure

- Revision 29404, the current revision also has a bug in store_file.c:

...
    tmp = malloc(sizeof(char) * strlen(meta_path) + 12);
    sprintf(tmp, "%s.%lu", meta_path, pthread_self());

    if (mkdirp(tmp))
        free(tmp);
        return -1;

    fd = open(tmp, O_WRONLY | O_TRUNC | O_CREAT, 0666);
...

'{}' are missing around the free/return following a failed call to mkdirp(). https://github.com/openstreetmap/mod_tile/blob/master/store_file.c shows the same problem.
That bug has now been fixed in SVN (the github mirror hasn't caught up yet). Thanks for reporting it.


https://github.com/openstreetmap/mod_tile/commit/398310fdf662d56a283801a8274a072c07983a8b#store_file.c shows that the file was initially added with the bug.

Such a bug makes me think that the current revision isn't a stable one since it breaks to whole functioning of renderd and not a single tile can be generated.

So I'm looking for a SVN revision known to be stable (or a repo that holds stable revisions) and wasn't able to locate one yet.

There have been some major refactorings a couple of days ago in mod_tile / renderd to support new storage backends. That is when the error you reported got introduced. So if you take a snapshot from prior to March 23rd it should be more stable.

However, Fedora seems to have upgraded to Apache 2.4, and until a commit 2 days ago, mod_tile had build issues as the apache 2.4 and 2.2 apis are not compatible.

I am also hopping to expand http://ci.openstreetmap.org/ to provide automatic (build) testing on a variety of different platforms, including Fedora 18, so that errors with incompatibilities between systems can be spotted faster.

So far there are no releases or stable branches of mod_tile / renderd or osm2pgsql. However, as things mature and more and more people rely on them, it is time to have a more proper release process for these software.

Kai

Any hint welcome!

Thanks,

  Bernard




_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Sven Geggus
In reply to this post by Bernard Fouché
Bernard Fouché <[hidden email]> wrote:

> Tirex compilation completed, now I'm battling to understand where to fix
> a mysterious "renderer internal error"  generated by
> tirex-backend-manager: I have mod_tile talking to tirex-master, the
> back-end manager sees the requests and then takes some time before
> failing with this message for each tile to generate.

Both, tirex-master and tirex-backend-manager can be run in foreground in
debug mode (-d) which is very helpful. And then there is tirex-status, which
is also a very helpful tool (of course only after tirex is already running).

Sven

--
"linux is evolution, not intelligent design"
(Linus Torvalds)

/me is giggls@ircnet, http://sven.gegg.us/ on the Web

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
Le 26/03/2013 17:51, Sven Geggus a écrit :

> Bernard Fouché <[hidden email]> wrote:
>
>> Tirex compilation completed, now I'm battling to understand where to fix
>> a mysterious "renderer internal error"  generated by
>> tirex-backend-manager: I have mod_tile talking to tirex-master, the
>> back-end manager sees the requests and then takes some time before
>> failing with this message for each tile to generate.
> Both, tirex-master and tirex-backend-manager can be run in foreground in
> debug mode (-d) which is very helpful. And then there is tirex-status, which
> is also a very helpful tool (of course only after tirex is already running).
>
> Sven
In the end it was a font problem. All in all with the F18 mapnik RPM, I
still have to get the SVN mapnik version to get the osm.xml file + the
fonts. Messages like 'renderer internal error" don't help much! But at
least it seems to work correctly (no crash) and rendering seems faster
with Tirex than renderd.

Thanks,

   Bernard

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
In reply to this post by Kai Krueger
Le 26/03/2013 17:16, Kai Krueger a écrit :

There have been some major refactorings a couple of days ago in mod_tile / renderd to support new storage backends. That is when the error you reported got introduced. So if you take a snapshot from prior to March 23rd it should be more stable.

However, Fedora seems to have upgraded to Apache 2.4, and until a commit 2 days ago, mod_tile had build issues as the apache 2.4 and 2.2 apis are not compatible.

I am also hopping to expand http://ci.openstreetmap.org/ to provide automatic (build) testing on a variety of different platforms, including Fedora 18, so that errors with incompatibilities between systems can be spotted faster.

So far there are no releases or stable branches of mod_tile / renderd or osm2pgsql. However, as things mature and more and more people rely on them, it is time to have a more proper release process for these software.

Kai

Hello Kai,

I stopped using renderd because of the crashes I got (it can be a thread race condition bug since the bug disappears when running renderd with valgrind).

I fully agree about the need of a real release process: it took me many days to figure how to have Nominatim and Tirex running, mainly because the information is split in different wiki pages written from 2010 to early 2013 (I ignored ealier info)  and none of them reports the release numbers used when wiki entries were written. One also have to search in mailing list to fix issues that show up one after the other. It was a painful road, nearly each step of the installation brings a new step that does not work at first, it may be necessary to backtrack to some earlier step, change of version/package, retry, etc.

For instance F18 provides postgreql 9.2 but postgis 1.5: one has to get postgis 2 but also to apply legacy.sql, bring stuff from mapnik source code, get files (like coastlines) from here and there, figure how to configure postgresql (that I never used before) etc.

I'm actually trying to configure a secondary system to check that I didn't forget to note something, to be able to reinstall a system if this is needed in the future: I have currently a list of about *60* different things to do one after the other and I still have to add Tirex setup (and pray that the resulting system is a working one). I have to save copies of the versions of the different projects to be sure of what I use since I can't rely on stable versions of different components. And if someone is using that list in a couple of weeks, some parts will have to be updated/rechecked etc. so the installation is messy at best.

  Bernard


_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Yves
For the record, methods described in http://switch2osm.org works rather
well, thanks to Kai's packages.
Maybe packaging work could be shared with
https://launchpad.net/~ubuntugis too.

I somewhat would expect better results in building an osm tool-chain on
a debian-based system (Ubuntu), maybe F18 wasn't the good choice at the
beginning.

Yves

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
Le 27/03/2013 10:39, yvecai a écrit :

> For the record, methods described in http://switch2osm.org works
> rather well, thanks to Kai's packages.
> Maybe packaging work could be shared with
> https://launchpad.net/~ubuntugis too.
>
> I somewhat would expect better results in building an osm tool-chain
> on a debian-based system (Ubuntu), maybe F18 wasn't the good choice at
> the beginning.
>
> Yves
>
>
We have everything else on F-something, I want to avoid maintaining
different distros, what I may have spared using Ubuntu for the install I
lose it in the long term by having to support another distro.

My time was mostly spent because of the lack of stable releases and
documentation related to package versions, quirks here and there (for
instance when making Tirex there is no check of what perl packages are
already available, you discover the list of missing packages when Tirex
runs, or how do you know that you'll need to setup a definition for a
'default' map?) , broken SVN trunk of mod_tile, cryptic error messages,
configuration magic (when applying postgresql/system recommended values
for a 32GB system I saw it swapping, the values do not consider the
number of running cores in the system), some broken URL ("get that file
here" and you get a 404), etc., and of course, never having used OSM
related tools before.

Using Ubuntu may have save time on postgres/postgis issues, probably not
much else.

     Bernard


_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Pieren
On Wed, Mar 27, 2013 at 2:23 PM, Bernard Fouché
<[hidden email]> wrote:
> My time was mostly spent because of the lack of stable releases and
> documentation related to package versions, quirks here and there (for
> instance when making Tirex there is no check of what perl packages are
> already available, you discover the list of missing packages when Tirex
> runs, or how do you know that you'll need to setup a definition for a
> 'default' map?) , broken SVN trunk of mod_tile, cryptic error messages,
> configuration magic

OSM data & tools are mostly developed by volunteers on their free
time. Feel free to improve the wiki documentation if you find issues
or incompleteness. Creating an account for wiki edition takes less
time than writing your last 5 messages. Do something positive for the
project instead of complaining. Thus, the next one installing a tile
server on F18 will be grateful for your help.

Pieren

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
Le 27/03/2013 15:09, Pieren a écrit :

> On Wed, Mar 27, 2013 at 2:23 PM, Bernard Fouché
> <[hidden email]> wrote:
>> My time was mostly spent because of the lack of stable releases and
>> documentation related to package versions, quirks here and there (for
>> instance when making Tirex there is no check of what perl packages are
>> already available, you discover the list of missing packages when Tirex
>> runs, or how do you know that you'll need to setup a definition for a
>> 'default' map?) , broken SVN trunk of mod_tile, cryptic error messages,
>> configuration magic
> OSM data & tools are mostly developed by volunteers on their free
> time. Feel free to improve the wiki documentation if you find issues
> or incompleteness. Creating an account for wiki edition takes less
> time than writing your last 5 messages. Do something positive for the
> project instead of complaining. Thus, the next one installing a tile
> server on F18 will be grateful for your help.
>
> Pieren
I apologize if the feed back I gave seems to be a complaint, it is not,
I'm very happy to have an alternative to Google Maps and similar systems.

I can update a wiki page, but what wiki page? When googling
"openstreetmap fedora install" there are different pages that show up
and I had to look at all of these. What should I write about obtaining
mod_tile or osm2pgsql? What revisions a user is supposed to get to have
a working system? I can list the revision I have but in a few days, or
even already, it may be better to get a later version: I'm the last to
be able to declare that a particular version is better than another. At
the moment I can write the complete set of things I had to consider to
have a working system, that would give a general view of the set of
problems to handle, but it will be far from a "do this and it will 100%
work" recipe, mostly because of the lack of stable versions for the most
important components of the tool chain and the lack of a set of stable
versions known to work correctly together.

I don't complain, I supplicate stable versions :-)

   Bernard

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Sven Geggus
In reply to this post by Bernard Fouché
Bernard Fouché <[hidden email]> wrote:

> My time was mostly spent because of the lack of stable releases and
> documentation related to package versions, quirks here and there (for
> instance when making Tirex there is no check of what perl packages are
> already available

Hm, this is at least kind of a distro problem. If debian packages are build
the required packages are installed automatically via dependencies.

Sven

--
"Der wichtigste Aspekt, den Sie vor der Entscheidung für ein Open
Source-Betriebssystem bedenken sollten, ist, dass Sie kein
Windows-Betriebssystem erhalten." (von http://www.dell.de/ubuntu)
/me is giggls@ircnet, http://sven.gegg.us/ on the Web

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Andy Allan
In reply to this post by Pieren
On 27 March 2013 14:09, Pieren <[hidden email]> wrote:

> OSM data & tools are mostly developed by volunteers on their free
> time.

And I'm sure we all strive to make high-quality tools. If someone
encounters bugs, like crashes and broken documentation, then even just
being told by email is valuable, or logging tickets, or mentioning it
on IRC. There's no need to round on them for not having fixed all the
problems single-handedly.

Of course, some people have unreasonable expectations, and we all tire
of their demands. But things like being able to compile the software
is a perfectly reasonable expectation.

> Feel free to improve the wiki documentation if you find issues
> or incompleteness. Creating an account for wiki edition takes less
> time than writing your last 5 messages. Do something positive for the
> project instead of complaining. Thus, the next one installing a tile
> server on F18 will be grateful for your help.

And this is one of the reasons our documentation is such a mess. With
the best will in the world, someone who doesn't understand the
components and has just managed to get them working - for the very
first time - isn't in the best position to write clear documentation.

The osm2pgsql page on the wiki, like many others, is degrading over
the years as people add to it and nobody edits it. Such pages are not
converging towards "great documentation", they start off as useful
documentation and slowly degenerate, while accumulating increasing
amounts of cruft, inconsistencies and even inline-patches that you are
encouraged to apply. Sheesh. And that doesn't even count the
increasing numbers of other pages you can stumble across which
duplicate one another and confuse anyone who reads them.

So think twice before telling a new user that they should start
editing wiki pages.

Cheers,
Andy

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Kai Krueger
In reply to this post by Bernard Fouché
On 03/27/2013 03:12 AM, Bernard Fouché wrote:
Le 26/03/2013 17:16, Kai Krueger a écrit :

There have been some major refactorings a couple of days ago in mod_tile / renderd to support new storage backends. That is when the error you reported got introduced. So if you take a snapshot from prior to March 23rd it should be more stable.

However, Fedora seems to have upgraded to Apache 2.4, and until a commit 2 days ago, mod_tile had build issues as the apache 2.4 and 2.2 apis are not compatible.

I am also hopping to expand http://ci.openstreetmap.org/ to provide automatic (build) testing on a variety of different platforms, including Fedora 18, so that errors with incompatibilities between systems can be spotted faster.

So far there are no releases or stable branches of mod_tile / renderd or osm2pgsql. However, as things mature and more and more people rely on them, it is time to have a more proper release process for these software.

Kai

Hello Kai,

I stopped using renderd because of the crashes I got (it can be a thread race condition bug since the bug disappears when running renderd with valgrind).
I think that crash was a buffer overflow and should also now be fixed, thanks to Jon spotting the error and the help of some people on the irc channel #osm-dev. This bug was also introduced just 4 days days ago.

Normally mod_tile and renderd should be pretty stable with not many bigger changes. So normally taking the latest SVN snapshot should work fairly reliable. Unfortunately you seem to have hit just the two or three days where I committed some bigger changes to help mod_tile scale up to larger installations that caused some instability.

I fully agree about the need of a real release process: it took me many days to figure how to have Nominatim and Tirex running, mainly because the information is split in different wiki pages written from 2010 to early 2013 (I ignored ealier info)  and none of them reports the release numbers used when wiki entries were written.
Yes, the information is split into too many different wiki pages, which makes keeping them updated pretty much an impossibility. I think that is at least partly the reason why Richard started the webpage switch2osm.org to have a single source of good documentation of how to set things up. Of cause as any documentation (in OSM) there is also room for improvement and expansion but I think that is probably the most up to date and clear descriptions of how to set up the tile rendering infrastructure. It could probably do with an equivalent description for nominatim.
One also have to search in mailing list to fix issues that show up one after the other. It was a painful road, nearly each step of the installation brings a new step that does not work at first, it may be necessary to backtrack to some earlier step, change of version/package, retry, etc.
Well, it is a system with many components. I always try to make the installation process as easy as possible and improve the error messages to more easily track down setup issues, but the fact that every single linux distribution has its own set of versions of key components, has different packaging systems, puts files into different locations and therefore unique set of issues doesn't directly make that easier. And that is without even touching other unix derivatives like freeBSD, Solaris or Mac OSX, let alone Windows.

As mentioned I'd like to try and automate more of the testing on different systems, but so far that infrastructure doesn't exist in OSM. So all we have to go on at the moment are bug reports by people

If you have any suggestions on how to improve the situation further, I'd be interested to here them.

For instance F18 provides postgreql 9.2 but postgis 1.5: one has to get postgis 2 but also to apply legacy.sql, bring stuff from mapnik source code, get files (like coastlines) from here and there, figure how to configure postgresql (that I never used before) etc.
Well, for Ubuntu I have created the PPA packages as linked to on switch2osm.org. Those try and take care of all the necessary steps as much as possible. To the extent even that they make many default decision on things  and do other "magic setup steps" which violate the official packaging guidelines. However, as mentioned above, it is unfortunately difficult to create similar packages for all the different systems, as there are simply not enough developers around to do that.

However, there are definitely things that can and should be improved, and figuring out a better release process is clearly one of them.

Kai

I'm actually trying to configure a secondary system to check that I didn't forget to note something, to be able to reinstall a system if this is needed in the future: I have currently a list of about *60* different things to do one after the other and I still have to add Tirex setup (and pray that the resulting system is a working one). I have to save copies of the versions of the different projects to be sure of what I use since I can't rely on stable versions of different components. And if someone is using that list in a couple of weeks, some parts will have to be updated/rechecked etc. so the installation is messy at best.

  Bernard



_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

PPA vs 64bit IDs

Lynn W. Deffenbaugh (Mr)
On 3/27/2013 2:03 PM, Kai Krueger wrote:
Well, for Ubuntu I have created the PPA packages as linked to on switch2osm.org. Those try and take care of all the necessary steps as much as possible. To the extent even that they make many default decision on things  and do other "magic setup steps" which violate the official packaging guidelines. However, as mentioned above, it is unfortunately difficult to create similar packages for all the different systems, as there are simply not enough developers around to do that.

I'm just about to rebuild my tile server and was wondering if the PPA packages fully support the 64bit IDs?

Lynn (D)

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Pieren
In reply to this post by Andy Allan
On Wed, Mar 27, 2013 at 6:33 PM, Andy Allan <[hidden email]> wrote:

> So think twice before telling a new user that they should start
> editing wiki pages.

Instead of 'new user', I would say 'the end users, those from whom the
documentation is intended'. Since the devs do not take a high priority
to keep the wiki up-to-date and consistent (sigh), anybody else taking
the time to search, read the docs with new eyes, follow the process,
find and fix issues for himself, etc is able to improve the doc for
the next ones.
I said "improve the doc", not "write another 'how-to setup my own tile server'".

Pieren

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Kai Krueger
In reply to this post by Andy Allan
Andy Allan wrote
On 27 March 2013 14:09, Pieren <[hidden email]> wrote:

> OSM data & tools are mostly developed by volunteers on their free
> time.

And I'm sure we all strive to make high-quality tools. If someone
encounters bugs, like crashes and broken documentation, then even just
being told by email is valuable, or logging tickets, or mentioning it
on IRC.
Indeed, although we as developers obviously try and catch as many bugs and variants as possible before committing things, in the end we do rely on people reporting bugs so that they can be fixed.

And reporting bugs often does work. For example I just had a very pleasant experience with that today. A couple of weeks (or months ago), I was trying to get renderd working with the latest snapshot of mapnik. It failed due to a projection bug in (the unstable branch of) mapnik. However, instead of reporting it, I just used mapnik 2.1 instead. Now yesterday some one else on IRC ran into the same issue  again and wasted more time with it.  After finally reporting it, the bug was fixed in less than 24 hours and no one else needs to run into the issue any more. And in the process, I needed to know nothing about the internals of Mapnik or be able to debug it, so it is something that anyone can do.


Granted, the bug reports in mod_tile / renderd have not always been dealt with that quickly.

Partly I think it is due to the fact that there are too many channels to report bugs through. E.g. the osm track ticketing system, the github osm repository issue tracking, the various forums, help.osm.org the mailing list, irc, the wiki and other random pages were people mention / complain about problems.

I am hoping to improve the situation, yes by creating yet another communication channel... ;-) There is a new mailing list tile-serving ( http://lists.openstreetmap.org/listinfo/tile-serving ) where hopefully all those bug reports from different sources can get collated and make sure they reach all the people who can potentially fix them. Hopefully that will improve the response time to bug reports and thus make overall better and more reliable software.

Andy Allan wrote
 There's no need to round on them for not having fixed all the
problems single-handedly.

Of course, some people have unreasonable expectations, and we all tire
of their demands. But things like being able to compile the software
is a perfectly reasonable expectation.
Indeed, it would be helpful if users better realise the limitations of developers and their time to spend on hoby projects, but at least as useful would be if developers don't always respond to requests with the "Oh it is open source, go fix it your self" attitude, as "fixing it your self" is clearly not a reasonable response for the majority of people who don't have the necessary specific skills.

Kai
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
In reply to this post by Pieren
Le 27/03/2013 19:13, Pieren a écrit :

> On Wed, Mar 27, 2013 at 6:33 PM, Andy Allan <[hidden email]> wrote:
>
>> So think twice before telling a new user that they should start
>> editing wiki pages.
> Instead of 'new user', I would say 'the end users, those from whom the
> documentation is intended'. Since the devs do not take a high priority
> to keep the wiki up-to-date and consistent (sigh), anybody else taking
> the time to search, read the docs with new eyes, follow the process,
> find and fix issues for himself, etc is able to improve the doc for
> the next ones.
> I said "improve the doc", not "write another 'how-to setup my own tile server'".
>
> Pieren
>
> _______________________________________________

No problem with Pieren's message, I know how painful it is when you work
all day on something and a newcomer complains because of a free lunch
not included yet.

Anyway I'm close to finish an ansible playbook to setup a VM just
installed with the core F18 packages, to have a working tile server +
nominatim. I choose ansible because it is extremely easy to setup and
requires no server in the system being targeted, but sshd and to have
setup ~/.ssh/authorized_keys. It seems a reasonable approach to list
exhaustively every step that must be done.

Of course this playbook needs to be checked by people knowing the
different OSM components, I probably have done very stupid things and I
also use ansible for a few days and didn't take time to make the
playbook a very nice one (for instance I had problems understanding
symlinks handling so I just remove them and 'ln -fs' them afterwards).
So much to do and not much time available so I have to follow an
aggressive approach about everything these days :-)

Bu there is the problem of copying files (.tgz/conf/.sh/etc.) that I
built during the process. For instance for Tirex I applied such a patch:

Index: backend-mapnik/Makefile
===================================================================
--- backend-mapnik/Makefile (revision 29404)
+++ backend-mapnik/Makefile (working copy)
@@ -1,7 +1,7 @@
  INSTALLOPTS=-g root -o root
  CFLAGS +=   -Wall -Wextra -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
  CXXFLAGS = $(CFLAGS) `mapnik-config --cflags`
-LDFLAGS= `mapnik-config --libs --ldflags --dep-libs`
+LDFLAGS= `mapnik-config --libs --ldflags --dep-libs| sed -e s/-l.agg.//`
 
  backend-mapnik: renderd.o metatilehandler.o networklistener.o networkmessage.o networkrequest.o networkresponse.o debuggable.o requesthandler.o
  $(CXX) -o $@ $^ $(LDFLAGS)
Index: backend-mapnik/metatilehandler.cc
===================================================================
--- backend-mapnik/metatilehandler.cc (revision 29404)
+++ backend-mapnik/metatilehandler.cc (working copy)
@@ -23,7 +23,7 @@
  #include <mapnik/datasource_cache.hpp>
  #include <mapnik/font_engine_freetype.hpp>
  #include <mapnik/agg_renderer.hpp>
-#include <mapnik/expression.hpp>
+//#include <mapnik/expression.hpp>
  #include <mapnik/image_util.hpp>
  #include <mapnik/load_map.hpp>
  #include <mapnik/box2d.hpp>

So for Tirex I made up a .tgz of the revision I got including this
patch, and have ansible to copy it to the target and recompile it
locally. I also copy mapnik fonts, there still a few 'wget svn..' (with
no revision number ;-)), etc: this is purely an approach to reach a
working system and to avoid digging into the details to make it a
perfect one.

   Bernard

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Dane Springmeyer

>  For instance for Tirex I applied such a patch:
>
> Index: backend-mapnik/Makefile
> ===================================================================
> --- backend-mapnik/Makefile (revision 29404)
> +++ backend-mapnik/Makefile (working copy)
> @@ -1,7 +1,7 @@
> INSTALLOPTS=-g root -o root
> CFLAGS +=   -Wall -Wextra -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
> CXXFLAGS = $(CFLAGS) `mapnik-config --cflags`
> -LDFLAGS= `mapnik-config --libs --ldflags --dep-libs`
> +LDFLAGS= `mapnik-config --libs --ldflags --dep-libs| sed -e s/-l.agg.//`

^^ this looks either like a mapnik-config bug or a packaging bug. Working around in tirex is fine, but personally I would want to understand why -lagg is in the config flags in the first place before fixing this way. As far as I can tell there is not a mapnik version I see that reports -lagg like this.

Can you provide more details about how you installed mapnik and the exact version of mapnik you see this with to a support ticket at:

https://github.com/mapnik/mapnik/issues

Also, if you are using mapnik from some fedora package, then this may be a bug in fedora packaging. Please provide details.


> Index: backend-mapnik/metatilehandler.cc
> ===================================================================
> --- backend-mapnik/metatilehandler.cc (revision 29404)
> +++ backend-mapnik/metatilehandler.cc (working copy)
> @@ -23,7 +23,7 @@
> #include <mapnik/datasource_cache.hpp>
> #include <mapnik/font_engine_freetype.hpp>
> #include <mapnik/agg_renderer.hpp>
> -#include <mapnik/expression.hpp>
> +//#include <mapnik/expression.hpp>

Thanks for reporting, this is now fixed: https://trac.openstreetmap.org/changeset/29407/subversion

Note that I am not a tirex user or developer and I did not test this but as a Mapnik developer I know the header is both 1) not needed and 2) was only added in Mapnik 2.x, so including it makes for harder compatibility with multiple versions of Mapnik. So, nice catch.

> So for Tirex I made up a .tgz of the revision I got including this patch, and have ansible to copy it to the target and recompile it locally. I also copy mapnik fonts,

You should not need to copy mapnik fonts around. Sounds like a configuration problem or something messed up with the install.

Dane
_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: mod_tile stable version ?

Bernard Fouché
Le 28/03/2013 02:44, Dane Springmeyer a écrit :

>>   For instance for Tirex I applied such a patch:
>>
>> Index: backend-mapnik/Makefile
>> ===================================================================
>> --- backend-mapnik/Makefile (revision 29404)
>> +++ backend-mapnik/Makefile (working copy)
>> @@ -1,7 +1,7 @@
>> INSTALLOPTS=-g root -o root
>> CFLAGS +=   -Wall -Wextra -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
>> CXXFLAGS = $(CFLAGS) `mapnik-config --cflags`
>> -LDFLAGS= `mapnik-config --libs --ldflags --dep-libs`
>> +LDFLAGS= `mapnik-config --libs --ldflags --dep-libs| sed -e s/-l.agg.//`
> ^^ this looks either like a mapnik-config bug or a packaging bug. Working around in tirex is fine, but personally I would want to understand why -lagg is in the config flags in the first place before fixing this way. As far as I can tell there is not a mapnik version I see that reports -lagg like this.
>
> Can you provide more details about how you installed mapnik and the exact version of mapnik you see this with to a support ticket at:
>
> https://github.com/mapnik/mapnik/issues
>
> Also, if you are using mapnik from some fedora package, then this may be a bug in fedora packaging. Please provide details.
Hello Dan,

done here: https://github.com/mapnik/mapnik/issues/1779

>
>
>> Index: backend-mapnik/metatilehandler.cc
>> ===================================================================
>> --- backend-mapnik/metatilehandler.cc (revision 29404)
>> +++ backend-mapnik/metatilehandler.cc (working copy)
>> @@ -23,7 +23,7 @@
>> #include <mapnik/datasource_cache.hpp>
>> #include <mapnik/font_engine_freetype.hpp>
>> #include <mapnik/agg_renderer.hpp>
>> -#include <mapnik/expression.hpp>
>> +//#include <mapnik/expression.hpp>
> Thanks for reporting, this is now fixed: https://trac.openstreetmap.org/changeset/29407/subversion
>
> Note that I am not a tirex user or developer and I did not test this but as a Mapnik developer I know the header is both 1) not needed and 2) was only added in Mapnik 2.x, so including it makes for harder compatibility with multiple versions of Mapnik. So, nice catch.

As stated in the github issue report I just did, I'm using mapnik
packages tagged 2.0.0-7 : these packages are broken or this file
appeared after 2.0.0-7.

>
>> So for Tirex I made up a .tgz of the revision I got including this patch, and have ansible to copy it to the target and recompile it locally. I also copy mapnik fonts,
> You should not need to copy mapnik fonts around. Sounds like a configuration problem or something messed up with the install.

In which packages should the fonts appear? I reported the font problem
in the same issue, which seems a broken set of RPM for F18 and/or
missing packages.

   Bernard

_______________________________________________
dev mailing list
[hidden email]
http://lists.openstreetmap.org/listinfo/dev
12