Is gosmore broken with the latest planet file?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Is gosmore broken with the latest planet file?

Petter Reinholdtsen

Is gosmore broken?  It crashes when I try to load the latest planet
file.  It work with an older file (planet-061227.osm.bz2).

This is the error message and how it crashes (gdb backtrace):

  *** glibc detected *** double free or corruption (out): 0xa9002008 ***

  Program received signal SIGABRT, Aborted.
  [Switching to Thread -1218394432 (LWP 9651)]
  0xb7fdc410 in ?? ()
  (gdb) bt
  #0  0xb7fdc410 in ?? ()
  #1  0xbfc5abbc in ?? ()
  #2  0x00000006 in ?? ()
  #3  0x000025b3 in ?? ()
  #4  0xb7747811 in raise () from /lib/tls/i686/cmov/libc.so.6
  #5  0xb7748fb9 in abort () from /lib/tls/i686/cmov/libc.so.6
  #6  0xb777cd3a in __fsetlocking () from /lib/tls/i686/cmov/libc.so.6
  #7  0xb77845cf in mallopt () from /lib/tls/i686/cmov/libc.so.6
  #8  0xb7784672 in free () from /lib/tls/i686/cmov/libc.so.6
  #9  0x0804de87 in main (argc=2, argv=0xbfc5b4c4) at gosmore.cc:1035
  (gdb)

Any idea how to fix it?

BTW: I just uploaded a Debian package of gosmore based on the gosmore
     code in subversion to Debian.  I expect it to show up in the
     archive within the month.

Happy hacking,
--
Petter Reinholdtsen


_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Is gosmore broken with the latest planet file?

Nic Roets
I wouldn't be able to investigate the problem soon, but if I have to
make a guess, it would have to be that the hardcoded constants need
adjustment after the planet file has grown. If you have >= 1GB RAM,
you can try this:

line 38:
#define MAX_NODES 30100800
#define MAX_SEGMENTS 30100800
#define MAX_NAMED_NODES 150000
line 891 :
  wayBuildType *w = (wayBuildType *) calloc (sizeof (*w), 3000000);

_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Is gosmore broken with the latest planet file?

Petter Reinholdtsen
[Nic Roets]
> I wouldn't be able to investigate the problem soon, but if I have to
> make a guess, it would have to be that the hardcoded constants need
> adjustment after the planet file has grown.

Right.  Sounds like something should give a better error message
instead of crashing with a bogus free(). :)

> If you have >= 1GB RAM, you can try this:
>
> line 38:
> #define MAX_NODES 30100800
> #define MAX_SEGMENTS 30100800
> #define MAX_NAMED_NODES 150000
> line 891 :
>   wayBuildType *w = (wayBuildType *) calloc (sizeof (*w), 3000000);

Right.  I foudn the three top ones, but not the one deep down in the
code.  What is the last value, and is it an idea to move it up among
the other constants?

Increasing the values helped.  Now the program completed without
crashing, and draw the updated map.  Should I increase the values in
svn?  I just add *2 to the values, but suspect some smaller value
would be a better choice.

Is there some way to get gosmore to use less memory?  It is getting
harder and harder to find a machine to run it on. :)

Happy hacking,
--
Petter Reinholdtsen

_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Is gosmore broken with the latest planet file?

Nic Roets
In reply to this post by Nic Roets
>Should I increase the values in svn?

Good idea.

>Is there some way to get gosmore to use less memory?

Not high up on the TODO list.

_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Is gosmore broken with the latest planet file?

Petter Reinholdtsen
[Nic Roets]
>> Should I increase the values in svn?
>
> Good idea.

Done.  I added 20% to all the values, and will see if that is enough.

>> Is there some way to get gosmore to use less memory?
>
> Not high up on the TODO list.

OK.

> line 891 :
>   wayBuildType *w = (wayBuildType *) calloc (sizeof (*w), 3000000);

Your code do not seem to be the same as the code in svn.  The code in
svn had 2000000 as this value.  I am guessing it is the max number of
ways, and moved it as a constant next to the others using MAX_WAYS as
the name.

I've added a simple README file to svn.  I would like to include
information about where to send patches and improvements.  What is the
prefered address for that?

Also, would it be an idea to change the path to gosmore.pak from the
current directory to some location under the users home directory like
~/.gosmore/gosmore.pak, to make the program more useful to start from
a graphical desktop using the menu?

Happy hacking,
--
Petter Reinholdtsen

_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Is gosmore broken with the latest planet file?

Nic Roets
In reply to this post by Nic Roets
>>   wayBuildType *w = (wayBuildType *) calloc (sizeof (*w), 3000000);

That was the new thumb suck value.

> I would like to include information about where to send patches and
> improvements

I don't mind where the latest code is stored, as long as
http://wiki.openstreetmap.org/index.php/Gosmore
tells you where the "best" version is and what it does.

> Also, would it be an idea to change the path to gosmore.pak
Feel free to change it. For creating packages a short term solution
may be to hardcode it to something like /usr/share/gosmore/gosmore.pak

_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: Is gosmore broken with the latest planet file?

Petter Reinholdtsen
[Nic Roets]
> >>   wayBuildType *w = (wayBuildType *) calloc (sizeof (*w), 3000000);
>
> That was the new thumb suck value.

I am not sure I understand this comment.  Some English idiom?

>> I would like to include information about where to send patches and
>> improvements
>
> I don't mind where the latest code is stored, as long as
> http://wiki.openstreetmap.org/index.php/Gosmore
> tells you where the "best" version is and what it does.

I mean, which email address to recommend as the contact address for
gosmore.  It is lacking from the wiki page and the README file.

>> Also, would it be an idea to change the path to gosmore.pak
> Feel free to change it. For creating packages a short term solution
> may be to hardcode it to something like /usr/share/gosmore/gosmore.pak

I suspect /usr/share/gosmore/gosmore.pak, /var/lib/gosmore/gosmore.pak
and ~/.gosmore/gosmore.pak should be checked, in that order, and when
writing it should try to write to /var/lib/gosmore/gosmore.pak first
and fail over to ~/.gosmore/gosmore.pak if the access is missing.  At
least to me that seem closest to the Linux Software Base spec.

I'll see if I can find time to code such behaviour.

Happy hacking,
--
Petter Reinholdtsen

_______________________________________________
talk mailing list
[hidden email]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk