How to be more efficient inserting tracks?

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

How to be more efficient inserting tracks?

Johnny Doe-2
>> - The applet is slow on my 2.4 GHz box (Linux,
Firefox, SUN Jre)
> Maybe a problem of memory? How much RAM do you have?
> How much memory do you allow your Java task to have?

Is dual 2.4GHz opteron with 8Gig of RAM good enough
for java ?
I'm not joking, the applet performance on this machine
in firefox is simply pathetic.
I'll definitely go the osm-editor route
when its track deleting bugs are corrected.



               
__________________________________
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Steve Coast
* @ 24/10/05 10:28:52 PM [hidden email] wrote:
> >> - The applet is slow on my 2.4 GHz box (Linux,
> Firefox, SUN Jre)
> > Maybe a problem of memory? How much RAM do you have?
> > How much memory do you allow your Java task to have?
>
> Is dual 2.4GHz opteron with 8Gig of RAM good enough

opteron? Is your jdk compiled for that? Does it JIT to opteron or is it
non-JIT?

have fun,

SteveC [hidden email] http://www.asklater.com/steve/

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Immanuel Scholz
Hi,

> > Is dual 2.4GHz opteron with 8Gig of RAM good enough
>
> opteron? Is your jdk compiled for that? Does it JIT to opteron or is it
> non-JIT?

I could reproduce the problem with my Athlon running 32-bit linux (1 Gig
of RAM, JRE1.5.0_04 from Sun).

The applet is consuming 100% of my cpu, slowing down mouse movement and
draining overall performance. This does not look like typical memory
problems (no swap access, steady performance drain instead of periodical
total blocks etc.) It happens regardless of how big I set the -Xmx
parameter.

No net access. It just looks like the CPU. I don't got these problems
running Java under Windows (JRE1.5 from Sun, Firefox too)

renicing the java process helps. It still consume 100% CPU, but now the
rest of the system is not involved.


Ciao, Imi.

PS: Maybe someone move this to bugzilla?

PPS: To Johnny: Open a root-shell and start "top". Then press r and
enter the id of the java_vm - process (the top one ;). Then renice it
to priority 20. This helps as a quick fix.



_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Nick Whitelegg
In reply to this post by Johnny Doe-2

> Is dual 2.4GHz opteron with 8Gig of RAM good enough
> for java ?
> I'm not joking, the applet performance on this machine
> in firefox is simply pathetic.
> I'll definitely go the osm-editor route
> when its track deleting bugs are corrected.

Could you give me details of the track deleting bugs?

Thanks,
Nick

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Adam Bower
In reply to this post by Johnny Doe-2
On Mon, Oct 24, 2005 at 03:28:52PM -0700, Johnny Doe wrote:

> >> - The applet is slow on my 2.4 GHz box (Linux,
> Firefox, SUN Jre)
> > Maybe a problem of memory? How much RAM do you have?
> > How much memory do you allow your Java task to have?
>
> Is dual 2.4GHz opteron with 8Gig of RAM good enough
> for java ?
> I'm not joking, the applet performance on this machine
> in firefox is simply pathetic.
> I'll definitely go the osm-editor route
> when its track deleting bugs are corrected.

For the first time tonight I tried the applet on my desktop in Firefox
and it was so slow that I couldn't get it to do anything useful (apart
from consume 100% of my cpu), I figured maybe java was broken on this
machine so moved to using the (spit) Windows box, with both IE and
Firefox and again... 100% cpu, couldn't get the applet to do anything
useful (i could get it to display the points in the track, etc. but not
actually get it to create any paths etc.):(

Oh, and I did try sending an email to [hidden email] earlier,
but it bounced as there is no MX record for openstreetmap.org.

Thanks
Adam
--
jabberid = [hidden email]
AFFS || http://www.affs.org.uk/ || Not a filesystem

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Johnny Doe-2
In reply to this post by Steve Coast

--- SteveC <[hidden email]> wrote:
>
> opteron? Is your jdk compiled for that?
>
No. It is just a standard SuSE 10.0 install
with 32bit java, 32bit firefox, 32bit openoffice
and 32bit eclipse. I didn't want to mess with this
setup, but it should not be a real problem.



       
               
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Steve Coast
In reply to this post by Adam Bower
* @ 24/10/05 11:16:22 PM [hidden email] wrote:

> On Mon, Oct 24, 2005 at 03:28:52PM -0700, Johnny Doe wrote:
> > >> - The applet is slow on my 2.4 GHz box (Linux,
> > Firefox, SUN Jre)
> > > Maybe a problem of memory? How much RAM do you have?
> > > How much memory do you allow your Java task to have?
> >
> > Is dual 2.4GHz opteron with 8Gig of RAM good enough
> > for java ?
> > I'm not joking, the applet performance on this machine
> > in firefox is simply pathetic.
> > I'll definitely go the osm-editor route
> > when its track deleting bugs are corrected.
>
> For the first time tonight I tried the applet on my desktop in Firefox
> and it was so slow that I couldn't get it to do anything useful (apart
> from consume 100% of my cpu), I figured maybe java was broken on this
> machine so moved to using the (spit) Windows box, with both IE and
> Firefox and again... 100% cpu, couldn't get the applet to do anything
> useful (i could get it to display the points in the track, etc. but not
> actually get it to create any paths etc.):(

With all the positive comments I rushed through a new copy of the applet
which is live now.

It

* doesn't use 100% CPU all the time
* makes the nodes and lines thicker
* makes it so you don't have to be directly on a node to select/kill it.

You might need to restart your browser / kill the cache to see the new
applet.

There might have been a couple of bugs introduced.

> Oh, and I did try sending an email to [hidden email] earlier,
> but it bounced as there is no MX record for openstreetmap.org.

EEK. I'm on it.

have fun,

SteveC [hidden email] http://www.asklater.com/steve/

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Johnny Doe-2
In reply to this post by Nick Whitelegg

--- [hidden email] wrote:
>
> Could you give me details of the track deleting
> bugs?
>

If you delete the last (or the only) point in a
trackseg, the program crashes. If one trackseg is
empty, osm-editor
crashes on loading the gpx file.
While we are at it, there are two other usability
concerns with osm-editor:
1. There is no way to rescale the canvas to the
   track bounding box, which results in searching for
   it through the whole world.
2. it casts the utf-8 strings to char *. Looks
unpleasant when you leave the UK :)

Maybe it is possible to adapt the gpx plugin
  in qgis http://qgis.org to write out the
OSM-compatible gpx ? This will provide a fullscale GIS
environment to openstreetmap, although i like the
lightweight nature of osm-editor.



       
               
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

How to be more efficient inserting tracks?

Johnny Doe-2
In reply to this post by Steve Coast

--- SteveC <[hidden email]> wrote:
>
> With all the positive comments I rushed through a
> new copy of the applet
> which is live now.
>
 
 Steve,
 
  nobody can deny an important detail: the applet
  works :)



               
__________________________________
Start your day with Yahoo! - Make it your home page!
http://www.yahoo.com/r/hs

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

frank mohr
In reply to this post by Immanuel Scholz
Immanuel Scholz wrote:

> Hi,
>
>
>>>Is dual 2.4GHz opteron with 8Gig of RAM good enough
>>
>>opteron? Is your jdk compiled for that? Does it JIT to opteron or is it
>>non-JIT?
>
>
> I could reproduce the problem with my Athlon running 32-bit linux (1 Gig
> of RAM, JRE1.5.0_04 from Sun).
>
> The applet is consuming 100% of my cpu, slowing down mouse movement and
> draining overall performance. This does not look like typical memory
> problems (no swap access, steady performance drain instead of periodical
> total blocks etc.) It happens regardless of how big I set the -Xmx
> parameter.

seems to be a CPU/JRE problem
i just tried it on a AthlonXP 2000 with 512 MB
with sun-j2re1.5-1.5.0+update01 and Mozilla (on Debian sarge)

java uses between 9 and 25% CPU time

but i found a different problem:

i deleted some nodes/lines from my failed Griesheim upload
and while i deleted some lines, other already deleted
nodes and lines reapeared in other regions on the map

frank

       

       
               
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de


_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Nick Whitelegg-2
In reply to this post by Johnny Doe-2






Johnny Doe <[hidden email]>@vr.ucl.ac.uk on 25/10/2005 00:49:09



>If you delete the last (or the only) point in a
>trackseg, the program crashes. If one trackseg is
>empty, osm-editor
>crashes on loading the gpx file.

OK thanks. I think someone may have mentioned one of these bugs before.
Should be easily fixed.

>While we are at it, there are two other usability
>concerns with osm-editor:
>1. There is no way to rescale the canvas to the
>track bounding box, which results in searching for
>it through the whole world.

OK, that's easily done.

>2. it casts the utf-8 strings to char *. Looks
>unpleasant when you leave the UK :)

The cURL library (used for communication) uses char *, not sure what the
solution to this is. Find a communications library which supports Unicode?

>Maybe it is possible to adapt the gpx plugin
>in qgis http://qgis.org to write out the
>OSM-compatible gpx ? This will provide a fullscale GIS
>environment to openstreetmap, although i like the
>lightweight nature of osm-editor.

The lightweight nature is deliberate :-) Also when writing osm-editor there
was a specific requirement I had (particularly important in surveying
countryside paths) for dividing up the track into particular types of
segment e.g. path, bridleway, cycle path etc, and no existing software
seemed tailored to that particular requirement. However it would obviously
be a good idea to have as many OSM clients as possible so adapting existing
packages (in addition to osm-editor, JOSM, the applet....) seems a good
plan.

Nick




__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap





_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Steve Coast
In reply to this post by frank mohr
* @ 25/10/05 08:56:31 AM [hidden email] wrote:

> Immanuel Scholz wrote:
> > Hi,
> >
> >
> >>>Is dual 2.4GHz opteron with 8Gig of RAM good enough
> >>
> >>opteron? Is your jdk compiled for that? Does it JIT to opteron or is it
> >>non-JIT?
> >
> >
> > I could reproduce the problem with my Athlon running 32-bit linux (1 Gig
> > of RAM, JRE1.5.0_04 from Sun).
> >
> > The applet is consuming 100% of my cpu, slowing down mouse movement and
> > draining overall performance. This does not look like typical memory
> > problems (no swap access, steady performance drain instead of periodical
> > total blocks etc.) It happens regardless of how big I set the -Xmx
> > parameter.
>
> seems to be a CPU/JRE problem
> i just tried it on a AthlonXP 2000 with 512 MB
> with sun-j2re1.5-1.5.0+update01 and Mozilla (on Debian sarge)
>
> java uses between 9 and 25% CPU time
>
> but i found a different problem:
>
> i deleted some nodes/lines from my failed Griesheim upload
> and while i deleted some lines, other already deleted
> nodes and lines reapeared in other regions on the map

On the static map? Those images are cached, reload them.

have fun,

SteveC [hidden email] http://www.asklater.com/steve/

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

frank mohr


--- SteveC <[hidden email]> wrote:
> > but i found a different problem:
> >
> > i deleted some nodes/lines from my failed Griesheim upload
> > and while i deleted some lines, other already deleted
> > nodes and lines reapeared in other regions on the map
>
> On the static map? Those images are cached, reload them.

no it was in the editor-applet

frank


       
               
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Adam Bower
In reply to this post by Steve Coast
On Mon, Oct 24, 2005 at 11:49:17PM +0000, SteveC wrote:
>
> With all the positive comments I rushed through a new copy of the applet
> which is live now.

I'm much happier with the new applet, I actually got some data entered
with it!

Thanks
Adam
--
jabberid = [hidden email]
AFFS || http://www.affs.org.uk/ || Not a filesystem

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

frank mohr
In reply to this post by frank mohr
frank mohr wrote:

>
> --- SteveC <[hidden email]> wrote:
>
>>>but i found a different problem:
>>>
>>>i deleted some nodes/lines from my failed Griesheim upload
>>>and while i deleted some lines, other already deleted
>>>nodes and lines reapeared in other regions on the map
>>
>>On the static map? Those images are cached, reload them.
>
>
> no it was in the editor-applet
>
> frank

i had an idea last night what might have happened

the best explanation is that the applet has a limitation
for the amount of downloaded nodes and lines

- i had a very high amount of small lines
  (the upload was about 60 different gpstracks
  of the same street converted to nodes/lines)
- as those lines where overlapping i'm not shure,
  if those apearing lines where previously deleted
  of just not displayed
- i could delete them when they apeared

frank

       

       
               
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de


_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

How to be more efficient inserting tracks?

Johnny Doe-2

--- Frank Mohr <[hidden email]> wrote:
>
> - i could delete them when they apeared
>

 Steve,

  can you explain the track clipping algorithm ?
My first thought was that it is a scale dependent
thing,
but it is not. Very often the long tracks which are
definitely inside the canvas "bounding box" are
invisible, but a simple move makes them visible again,
while others disappear.



       
               
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Tom Carden
On 10/26/05, Johnny Doe <[hidden email]> wrote:

--- Frank Mohr <[hidden email]> wrote:
>
> - i could delete them when they apeared
>

Steve,

Not Steve ;)

  can you explain the track clipping algorithm ?

Do you mean the yellow dots for GPX tracks?

My first thought was that it is a scale dependent
thing,
but it is not.

Well it is actually, kind of.

Very often the long tracks which are
definitely inside the canvas "bounding box" are
invisible, but a simple move makes them visible again,
while others disappear.

I think the current renderer only displays the most recent 50000, which will vary from bounding box to bounding box.

Steve's starting to document this stuff at http://www.openstreetmap.org/wiki/index.php/WMS

Best,

Tom.


_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Tom Carden

Steve's starting to document this stuff at <a href="http://www.openstreetmap.org/wiki/index.php/WMS" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://www.openstreetmap.org/wiki/index.php/WMS

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Johnny Doe-2
In reply to this post by Tom Carden

--- Tom Carden <[hidden email]> wrote:
>
> Do you mean the yellow dots for GPX tracks?
>

Yes.

>
> I think the current renderer only displays the most
> recent 50000, which will
> vary from bounding box to bounding box.
>

Hmm, now i'm really confused. What do you mean with
'recent' ? I can see all my uploaded tracks, some
of then just disappear depending on the canvas
bounding box (while being topologically partly
inside), and i don't see any logic behind it.
One can argue that you can't show all uploaded tracks
if the user requests the global map, but when the
tracks randomly disappear in a 1-10km rectangular box
then
it is really annoying.
 



               
__________________________________
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
Reply | Threaded
Open this post in threaded view
|

Re: How to be more efficient inserting tracks?

Tom Carden


On 10/26/05, Johnny Doe <[hidden email]> wrote:

--- Tom Carden <[hidden email]> wrote:
>
> Do you mean the yellow dots for GPX tracks?
>

Yes.

>
> I think the current renderer only displays the most
> recent 50000, which will
> vary from bounding box to bounding box.
>

Hmm, now i'm really confused. What do you mean with
'recent' ?
 
Either the newest tracks by uploaded date, or the newest tracks by GPS time-stamp.  Off the top of my head I'm not sure which Steve orders by.
 

I can see all my uploaded tracks, some
of then just disappear depending on the canvas
bounding box (while being topologically partly
inside),
 
Links?

and i don't see any logic behind it.
 
The logic *might* be that in one bounding box your tracks are the newest, but in another there are tracks which are newer than your tracks.  Maybe. 

One can argue that you can't show all uploaded tracks
if the user requests the global map,
 
I can argue that you can't show them all in a small area either.

but when the
tracks randomly disappear in a 1-10km rectangular box
then
it is really annoying.
 
Sure, it's frustrating, and there are plans to allow more manipulation of which tracks are shown.  But at the moment for performance reasons the tracks are cached in images.  We can go back to waiting 2 minutes per refresh for the applet to load if you'd like? 
 
Honestly, if it's "really annoying" (is it?) then perhaps you could give more details?  A link to a location which demonstrates the problem, or a screen grab which highlights the missing points.  Something like that.
 
Best,
 
Tom.
 
 

 

_______________________________________________
Openstreetmap mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap
12