Caching mechanisms on GPX layers

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Caching mechanisms on GPX layers

Niklas B
(maybe this time..?)

Hi,

I'm currently implementing an option to cut overlapping GPX tracks when
merging them into one layer (followup of #16681
<https://josm.openstreetmap.de/ticket/16681> "*allow multiple tracks to be
processed and prioritized (from different sources, e.g. GPS + phone +
Google Timeline)*", but when merging layers because this is not
specifically related to geotagging / geoimage layers).

Everything works fine, but the resulting GPX layer is not displayed
correctly: Some lines are displayed between tracks that are not connected
anymore (because another track was on top of it, so the underlying track
needed to be split) and also some weird lines connecting points that were
never connected appear - haven't figured them out yet.

Before I debug that any further: What caching mechanisms are there in
place, what do I have to reset? I'm already calling invalidate() (in
GpxData, which calls the appropriate listeners) and tried removing the
cached "dir" values from all of the waypoints, but it didn't help.
The layer will however be displayed perfectly fine if it's converted to a
data layer (and back) or saved as GPX and reloaded.

Any ideas on what might be the reason for that?

Thanks,
Niklas (Bjoeni)
Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Dirk Stöcker
On Sat, 15 Sep 2018, Niklas B wrote:

> (maybe this time..?)
>
> Hi,
>
> I'm currently implementing an option to cut overlapping GPX tracks when
> merging them into one layer (followup of #16681
> <https://josm.openstreetmap.de/ticket/16681> "*allow multiple tracks to be
> processed and prioritized (from different sources, e.g. GPS + phone +
> Google Timeline)*", but when merging layers because this is not
> specifically related to geotagging / geoimage layers).
>
> Everything works fine, but the resulting GPX layer is not displayed
> correctly: Some lines are displayed between tracks that are not connected
> anymore (because another track was on top of it, so the underlying track
> needed to be split) and also some weird lines connecting points that were
> never connected appear - haven't figured them out yet.
>
> Before I debug that any further: What caching mechanisms are there in
> place, what do I have to reset? I'm already calling invalidate() (in
> GpxData, which calls the appropriate listeners) and tried removing the
> cached "dir" values from all of the waypoints, but it didn't help.
> The layer will however be displayed perfectly fine if it's converted to a
> data layer (and back) or saved as GPX and reloaded.
>
> Any ideas on what might be the reason for that?

Maybe your troubles come from the fact that JOSM has different settings
for local and server based GPS data (i.e. distances for continuous track
handling)?

Ciao
--
http://www.dstoecker.eu/ (PGP key available)

Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Niklas B
Am Sa., 15. Sep. 2018 um 22:41 Uhr schrieb Dirk Stöcker <
[hidden email]>:

> Maybe your troubles come from the fact that JOSM has different settings
> for local and server based GPS data (i.e. distances for continuous track
> handling)?


No, unfortunately not. All tracks were loaded locally and the connected
points don't really make any sense at all, on productive data I had
connected trackpoints that were multiple months apart and on the other side
of the globe... (just for clarification btw: My implementation cuts timewise
 overlapping tracks)

Here's an example (it looks that weird because I'm planning on using it for
unit tests and it's supposed to cover pretty much every specific case, e.g.
entire "high level" track segments being between two waypoints of a "low
level" track etc.)

This is how it looks like directly after merging:


And this is how it's supposed to look like / how it looks after converting
it back and forth:


The GpxData is correct, at least when being exported, so I'm pretty sure
it's some kind of caching / rendering issue...

But if nobody has an idea I'll just finish the patch first, maybe someone
finds the bug after I published it. I just thought that there might be some
kind of caching somewhere that I wasn't aware of.

Cheers
Niklas
Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Niklas B
... and this is why I hate mailing lists.
How it looks like after merging:
http://fs5.directupload.net/images/180915/n57lqu5o.png
And after converting back and forth:
http://fs1.directupload.net/images/180915/bjg4nn5a.png

Sorry for the mess
Niklas

Am So., 16. Sep. 2018 um 01:20 Uhr schrieb Niklas B <[hidden email]
>:

> Am Sa., 15. Sep. 2018 um 22:41 Uhr schrieb Dirk Stöcker <
> [hidden email]>:
>
>> Maybe your troubles come from the fact that JOSM has different settings
>> for local and server based GPS data (i.e. distances for continuous track
>> handling)?
>
>
> No, unfortunately not. All tracks were loaded locally and the connected
> points don't really make any sense at all, on productive data I had
> connected trackpoints that were multiple months apart and on the other side
> of the globe... (just for clarification btw: My implementation cuts
> timewise overlapping tracks)
>
> Here's an example (it looks that weird because I'm planning on using it
> for unit tests and it's supposed to cover pretty much every specific case,
> e.g. entire "high level" track segments being between two waypoints of a
> "low level" track etc.)
>
> This is how it looks like directly after merging:
>
>
> And this is how it's supposed to look like / how it looks after converting
> it back and forth:
>
>
> The GpxData is correct, at least when being exported, so I'm pretty sure
> it's some kind of caching / rendering issue...
>
> But if nobody has an idea I'll just finish the patch first, maybe someone
> finds the bug after I published it. I just thought that there might be some
> kind of caching somewhere that I wasn't aware of.
>
> Cheers
> Niklas
>
Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Vincent Privat
Did you check if GpxDrawHelper.gpxDataChanged is called? This is where the
GPX drawing cache is reset.

Le sam. 15 sept. 2018 à 15:25, Niklas B <[hidden email]> a écrit :

> ... and this is why I hate mailing lists.
> How it looks like after merging:
> http://fs5.directupload.net/images/180915/n57lqu5o.png
> And after converting back and forth:
> http://fs1.directupload.net/images/180915/bjg4nn5a.png
>
> Sorry for the mess
> Niklas
>
> Am So., 16. Sep. 2018 um 01:20 Uhr schrieb Niklas B <
> [hidden email]
> >:
>
> > Am Sa., 15. Sep. 2018 um 22:41 Uhr schrieb Dirk Stöcker <
> > [hidden email]>:
> >
> >> Maybe your troubles come from the fact that JOSM has different settings
> >> for local and server based GPS data (i.e. distances for continuous track
> >> handling)?
> >
> >
> > No, unfortunately not. All tracks were loaded locally and the connected
> > points don't really make any sense at all, on productive data I had
> > connected trackpoints that were multiple months apart and on the other
> side
> > of the globe... (just for clarification btw: My implementation cuts
> > timewise overlapping tracks)
> >
> > Here's an example (it looks that weird because I'm planning on using it
> > for unit tests and it's supposed to cover pretty much every specific
> case,
> > e.g. entire "high level" track segments being between two waypoints of a
> > "low level" track etc.)
> >
> > This is how it looks like directly after merging:
> >
> >
> > And this is how it's supposed to look like / how it looks after
> converting
> > it back and forth:
> >
> >
> > The GpxData is correct, at least when being exported, so I'm pretty sure
> > it's some kind of caching / rendering issue...
> >
> > But if nobody has an idea I'll just finish the patch first, maybe someone
> > finds the bug after I published it. I just thought that there might be
> some
> > kind of caching somewhere that I wasn't aware of.
> >
> > Cheers
> > Niklas
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Niklas B
Just checked that again, yes it is being called (by the event listener, as
expected). And I assume it wouldn't update the layer at all if that wasn't
the case.

Thread [main-worker-0] (Suspended (breakpoint at line 1485 in

> GpxDrawHelper))
> owns: GpxData  (id=703)
> GpxDrawHelper.gpxDataChanged(GpxData$GpxDataChangeEvent) line: 1485
> GpxData.lambda$38(GpxData$GpxDataChangeEvent,
> GpxData$GpxDataChangeListener) line: 959
> 429616353.fire(Object) line: not available
> ListenerList<T>.fireEvent(EventFirerer<T>) line: 150
> GpxData.fireInvalidate() line: 959
> GpxData.mergeFrom(GpxData, boolean) line: 234
> GpxLayer.mergeFrom(GpxLayer, boolean) line: 318
> MergeLayerAction.lambda$0(List, Object) line: 93
> 589689845.run() line: not available
> Executors$RunnableAdapter<T>.call() line: 514
> FutureTask<V>.run() line: 264
> ProgressMonitorExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker)
> line: 1135
> ThreadPoolExecutor$Worker.run() line: 635
> Thread.run() line: 844



Am So., 16. Sep. 2018 um 01:33 Uhr schrieb Vincent Privat <
[hidden email]>:

> Did you check if GpxDrawHelper.gpxDataChanged is called? This is where the
> GPX drawing cache is reset.
>
> Le sam. 15 sept. 2018 à 15:25, Niklas B <[hidden email]> a écrit :
>
>> ... and this is why I hate mailing lists.
>> How it looks like after merging:
>> http://fs5.directupload.net/images/180915/n57lqu5o.png
>> And after converting back and forth:
>> http://fs1.directupload.net/images/180915/bjg4nn5a.png
>>
>> Sorry for the mess
>> Niklas
>>
>> Am So., 16. Sep. 2018 um 01:20 Uhr schrieb Niklas B <
>> [hidden email]
>> >:
>>
>> > Am Sa., 15. Sep. 2018 um 22:41 Uhr schrieb Dirk Stöcker <
>> > [hidden email]>:
>> >
>> >> Maybe your troubles come from the fact that JOSM has different settings
>> >> for local and server based GPS data (i.e. distances for continuous
>> track
>> >> handling)?
>> >
>> >
>> > No, unfortunately not. All tracks were loaded locally and the connected
>> > points don't really make any sense at all, on productive data I had
>> > connected trackpoints that were multiple months apart and on the other
>> side
>> > of the globe... (just for clarification btw: My implementation cuts
>> > timewise overlapping tracks)
>> >
>> > Here's an example (it looks that weird because I'm planning on using it
>> > for unit tests and it's supposed to cover pretty much every specific
>> case,
>> > e.g. entire "high level" track segments being between two waypoints of a
>> > "low level" track etc.)
>> >
>> > This is how it looks like directly after merging:
>> >
>> >
>> > And this is how it's supposed to look like / how it looks after
>> converting
>> > it back and forth:
>> >
>> >
>> > The GpxData is correct, at least when being exported, so I'm pretty sure
>> > it's some kind of caching / rendering issue...
>> >
>> > But if nobody has an idea I'll just finish the patch first, maybe
>> someone
>> > finds the bug after I published it. I just thought that there might be
>> some
>> > kind of caching somewhere that I wasn't aware of.
>> >
>> > Cheers
>> > Niklas
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Michael Zangl-3
In reply to this post by Niklas B
Hi,

I just had a short look at the patch that is attached to the ticket,
assuming it contains the code you used to split things.

I did not have a deep look at it, but from what I can tell, you copy
over the waypoints to a new layer.

Mind that this should not be done. New WayPoints always need to be
created for the new layer. So you need to call the WayPoint copy
constructor for every waypoint you add to the new layer.

The connectivity is cached using those waypoint objects.

Michael

On 15.09.2018 03:48, Niklas B wrote:

> (maybe this time..?)
>
> Hi,
>
> I'm currently implementing an option to cut overlapping GPX tracks when
> merging them into one layer (followup of #16681
> <https://josm.openstreetmap.de/ticket/16681> "*allow multiple tracks to be
> processed and prioritized (from different sources, e.g. GPS + phone +
> Google Timeline)*", but when merging layers because this is not
> specifically related to geotagging / geoimage layers).
>
> Everything works fine, but the resulting GPX layer is not displayed
> correctly: Some lines are displayed between tracks that are not connected
> anymore (because another track was on top of it, so the underlying track
> needed to be split) and also some weird lines connecting points that were
> never connected appear - haven't figured them out yet.
>
> Before I debug that any further: What caching mechanisms are there in
> place, what do I have to reset? I'm already calling invalidate() (in
> GpxData, which calls the appropriate listeners) and tried removing the
> cached "dir" values from all of the waypoints, but it didn't help.
> The layer will however be displayed perfectly fine if it's converted to a
> data layer (and back) or saved as GPX and reloaded.
>
> Any ideas on what might be the reason for that?
>
> Thanks,
> Niklas (Bjoeni)
>

Reply | Threaded
Open this post in threaded view
|

Re: Caching mechanisms on GPX layers

Niklas B
Hi,

no the patch attached to the ticket is not related to this issue. There the
waypoints are just put in a list and sorted by time, it is intentional that
no new instance is created.

Here however I thought that the (maybe expensive) copying of the instances
was not necessary if I removed the cached values (dir), because the old
layer was going to be removed in the process anyway. But now with new
instances it works, so thanks for mentioning this :)

For some reason now some lines between waypoints in the old layer (that are
not even touched when adding a new one) disappear... but I haven't had a
closer look at that yet.

Cheers
Niklas


Michael Zangl <[hidden email]> schrieb am So., 16. Sep.
2018, 8:22 AM:

> Hi,
>
> I just had a short look at the patch that is attached to the ticket,
> assuming it contains the code you used to split things.
>
> I did not have a deep look at it, but from what I can tell, you copy
> over the waypoints to a new layer.
>
> Mind that this should not be done. New WayPoints always need to be
> created for the new layer. So you need to call the WayPoint copy
> constructor for every waypoint you add to the new layer.
>
> The connectivity is cached using those waypoint objects.
>
> Michael
>
> On 15.09.2018 03:48, Niklas B wrote:
> > (maybe this time..?)
> >
> > Hi,
> >
> > I'm currently implementing an option to cut overlapping GPX tracks when
> > merging them into one layer (followup of #16681
> > <https://josm.openstreetmap.de/ticket/16681> "*allow multiple tracks to
> be
> > processed and prioritized (from different sources, e.g. GPS + phone +
> > Google Timeline)*", but when merging layers because this is not
> > specifically related to geotagging / geoimage layers).
> >
> > Everything works fine, but the resulting GPX layer is not displayed
> > correctly: Some lines are displayed between tracks that are not connected
> > anymore (because another track was on top of it, so the underlying track
> > needed to be split) and also some weird lines connecting points that were
> > never connected appear - haven't figured them out yet.
> >
> > Before I debug that any further: What caching mechanisms are there in
> > place, what do I have to reset? I'm already calling invalidate() (in
> > GpxData, which calls the appropriate listeners) and tried removing the
> > cached "dir" values from all of the waypoints, but it didn't help.
> > The layer will however be displayed perfectly fine if it's converted to a
> > data layer (and back) or saved as GPX and reloaded.
> >
> > Any ideas on what might be the reason for that?
> >
> > Thanks,
> > Niklas (Bjoeni)
> >
>
>