generic geotools datastore support

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

generic geotools datastore support

Jody Garnett-2
It appears we have this working; and in the interest of feedback I am putting a SNAPSHOT release up.



This is simply a "nightly release" with no special meaning in order to solicit feedback and testing. The support of geotools datastores does not need to be included in uDig 1.2.0 (and can wait for a point release) - this is just an experiment to see if people enjoy the functionality.

This opens the door to several formats - as shown in the attached screen snap....


The wizard pages are pretty generic (well exactly generic). I have left all the "advanced" parameters in the mix; and it should give you good feedback as you type in values etc. Thus far I have tested with shapefile, property, versioned postgis and postgisng.

It should also let people try out DB2 and Oracle without getting stuck on incomplete wizard QA. If you get a chance skip the "JNDI" entries; they are designed for use in a Java EE application like GeoServer where Java Naming and Directory Interface can be used to look up a shared database.

If you are wanting to work on Oracle you will still need to use the preference page to tell udig about the jdbc drivers (since we cannot redistribute them).

Areas where there are likely to be difficulty?
- generating of an "id" for each service / resource (so it can be listed in the catalog) is the hardest problem; and where we need help testing! Let us know what formats work or do not work.
- generation of a title is similar to that of an id; and we are interested in any formats that cause trouble
- it is slightly easier to "Import" data into the catalog; and then add to the Map. If you are having trouble try this technique first (since the wizard only has to connect to the service there is less chance for error; and chances are you can expand the service in the catalog and let us know if any resources show up with out a title or icon or otherwise produce null pointer exceptions).

Any and all feedback welcome; I would love to see the H2 format working for example :-)

Finally if you know of any other geotools formats you would like us to add send an email and we can list them in refresh.xml.

Jody



_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

aaime
Jody Garnett ha scritto:
> Finally if you know of any other geotools formats you would like us to
> add send an email and we can list them in refresh.xml.

The newly contributed (today) DXF store for example? I believe it's
just on trunk though

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

aaime
In reply to this post by Jody Garnett-2
Jody Garnett ha scritto:
> It should also let people try out DB2 and Oracle without getting stuck
> on incomplete wizard QA. If you get a chance skip the "JNDI" entries;
> they are designed for use in a Java EE application like GeoServer where
> Java Naming and Directory Interface can be used to look up a shared
> database.

I guess it would make sense to skip all the datastores that do have JNDI
in the name or that are known to have their own wizard.
Hum... maybe for the latter have an option to re-enable them in the list
though, as the raw connection panel might have interesting options that
are not exposed by the wizard

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

Jody Garnett-2
Yeah I can skip JNDI ones; I would like to keep the others (simply because they allow for more settings).

I also note that there is a "property page" class I can use to display a single wizard page; chances are I can use this to allow configuration parameters to be set when right clicking on the catalog services in the future.

I am limited to working on 2.6.x until such time as 1.2.0 goes out (and the PMC agree to move to trunk). Just to keep the number of moving parts to a minimum.

Jody

On 21/04/2010, at 8:26 PM, Andrea Aime wrote:

> Jody Garnett ha scritto:
>> It should also let people try out DB2 and Oracle without getting stuck on incomplete wizard QA. If you get a chance skip the "JNDI" entries; they are designed for use in a Java EE application like GeoServer where Java Naming and Directory Interface can be used to look up a shared database.
>
> I guess it would make sense to skip all the datastores that do have JNDI
> in the name or that are known to have their own wizard.
> Hum... maybe for the latter have an option to re-enable them in the list
> though, as the raw connection panel might have interesting options that
> are not exposed by the wizard
>
> Cheers
> Andrea
>
> --
> Andrea Aime
> OpenGeo - http://opengeo.org
> Expert service straight from the developers.
> _______________________________________________
> User-friendly Desktop Internet GIS (uDig)
> http://udig.refractions.net
> http://lists.refractions.net/mailman/listinfo/udig-devel

_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

Ugo Taddei
In reply to this post by Jody Garnett-2
Hi Jody,

nice feature.

>
> If you are wanting to work on Oracle you will still need to use the
> preference page to tell udig about the jdbc drivers (since we cannot
> redistribute them).

Tried on oracle. It said "Exception occurred while trying to copy
driver" when I pointed the prefs page to an oracle driver jar.

It simply refused to work, though I can access the db from my machine
and so can geoserver. No Logs, except the below (which i think i got
before I tried to apply my driver jar). Any ideas?

Cheers,

Ugo

!MESSAGE Problems occurred when invoking code from plug-in:
"org.eclipse.jface".
!STACK 0
java.lang.RuntimeException: Unable to obtain connection: Cannot create
PoolableConnectionFactory (invalid arguments in call)
        at
org.geotools.jdbc.JDBCDataStore.createConnection(JDBCDataStore.java:1428)
        at org.geotools.jdbc.JDBCDataStore.createTypeNames(JDBCDataStore.java:746)
        at
org.geotools.data.store.ContentDataStore.getTypeNames(ContentDataStore.java:297)
        at
net.refractions.udig.catalog.internal.oracle.OracleServiceImpl$IServiceOracleInfo.<init>(OracleServiceImpl.java:228)
        at
net.refractions.udig.catalog.internal.oracle.OracleServiceImpl.createInfo(OracleServiceImpl.java:158)
        at net.refractions.udig.catalog.IService.getInfo(IService.java:331)
        at
net.refractions.udig.catalog.internal.oracle.OracleServiceImpl.getInfo(OracleServiceImpl.java:146)
        at
net.refractions.udig.catalog.internal.oracle.OracleServiceImpl.getInfo(OracleServiceImpl.java:1)
        at
net.refractions.udig.catalog.ui.ResolveLabelProviderSimple.getText(ResolveLabelProviderSimple.java:119)
        at
org.eclipse.jface.viewers.DecoratingLabelProvider.getText(DecoratingLabelProvider.java:129)
        at
org.eclipse.jface.viewers.DecoratingLabelProvider.updateLabel(DecoratingLabelProvider.java:351)
        at
org.eclipse.jface.viewers.WrappedViewerLabelProvider.update(WrappedViewerLabelProvider.java:183)
        at org.eclipse.jface.viewers.ViewerColumn.refresh(ViewerColumn.java:145)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.doUpdateItem(AbstractTreeViewer.java:932)
        at
org.eclipse.jface.viewers.AbstractTreeViewer$UpdateItemSafeRunnable.run(AbstractTreeViewer.java:102)
        at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
        at org.eclipse.core.runtime.Platform.run(Platform.java:888)
        at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:48)
        at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.doUpdateItem(AbstractTreeViewer.java:1012)
        at
org.eclipse.jface.viewers.StructuredViewer$UpdateItemSafeRunnable.run(StructuredViewer.java:481)
        at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
        at org.eclipse.core.runtime.Platform.run(Platform.java:888)
        at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:48)
        at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
        at
org.eclipse.jface.viewers.StructuredViewer.updateItem(StructuredViewer.java:2111)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.updateChildren(AbstractTreeViewer.java:2681)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.internalRefreshStruct(AbstractTreeViewer.java:1863)
        at
org.eclipse.jface.viewers.TreeViewer.internalRefreshStruct(TreeViewer.java:716)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1838)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1794)
        at
org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1780)
        at
org.eclipse.jface.viewers.StructuredViewer$7.run(StructuredViewer.java:1457)
        at
org.eclipse.jface.viewers.StructuredViewer.preservingSelection(StructuredViewer.java:1392)
        at
org.eclipse.jface.viewers.TreeViewer.preservingSelection(TreeViewer.java:402)
        at
org.eclipse.jface.viewers.StructuredViewer.preservingSelection(StructuredViewer.java:1353)
        at
org.eclipse.jface.viewers.StructuredViewer.refresh(StructuredViewer.java:1455)
        at org.eclipse.jface.viewers.ColumnViewer.refresh(ColumnViewer.java:537)
        at
org.eclipse.jface.viewers.StructuredViewer.refresh(StructuredViewer.java:1414)
        at
org.eclipse.jface.viewers.StructuredViewer.setSorter(StructuredViewer.java:1730)
        at
net.refractions.udig.catalog.ui.CatalogTreeViewer.<init>(CatalogTreeViewer.java:102)
        at
net.refractions.udig.catalog.ui.CatalogTreeViewer.<init>(CatalogTreeViewer.java:61)
        at
net.refractions.udig.catalog.internal.ui.CatalogView.createPartControl(CatalogView.java:121)
        at
org.eclipse.ui.internal.ViewReference.createPartHelper(ViewReference.java:367)
        at org.eclipse.ui.internal.ViewReference.createPart(ViewReference.java:226)
        at
org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
        at
org.eclipse.ui.internal.WorkbenchPage$ActivationList.setActive(WorkbenchPage.java:4212)
        at
org.eclipse.ui.internal.WorkbenchPage$18.runWithException(WorkbenchPage.java:3271)
        at
org.eclipse.ui.internal.StartupThreading$StartupRunnable.run(StartupThreading.java:31)
        at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
        at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
        at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3468)
        at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3115)
        at
org.eclipse.ui.application.WorkbenchAdvisor.openWindows(WorkbenchAdvisor.java:803)
        at
org.eclipse.ui.internal.Workbench$28.runWithException(Workbench.java:1384)
        at
org.eclipse.ui.internal.StartupThreading$StartupRunnable.run(StartupThreading.java:31)
        at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
        at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
        at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3468)
        at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3115)
        at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2316)
        at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2221)
        at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
        at
org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
        at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:493)
        at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
        at
net.refractions.udig.internal.ui.UDIGApplication.start(UDIGApplication.java:136)
        at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:194)
        at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
        at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
        at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
        at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
        at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
        at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (invalid arguments in call)
        at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1225)
        at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
        at
org.geotools.data.jdbc.datasource.AbstractManageableDataSource.getConnection(AbstractManageableDataSource.java:44)
        at
org.geotools.jdbc.JDBCDataStore.createConnection(JDBCDataStore.java:1420)
        ... 78 more
Caused by: java.sql.SQLException: invalid arguments in call
        at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
        at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
        at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:208)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:236)
        at
oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:414)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165)
        at
oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801)
        at
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
        at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
        at
org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1247)
        at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1221)
        ... 81 more
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

Ugo Taddei
Hello,

for the archives:

> Tried on oracle. It said "Exception occurred while trying to copy
> driver" when I pointed the prefs page to an oracle driver jar.
>

this was my fault. The driver wasn't readable or my user couldn't write
to the directory. Anyway, I got a step further: udig is reading the
tables, and the GUI blocked, but that's because one of the tables is
large. (geotools is probably calculating the bbox...)

Cheers,

Ugo
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

Jody Garnett-2
Yeah progress! Well progress bar; well not say yeah :-P
I am not sure what we should do about progress calculating bounding boxes :-(

I have thought of popping up a dialog and asking a user for the bounds if it "takes too long"; but if you have many many tables that will be a trouble. I could try putting off the calculation until you first use the layer ...

I think the best would be a dialog that shows the world; the bounds visually as they are being calculated and the option to stop the calculation at any point (adding 10% to the edges to buffer it a bit).

Cheers,
Jody

On 23/04/2010, at 7:26 PM, Ugo Taddei wrote:

> Hello,
>
> for the archives:
>
>> Tried on oracle. It said "Exception occurred while trying to copy driver" when I pointed the prefs page to an oracle driver jar.
>
> this was my fault. The driver wasn't readable or my user couldn't write to the directory. Anyway, I got a step further: udig is reading the tables, and the GUI blocked, but that's because one of the tables is large. (geotools is probably calculating the bbox...)
>
> Cheers,
>
> Ugo
> _______________________________________________
> User-friendly Desktop Internet GIS (uDig)
> http://udig.refractions.net
> http://lists.refractions.net/mailman/listinfo/udig-devel

_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

Ugo Taddei


Jody Garnett wrote:
> Yeah progress! Well progress bar; well not say yeah :-P

Progress yes, but progress bar, no. Grey dialogs.

> I am not sure what we should do about progress calculating bounding boxes :-(
>
> I have thought of popping up a dialog and asking a user for the bounds if it "takes too long"; but if you have many many tables that will be a trouble. I could try putting off the calculation until you first use the layer ...

I wonder why udig is trying to get the bbox at this point. It should
first list the tables, and only get the bbox after I select the tables.
But this is probably something coming from geotools.

FYI: the dialog is still blocked. I'm waiting to see if it'll return
before I start my weekend :-)

I also get loads of

WARNING: No primary key or unique index found for MY_TABLE.
Apr 23, 2010 11:13:59 AM org.geotools.jdbc.JDBCDataStore getPrimaryKey

But I think/hope this is only a warning.

Cheers,

Ugo

>
> I think the best would be a dialog that shows the world; the bounds visually as they are being calculated and the option to stop the calculation at any point (adding 10% to the edges to buffer it a bit).
>
> Cheers,
> Jody
>
> On 23/04/2010, at 7:26 PM, Ugo Taddei wrote:
>
>> Hello,
>>
>> for the archives:
>>
>>> Tried on oracle. It said "Exception occurred while trying to copy driver" when I pointed the prefs page to an oracle driver jar.
>> this was my fault. The driver wasn't readable or my user couldn't write to the directory. Anyway, I got a step further: udig is reading the tables, and the GUI blocked, but that's because one of the tables is large. (geotools is probably calculating the bbox...)
>>
>> Cheers,
>>
>> Ugo
>> _______________________________________________
>> User-friendly Desktop Internet GIS (uDig)
>> http://udig.refractions.net
>> http://lists.refractions.net/mailman/listinfo/udig-devel
>
> _______________________________________________
> User-friendly Desktop Internet GIS (uDig)
> http://udig.refractions.net
> http://lists.refractions.net/mailman/listinfo/udig-devel
>
>
>
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
Reply | Threaded
Open this post in threaded view
|

Re: generic geotools datastore support

Jody Garnett-2
On 23/04/2010, at 10:01 PM, Ugo Taddei wrote:

> Jody Garnett wrote:
>> Yeah progress! Well progress bar; well not say yeah :-P
>
> Progress yes, but progress bar, no. Grey dialogs.
>
>> I am not sure what we should do about progress calculating bounding boxes :-(
>> I have thought of popping up a dialog and asking a user for the bounds if it "takes too long"; but if you have many many tables that will be a trouble. I could try putting off the calculation until you first use the layer ...
>
> I wonder why udig is trying to get the bbox at this point. It should first list the tables, and only get the bbox after I select the tables. But this is probably something coming from geotools.

Bit of both.

uDig want to know the bounds now - so it can cache for later searching. When you search using the search view you have the option to search within the bounds of the current screen.

> FYI: the dialog is still blocked. I'm waiting to see if it'll return before I start my weekend :-)

And this is the part that is in geotools; we are dependent on the implementation provided to return us a good bounds - and many implementations default to scanning the entire table :-(

> I also get loads of
>
> WARNING: No primary key or unique index found for MY_TABLE.
> Apr 23, 2010 11:13:59 AM org.geotools.jdbc.JDBCDataStore getPrimaryKey
>
> But I think/hope this is only a warning.

I think so - it tries to figure out a good "FeatureID" using any primary keys.

Jody
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel