REST

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

REST

stevec-4
If we threw away the xmlrpc and had things RESTful

http://en.wikipedia.org/wiki/Representational_State_Transfer

would that make people happy or sad?

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Tom Carden
SteveC wrote:
> If we threw away the xmlrpc and had things RESTful
>
> http://en.wikipedia.org/wiki/Representational_State_Transfer
>
> would that make people happy or sad?
>

Both.  Happy because there would be less fluff over the wire (XML-RPC is
uglee), and Sad because I don't know of a neat REST library which just bungs
things into generic types in Java, and parsing XML the way I do it makes the
baby Jesus cry.

I've only ever used the Flickr REST API (which now has a complicated but
interesting authentication API too).  http://www.flickr.com/services/api Any
others we should be aware of?  (Anyone used the google API or the Amazon
API?)

Initial thoughts...

Needed for the applet, yesterday:

http GETs...
getTracks which takes a lat-lon bounding box, and returns a list of
available gps point collections (date, user, count)
getPoints which takes a lat-lon bounding box, date, user, perpage and
pagenumber, and returns a list of gps points (in GPX format?)
getNodes which takes a lat-lon bounding box, and returns a list of nodes
getLines which takes a list of node ids, and returns a list of lines (and
optional metadata?)
getLinesWithNodes which takes a lat-lon bounding box and a token, and
returns a list of lines with nodes and locations and metadata.  Hopefully in
a way which could be parsed as a stream.

http POSTs (or PUTs or DELETEs?)...
addNode which takes a lat/lon and a token and gets back an id
addLine which takes two node ids and a token and gets back an id
setMeta which takes a line id and a key value pair
deleteNode which takes an id and a token and also removes any lines involved
deleteLine which takes an id and a token

Not sure how these should be named exactly, or what the returned XML data
would look like.  Perhaps where possible we could return gpx data?  e.g.
getNodes (waypoints), getTracks (trksegs), etc.

All off the top of my head...

Tom.


_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Tom Carden
I should add that I have heard the flickr api referred to as not really
being RESTful, since it doesn't really specify URIs for things and
GET/PUT/POST/DELETE them.  I'm aware of that, and I'm aware that the methods
I suggested below look more like RPC methods than REST.

Hopefully someone else can point us in the right direction?  Until we get to
representing the modification history of the maps, would GPX be a sufficient
format?

Tom.

----- Original Message -----
From: "Tom Carden" <[hidden email]>
To: "SteveC" <[hidden email]>; <[hidden email]>
Sent: Monday, July 18, 2005 2:17 PM
Subject: Re: [Openstreetmap-dev] REST


> SteveC wrote:
>> If we threw away the xmlrpc and had things RESTful
>>
>> http://en.wikipedia.org/wiki/Representational_State_Transfer
>>
>> would that make people happy or sad?
>>
>
> Both.  Happy because there would be less fluff over the wire (XML-RPC is
> uglee), and Sad because I don't know of a neat REST library which just
> bungs things into generic types in Java, and parsing XML the way I do it
> makes the baby Jesus cry.
>
> I've only ever used the Flickr REST API (which now has a complicated but
> interesting authentication API too).  http://www.flickr.com/services/api 
> Any others we should be aware of?  (Anyone used the google API or the
> Amazon API?)
>
> Initial thoughts...
>
> Needed for the applet, yesterday:
>
> http GETs...
> getTracks which takes a lat-lon bounding box, and returns a list of
> available gps point collections (date, user, count)
> getPoints which takes a lat-lon bounding box, date, user, perpage and
> pagenumber, and returns a list of gps points (in GPX format?)
> getNodes which takes a lat-lon bounding box, and returns a list of nodes
> getLines which takes a list of node ids, and returns a list of lines (and
> optional metadata?)
> getLinesWithNodes which takes a lat-lon bounding box and a token, and
> returns a list of lines with nodes and locations and metadata.  Hopefully
> in a way which could be parsed as a stream.
>
> http POSTs (or PUTs or DELETEs?)...
> addNode which takes a lat/lon and a token and gets back an id
> addLine which takes two node ids and a token and gets back an id
> setMeta which takes a line id and a key value pair
> deleteNode which takes an id and a token and also removes any lines
> involved
> deleteLine which takes an id and a token
>
> Not sure how these should be named exactly, or what the returned XML data
> would look like.  Perhaps where possible we could return gpx data?  e.g.
> getNodes (waypoints), getTracks (trksegs), etc.
>
> All off the top of my head...
>
> Tom.
>
>
> _______________________________________________
> Openstreetmap-dev mailing list
> [hidden email]
> http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev 


_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Jo Walsh
In reply to this post by Tom Carden
On Mon, Jul 18, 2005 at 02:17:59PM +0100, Tom Carden wrote:
> Both.  Happy because there would be less fluff over the wire (XML-RPC is
> uglee), and Sad because I don't know of a neat REST library which just
> bungs things into generic types in Java, and parsing XML the way I do it
> makes the baby Jesus cry.

Happy, because the inconsistencies in the current xmlrpc API have made
it frustrating for me to work with. In the past i've done a fair bit
with RESTful APIs which return RDF/XML. mudlondon's brain worked this
way, and a few years ago i wrote a dodgy position paper on a generic
approach to this:
http://www.w3.org/2001/sw/Europe/events/20031113-storage/positions/walsh.html
 
> I've only ever used the Flickr REST API (which now has a complicated but
> interesting authentication API too).  http://www.flickr.com/services/api 
> Any others we should be aware of?  (Anyone used the google API or the
> Amazon API?)

it might be worth raising the EVNT API, which i think uses xcal xml,
and also Earle's work on a GETting interface, with bits of POSTing
growing into place, for openguides.org
http://wiki.wirelesslondon.info/RestfulRepresentations
has some links to thinkings about this. geocoder.us also has a
terribly simple one.

i had a lot of "where's the toolkit, what's the interface" type
thoughts before i realised the zen of rest is it's not really there.
re REST libraries, that makes it hard to state anything; the amazon
approach is to provide a little developer pack which abstracts
representations away in modules. count me as +1 to make simple perl
and python widgets if this approach seemed sane.

also count me as +1 to learn ruby and actually contrib code (!) to a
new RESTful interface for osm, whether that's RDFish directly (if it's
not, i'll make an RDF-output gateway service for the GETs which builds
a local model incrementally, as i've done in the past for del.icio.us
and evnt....


-jo

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Tom Carden
In reply to this post by Tom Carden
SteveC wrote:
>
> URL http://www.openstreetmap.org/api/node/1234
>

That's lovely - more please.  But for 5000 nodes?

> GET URL - lat/lon, last edited time/user
> PUT URL - update the above
> DELETE - delete...
>
> how do we deal with resurecting a previously deleted node?
>

Don't delete it, just hide it?  i.e. That particular node URI would still
work, but that node wouldn't be returned with requests for *current* map
data.

How's it dealt with in the database?  For that matter, how does versioning
work in general, and what were your thoughts on exposing editing history on
openstreetmap.org?

Tom.



_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

stevec-4
* @ 18/07/05 03:17:16 PM [hidden email] wrote:
> SteveC wrote:
> >
> >URL http://www.openstreetmap.org/api/node/1234
> >
>
> That's lovely - more please.  But for 5000 nodes?

POST

<nodelist>
  <node>1</node>
  <node>1</node>
  <node>1</node>
</nodelist>

to

http://../api/nodes ?

Doesn't quite work, but...

> Don't delete it, just hide it?  i.e. That particular node URI would still
> work, but that node wouldn't be returned with requests for *current* map
> data.

So you delete, you get 200 OK. If you then try to GET you get 330
TEMPORARILY UNAVAILABLE (or whatever the code is)

> How's it dealt with in the database?  For that matter, how does versioning
> work in general

Like this

nodeid | latitude | timestamp


and then you just select the latest timestamp, basically, to get the
current version.

> and what were your thoughts on exposing editing history on
> openstreetmap.org?

Its easy, it should be done...

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Nick Whitelegg-2
In reply to this post by stevec-4
Am I right in undestanding that this differs from XML-RPC in that you send
HTTP requests as URLs, rather than wrapping them up in method calls, and in
that way it's "lower-level" but perhaps more efficient than XML-RPC?

e.g. you could use something like cURL, or JavaScript XMLHttpRequest
(AJAX), to construct an HTTP request, get some XML (or whatever) back and
then have your application process your XML?

And that rather than x method calls you have x URLs which take different
query-string (HTTP GET) parameters?

Thanks,
Nick


_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Matt Amos-2
In reply to this post by stevec-4
On Monday 18 July 2005 13:56, SteveC wrote:
> If we threw away the xmlrpc and had things RESTful
> http://en.wikipedia.org/wiki/Representational_State_Transfer
> would that make people happy or sad?

interesting... this seems like a nice way to do the bridge pattern.
iirc you can do this stuff completely transparently in ruby using
method_missing and friends.

the only problem i can see is that of transactions, e.g:
1) A GETs foo.xml
2) B GETs foo.xml
3) A and B modify foo.xml
4) A PUTs foo.xml
5) B PUTs foo.xml

its the same kinds of problems that RCS, CVS and countless others have
been dealing with for years: do you prevent (5), as RCS does, in
which case you have to deal with horrible locking problems?

do you allow error recovery at (5), as CVS/SVN do, but make the client
and server slightly more complex?

or do you just allow B to overwrite A? this might be the simplest, but
if those changes have follow-on effects in the rest of the data
structures (e.g: the deletion of a node or line affects other
entities it may be part of)

as an aside: can anyone decode xml-speak well enough to tell me what
the differences between REST and document/literal are?

cya,

matt

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev

attachment0 (197 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: REST

stevec-4
* @ 19/07/05 12:41:42 AM [hidden email] wrote:
> or do you just allow B to overwrite A? this might be the simplest, but
> if those changes have follow-on effects in the rest of the data
> structures (e.g: the deletion of a node or line affects other
> entities it may be part of)

We do this for 1.0.

Don't forget this is all wiki'd and recoverable.

We can add locking later.

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Nick Whitelegg-2
In reply to this post by stevec-4

One question: I know little about REST but would it not require clients of
OSM to write more code than if XML-RPC was used? i.e. the client would have
to parse the XML themselves rather than let an XML-RPC library do it for
them? I can see this could be more flexible but for things like login
tokens it seems more work for the client.

Nick


_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Re: REST

Raphael Jacquot-2
Nick Whitelegg wrote:
> One question: I know little about REST but would it not require clients of
> OSM to write more code than if XML-RPC was used? i.e. the client would have
> to parse the XML themselves rather than let an XML-RPC library do it for
> them? I can see this could be more flexible but for things like login
> tokens it seems more work for the client.
>
> Nick

clients should use libxml anyhow :d

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Re: REST

stevec-4
In reply to this post by Nick Whitelegg-2
* @ 19/07/05 11:13:58 AM [hidden email] wrote:
>
> One question: I know little about REST but would it not require clients of
> OSM to write more code than if XML-RPC was used? i.e. the client would have
> to parse the XML themselves rather than let an XML-RPC library do it for
> them? I can see this could be more flexible but for things like login
> tokens it seems more work for the client.

Yes, and no.

Parsing xml in ruby is terribly simple. Java is a little bit more
complicated, I have no idea what its like in C land.

I would envisage a wrapper to speak to a REST api in the same way as the
java applet currently has a wrapper of the XML-RPC, talking native java
types at one end, and XML-RPC the other.

Also, REST solves a couple of problems - gzip encoding and Unicode. It
also stops the limit on integers we have with XML-RPC where we'd have to
turn every integer (and thus every uid for a node, segment, etc) into a
string and back at each end. You'll still have to of course, but it
won't be as nasty. These are big, big wins.

OTOH, the REST api doesn't _have_ to return xml to you. A simple thing
would be to grep out all the xml tags if you want plain text.

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

stevec-4
In reply to this post by Nick Whitelegg-2
* @ 18/07/05 05:34:15 PM [hidden email] wrote:
> Am I right in undestanding that this differs from XML-RPC in that you send
> HTTP requests as URLs, rather than wrapping them up in method calls, and in
> that way it's "lower-level" but perhaps more efficient than XML-RPC?

yup

> e.g. you could use something like cURL, or JavaScript XMLHttpRequest
> (AJAX), to construct an HTTP request, get some XML (or whatever) back and
> then have your application process your XML?

yup, or whatever http library your $LANGUAGE has

> And that rather than x method calls you have x URLs which take different
> query-string (HTTP GET) parameters?

yes, put you can POST xml to it too.

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: Re: REST

Tom Carden
In reply to this post by Nick Whitelegg-2
Nick Whitelegg wrote:
> One question: I know little about REST but would it not require clients of
> OSM to write more code than if XML-RPC was used? i.e. the client would have
> to parse the XML themselves rather than let an XML-RPC library do it for
> them? I can see this could be more flexible but for things like login
> tokens it seems more work for the client.
>

Possibly slightly more in static languages, yes, but things like xpath
really help.  There's also the benefit of flexibility.  With XML-RPC, I
think you really have no choice but to parse it all behind the scenes.
With REST, you can pick out just the bits you want.

Matt's point about being able to build interfaces on the fly with
languages like Ruby is why it makes more sense for that kind of language.

There's a full implementation of the flickr API in Python (FlickrClient)
that's only about 50 lines of code (including a test method), and never
needs updating.  That's what we're buying into here.
http://www.flickr.com/services/api/

T.



_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

stevec-4
In reply to this post by Tom Carden
* @ 18/07/05 03:17:16 PM [hidden email] wrote:

> SteveC wrote:
> >
> >URL http://www.openstreetmap.org/api/node/1234
> >
>
> That's lovely - more please.  But for 5000 nodes?
>
> >GET URL - lat/lon, last edited time/user
> >PUT URL - update the above
> >DELETE - delete...
> >
> >how do we deal with resurecting a previously deleted node?
> >
>
> Don't delete it, just hide it?  i.e. That particular node URI would still
> work, but that node wouldn't be returned with requests for *current* map
> data.

Ok. If the node hasn't been created, you get 404, if its just been
deleted then you get 403 Forbidden (updated wiki with this)

> How's it dealt with in the database?  For that matter, how does versioning
> work in general, and what were your thoughts on exposing editing history on
> openstreetmap.org?

I thought I'd replied to this but can't find it...

the dbase will have

id | longitude | timestamp
1   2            1
1   3             1234
1   3            2345342
2   1            2342

and the current revision of the node is just the one with the biggest
timestamp. and of course there is a 'deleted' flag

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

stevec-4
In reply to this post by Tom Carden
* @ 18/07/05 02:21:48 PM [hidden email] wrote:
> Hopefully someone else can point us in the right direction?  Until we get
> to representing the modification history of the maps, would GPX be a
> sufficient format?

Its extendable, so yes. In theory we could wrap everything in gpx which
would be very nice.

However I'm not sure that I'm up for exahstively doing the DTD
validation type stuff. Its very extensive in gpx.

So my answer is, I'd be willing to use gpx for everything, and update
the REST examples as I do them on the wiki to use it but making a DTD
and so on might melt my brain and will therefore wait for someone else
or me to have time... It would be nice to have it all 'proper' xml for
1.0 though, so I don't know.

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Tom Carden
SteveC wrote:
>
> However I'm not sure that I'm up for exahstively doing the DTD
> validation type stuff. Its very extensive in gpx.
>

Do people really do DTDs any more?  I mean, as long as your XML is
machine-written it should be valid, and that's half the battle.
Validation against a schema seems overkill for most things.  Aren't
there other, nicer, schema formats than DTD though, if we must?

Tom.


_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: REST

Nick Whitelegg-2
In reply to this post by stevec-4
I would concur with using GPX as the interchange format, as a typical
client (well certainly Freemap and osm-editor anyway :-) ) can already
parse at least some of the GPX spec.

It would need extensions for the key/value system, so that the "type" of a
<trkseg> is known.

What I do in osm-editor and Freemap's "extended" gpx is to use the
<extensions> tag:

<trkseg>
<trkpt....>

track points...

</trkpt>
<extensions>
<type>road</type>
<name>Church Road</name>
</extensions>
</trkseg>

but this could be revised in a way appropriate to the key value system e.g.
something like the following to indicate that a segment has a name of
Church Road and is accessible to car, horse, foot, bike:

<extensions>
<permissions car="1" horse="1" foot="1" bike="1"/>
<name>Chuch Road</name>
</extensions>

Nick


_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

request to add HUGE file to the database

Raphael Jacquot-2
In reply to this post by stevec-4
I have a 720152 fix dataset available at

http://www.navsys.org/mapeditor/tracks/sxpert.gpx.bz2

(it's 6.1M that expands to 97M gpx :D)

could you load it in the db ?

Amaury

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
Reply | Threaded
Open this post in threaded view
|

Re: request to add HUGE file to the database

stevec-4
No! You load it! :-)

It should handle it fine. The eCourier data is of that order of
magnitude and was ok. It takes a little while, but you get progress in
your browser.

* @ 19/07/05 06:54:14 PM [hidden email] wrote:

> I have a 720152 fix dataset available at
>
> http://www.navsys.org/mapeditor/tracks/sxpert.gpx.bz2
>
> (it's 6.1M that expands to 97M gpx :D)
>
> could you load it in the db ?
>
> Amaury
>

have fun,

SteveC [hidden email] http://www.fractalus.com/steve/

_______________________________________________
Openstreetmap-dev mailing list
[hidden email]
http://bat.vr.ucl.ac.uk/cgi-bin/mailman/listinfo/openstreetmap-dev
12