REST Navigator

Build Status Coverage Status Pypi Status Documentation Status

REST Navigator is a python library for interacting with hypermedia apis (REST level 3). Right now, it only supports HAL+JSON but it should be general enough to extend to other formats eventually. Its first goal is to make interacting with HAL hypermedia apis as painless as possible, while discouraging REST anti-patterns.

To install it, simply use pip:

$ pip install restnavigator

How to use it

To begin interacting with a HAL api, you’ve got to create a HALNavigator that points to the api root. Ideally, in a hypermedia API, the root URL is the only URL that needs to be hardcoded in your application. All other URLs are obtained from the api responses themselves (think of your api client as ‘clicking on links’, rather than having the urls hardcoded).

As an example, we’ll connect to the haltalk api.

>>> from restnavigator import Navigator
>>> N = Navigator.hal('http://haltalk.herokuapp.com/', default_curie="ht")
>>> N
HALNavigator(Haltalk)

GET requests

In addition, the root has some state associated with it which you can get in two different ways:

>>> N() # cached state of resource (obtained when we looked at N.links)
{u'hint_1': u'You need an account to post stuff..',
 u'hint_2': u'Create one by POSTing via the ht:signup link..',
 u'hint_3': u'Click the orange buttons on the right to make POST requests..',
 u'hint_4': u'Click the green button to follow a link with a GET request..',
 u'hint_5': u'Click the book icon to read docs for the link relation.',
 u'welcome': u'Welcome to a haltalk server.'}
>>> N.fetch() # will refetch the resource from the server
{u'hint_1': u'You need an account to post stuff..',
 u'hint_2': u'Create one by POSTing via the ht:signup link..',
 u'hint_3': u'Click the orange buttons on the right to make POST requests..',
 u'hint_4': u'Click the green button to follow a link with a GET request..',
 u'hint_5': u'Click the book icon to read docs for the link relation.',
 u'welcome': u'Welcome to a haltalk server.'}

Calling a HALNavigator will execute a GET request against the resource and returns its value (which it will cache).

POST requests

The docs for ht:signup explain the format of the POST request to sign up. So let’s actually sign up. Since we’ve set "ht" as our default curie, we can skip typing the curie for convenience. (Note: haltalk is a toy api for example purposes, don’t ever send plaintext passwords over an unencrypted connection in a real app!):

>>> fred23 = N['signup'].create(
... {'username': 'fred23',
...  'password': 'hunter2',
...  'real_name': 'Fred 23'}
... )
>>> fred23
HALNavigator(Haltalk.users.fred23)

Errors

If the user name had already been in use, a 400 would have been returned from the haltalk api. rest_navigator follows the Zen of Python guideline “Errors should never pass silently”. An exception would have been raised on a 400 or 500 status code. You can squelch this exception and just have the post call return a HALNavigator with a 400/500 status code if you want:

>>> dup_signup = N['ht:signup'].create({
...    'username': 'fred23',
...    'password': 'hunter2',
...    'real_name': 'Fred Wilson'
... }, raise_exc=False)
>>> dup_signup
OrphanHALNavigator(Haltalk.signup)  # 400!
>>> dup_signup.status
(400, 'Bad Request')
>>> dup_signup.state
{u"errors": {u"username": [u"is already taken"]}}

Authentication

In order to post something to haltalk, we need to authenticate with our newly created account. HALNavigator allows any authentication method that requests supports (so OAuth etc). For basic auth (which haltalk uses), we can just pass a tuple.

>>> N.authenticate(('fred23', 'hunter2'))  # All subsequent calls are authenticated

This doesn’t send anything to the server, it just sets the authentication details that we’ll use on the next request. Other authentication methods may contact the server immediately.

Now we can put it all together to create a new post:

>>> N_post = N['me'](name='fred23')['posts'].create({'content': 'My first post'})
>>> N_post
HALNavigator(Haltalk.posts.523670eff0e6370002000001)
>>> N_post()
{'content': 'My first post', 'created_at': '2015-06-13T19:38:59+00:00'}

It is also possible to specify a custom requests Session object when creating a new navigator.

For example, if you want to talk to a OAuth2 protected api, simply pass an OAuth2 Session object that will be used for all requests done by HALNavigator:

>>> from requests_oauthlib import OAuth2Session
>>> oauth2_session = OAuth2Session(r'client_id', token='token')
>>> N = Navigator.hal('https://api.example.com', session=oauth2_session)

Additional Topics

Identity Map

You don’t need to worry about inadvertently having two different navigators pointing to the same resource. rest_navigator will reuse the existing navigator instead of creating a new one

Iterating over a Navigator

If a resource has a link with the rel “next”, the navigator for that resource can be used as a python iterator. It will automatically raise a StopIteration exception if a resource in the chain does not have a next link. This makes moving through paged resources really simple and pythonic:

post_navigator = fred['ht:posts']
for post in post_navigator:
    # the first post will be post_navigator itself
    print(post.state)

Headers (Request vs. Response)

HTTP response headers are available in N.response.headers

Headers that will be sent on each request can be obtained through the session:

>>> N.session.headers
# Cookies, etc

Bracket mini-language

The bracket ([]) operator on Navigators has a lot of power. As we saw earlier, the main use is to get a new Navigator from a link relation:

>>> N2 = N['curie:link_rel']

But, it can also go more than one link deep, which is equivalent to using multiple brackets in a row:

>>> N3 = N['curie:first_link', 'curie:second_link']
# equivalent to:
N3 = N['curie:first_link']['curie:second_link']

And of course, if you set a default curie, you can omit it:

>>> N3 = N['first_link', 'second_link']

Internally, this is completely equivalent to repeatedly applying the bracket operator, so you can even use it to jump over intermediate objects that aren’t Navigators themselves:

>>> N['some-link', 3, 'another-link']

This would use the some-link link relation, select the third link from the list, and then follow another-link from that resource.

Default curie

You may specify a default curie when creating your Navigator:

>>> N = HALNavigator('http://haltalk.herokuapp.com', curie='ht')

Now, when you follow links, you may leave off the default curie if you want:

>>> N.links
{'ht:users': [HALNavigator(Haltalk.users)],
 'ht:signup': [HALNavigator(Haltalk.signup)],
 'ht:me': [HALNavigator(Haltalk.users.{name})],
 'ht:latest-posts': [HALNavigator(Haltalk.posts.latest)]
}
>>> N['ht:users']
HALNavigator(Haltalk.users)
>>> N['users']
HALNavigator(Haltalk.users)

The only exception is where the key being supplied is a IANA registered link relation, and there is a conflict (hint: this should be quite rare):

>>> N.links
{'ht:next': HALNavigator(Haltalk.unregistered),
  'next': HALNavigator(Haltalk.registered)}
>>> N['next']
HALNavigator(Haltalk.registered)

Specifying an api name

Sometimes the automatic api naming guesses poorly. If you’d like to override the default name, you can specify it when creating the navigator:

>>> N = Navigator.hal('http://api.example.com', apiname='MySpecialAPI')
HALNavigator(MySpecialAPI)

Embedded documents

In rest_navigator, embedded documents are treated transparently. This means that in many cases you don’t need to worry about whether a document is embedded or whether it’s just linked.

As an example, assume we have a resource like the following:

{
  "_links": {
     ...
     "xx:yams": {
        "href": "/yams"
     }
     ...
  },
  "_embedded": {
     "xx:pickles": {
       "_links": {
         "self": {"href": "/pickles"}
       },
       "state": "A pickle"
     }
  }
  ...
}

From here, you would access both the yams and the pickles resource with normal bracket syntax:

>>> Yams = N['xx:yams']
>>> Pickles = N['xx:pickles']

The only difference here is that Yams hasn’t been fetched yet, while Pickles is considered “resolved” already because we got it as an embedded document.

>>> Yams.resolved
False
>>> Yams.state # None
>>> Pickles.resolved
True
>>> Pickles.state
{'state': 'A pickle'}

If an embedded document has a self link, you can treat it just like you would any other resource. So if you want to refresh the resource, it’s as easy as:

>>> Pickles.fetch()

This will fetch the current state of the resource from the uri in its self link, even if you’ve never directly requested that uri before. If an embedded resource doesn’t have a self link, it will be an OrphanNavigator with the parent set to the resource it was embedded in.

Of course, if you need to directly distinguish between linked resources and embedded resources, there is an out:

>>> N.embedded()
{'xx:pickles': HALNavigator(api.pickles)
>>> N.links()
{'xx:yams': HALNavigator(api.yams)

However, when using the in operator, it will look in both for a key you’re interested in:

>>> 'yams' in N  # default curie is taken into account!
True
>>> 'xx:yams in N
True
>>> 'xx:pickles' in N
True

Development

Testing

To run tests, first install the pytest framework:

$ pip install -U pytest

To run tests, execute following from the root of the source directory:

$ py.test

Planned for the future

  • Ability to add hooks for different types, rels and profiles. If a link has one of these properties, it will call your hook when doing a server call.
  • Since HAL doesn’t specify what content type POSTs, PUTs, and PATCHes need to have, you can specify the hooks based on what the server will accept. This can trigger off either the rel type of the link, or rest navigator can do content negotiation over HTTP with the server directly to see what content types that resource will accept.

Contributors

Thanks very much to rest navigator’s contributors:

Internal API

A library to allow navigating rest apis easy.

class restnavigator.halnav.APICore(root, nav_class, apiname=None, default_curie=None, session=None, id_map=None)[source]

Shared data between navigators from a single api.

This should contain all state that is generally maintained from one navigator to the next.

authenticate(auth)[source]

Sets the authentication for future requests to the api

cache(link, nav)[source]

Stores a navigator in the identity map for the current api. Can take a link or a bare uri

get_cached(link, default=None)[source]

Retrieves a cached navigator from the id_map.

Either a Link object or a bare uri string may be passed in.

is_cached(link)[source]

Returns whether the current navigator is cached. Intended to be overwritten and customized by subclasses.

class restnavigator.halnav.HALNavigator(link, core, response=None, state=None, curies=None, _links=None, _embedded=None)[source]

The main navigation entity

create(body=None, raise_exc=True, headers=None)[source]

Performs an HTTP POST to the server, to create a subordinate resource. Returns a new HALNavigator representing that resource.

body may either be a string or a dictionary representing json headers are additional headers to send in the request

delete(raise_exc=True, headers=None)[source]

Performs an HTTP DELETE to the server, to delete resource(s).

headers are additional headers to send in the request

fetch(raise_exc=True)[source]

Performs a GET request to the uri of this navigator

patch(body, raise_exc=True, headers=False)[source]

Performs an HTTP PATCH to the server. This is a non-idempotent call that may update all or a portion of the resource this navigator is pointing to. The format of the patch body is up to implementations.

body may either be a string or a dictionary representing json headers are additional headers to send in the request

upsert(body, raise_exc=True, headers=False)[source]

Performs an HTTP PUT to the server. This is an idempotent call that will create the resource this navigator is pointing to, or will update it if it already exists.

body may either be a string or a dictionary representing json headers are additional headers to send in the request

class restnavigator.halnav.HALNavigatorBase(link, core, response=None, state=None, curies=None, _links=None, _embedded=None)[source]

Base class for navigation objects

authenticate(auth)[source]

Authenticate with the api

docsfor(rel)[source]

Obtains the documentation for a link relation. Opens in a webbrowser window

embedded()[source]

Returns a dictionary of navigators representing embedded documents in the current resource. If the navigators have self links they can be fetched as well.

Returns a dictionary of navigators from the current resource. Fetches the resource if necessary.

Represents a HAL link. Does not store the link relation

relative_uri(root)[source]

Returns the link of the current uri compared against an api root

class restnavigator.halnav.Navigator[source]

A factory for other navigators. Makes creating them more convenient

static hal(root, apiname=None, default_curie=None, auth=None, headers=None, session=None)[source]

Create a HALNavigator

class restnavigator.halnav.OrphanHALNavigator(link, core, response=None, state=None, curies=None, _links=None, parent=None)[source]

A Special navigator that is the result of a non-GET

This navigator cannot be fetched or created, but has a special property called .parent that refers to the navigator this one was created from. If the result is a HAL document, it will be populated properly

class restnavigator.halnav.PartialNavigator(link, core=None)[source]

A lazy representation of a navigator. Expands to a full navigator when template arguments are given by calling it.

Expands with the given arguments and returns a new untemplated Link object

expand_uri(**kwargs)[source]

Returns the template uri expanded with the current arguments

variables

Returns a set of the template variables in this templated link

exception restnavigator.exc.HALNavigatorError(message, nav=None, status=None, response=None)[source]

Raised when a response is an error

Has all of the attributes of a normal HALNavigator. The error body can be returned by examining response.body

exception restnavigator.exc.NoResponseError[source]

Raised when accessing a field of a navigator that has not fetched a response yet

exception restnavigator.exc.OffTheRailsException(traversal, index, intermediates, e)[source]

Raised when a traversal specified to __getitem__ cannot be satisfied

exception restnavigator.exc.UnexpectedlyNotJSON(uri, response)[source]

Raised when a non-json parseable resource is gotten

exception restnavigator.exc.WileECoyoteException[source]

Raised when a url has a bad scheme

exception restnavigator.exc.ZachMorrisException[source]

Raised when a url has too many schemes

class restnavigator.utils.CurieDict(default_curie, d)[source]

dict subclass that allows specifying a default curie. This enables multiple ways to access an item

A list subclass that offers different ways of grabbing the values based on various metadata stored for each entry in the dictionary.

Note: Removing items from this list isn’t really the point, so no attempt has been made to make this convenient. Deleting items will not remove them from the list’s metadata.

append_with(obj, **properties)[source]

Add an item to the dictionary with the given metadata properties

get_by(prop, val, raise_exc=False)[source]

Retrieve an item from the dictionary with the given metadata properties. If there is no such item, None will be returned, if there are multiple such items, the first will be returned.

getall_by(prop, val)[source]

Retrieves all items from the dictionary with the given metadata

named(name)[source]

Returns .get_by(‘name’, name)

serialize

alias of unicode

restnavigator.utils.fix_scheme(url)[source]

Prepends the http:// scheme if necessary to a url. Fails if a scheme other than http is used

restnavigator.utils.getpath(d, json_path, default=None, sep='.')[source]

Gets a value nested in dictionaries containing dictionaries. Returns the default if any key in the path doesn’t exist.

restnavigator.utils.getstate(d)[source]

Deep copies a dict, and returns it without the keys _links and _embedded

restnavigator.utils.namify(root_uri)[source]

Turns a root uri into a less noisy representation that will probably make sense in most circumstances. Used by Navigator’s __repr__, but can be overridden if the Navigator is created with a ‘name’ parameter.

restnavigator.utils.normalize_getitem_args(args)[source]

Turns the arguments to __getitem__ magic methods into a uniform list of tuples and strings

restnavigator.utils.objectify_uri(relative_uri)[source]

Converts uris from path syntax to a json-like object syntax. In addition, url escaped characters are unescaped, but non-ascii characters a romanized using the unidecode library.

Examples:
“/blog/3/comments” becomes “blog[3].comments” “car/engine/piston” becomes “car.engine.piston”
restnavigator.utils.parse_media_type(media_type)[source]

Returns type, subtype, parameter tuple from an http media_type. Can be applied to the ‘Accept’ or ‘Content-Type’ http header fields.

Changelog

1.0

  • Embedded support
  • Ability to specify default curies
  • Resources with no URL are now represented by a special Navigator type called OrphanNavigators
  • IP addresses can be used in the url (@dudycooly)
  • All tests pass in python 2.6 -> 3.4 (@bubenkoff), and travis now runs tox to ensure they stay that way
  • Support the DELETE, and PATCH methods
  • posts allow an empty body (@bbsgfalconer)
  • Much improved content negotiation (@bbsgfalconer)
  • There was also a major refactoring that changed how Navigators are created and internally cleaned up a
    lot of really messy code.