RESTful Web Services - RESTful Web APIs

Welcome to 43.9°N 103.46°W on scenic Earth.

See this location on a map:

• Road
• Satellite
...

Things that are here:

• Mount Rushmore

Show nearby places, features, or businesses:

Places matching Springfield in or around the United States of America:

• Springfield, IL
• Springfield, MA
• Springfield, MO
...

This representation is made up almost entirely of links to places. There’s the link to the place that was searched, “the United States of America” (a place of type “country”). There are also links to various places that matched the search criteria. Each of these places is a resource, and exposes a representation that looks like Example 5-8. Each link contains enough scoping information to uniquely identify the Springfield in question. A client can follow the links in the representations to find information about the places, as well as maps of them. Figure 5-3 shows some of the links you can follow from /Earth/USA?show=Springfield. Google Maps presents search results by sewing image tiles together to make a largescale map, and annotating the map with graphical markers. You can write a client for this web service that does the same thing. The first step is to build the large-scale map. The client follows the initial link to /Earth/USA and gets a representation like the one in Example 5-4. This gives the client the address of one graphical tile. The client can get adjacent tiles by following navigation links, and stitch them together into a largescale tile map of the whole country. The second step is to stick markers on the map, one for each search result. To find out where a marker should go, the client follows one of the search result links, fetching a representation like the one in Example 5-8. This representation lists the latitude and longitude of the appropriate Springfield. That’s potentially a lot of link following, but my representations are simple and so are the rules for going from one to another. I’ve spread out my method="get" action="">

Show places, features, or businesses:



A person using a web browser would see this form as a set of GUI elements: a text box, a button, and a set of labels. They’d put some syntax before. It’s a feature of XHTML 5, which is still being designed as this book goes to press. Occasionally in this chapter and the next, I’ll introduce a feature of XHTML 5 to work around the shortcomings of XHTML 4 as a hypermedia format. The problem here is that my service accepts any number of values for the query variable show. A client can make a simple search such as ?show=diners or perform a complicated search such as ?show=diners&show=arsenic&show=towns&show=oil+tankers. A form in XHTML 4 could allow the latter request if it showed four text boxes, all called show. But an HTML form can never show an arbitrary number of text boxes, which is what I need to truly capture the capabilities of my service. XHTML 5 has a feature called the repetition model, which allows me to express an arbitrary number of text boxes without writing an infinitely long HTML page.

136 | Chapter 5: Designing Read-Only Resource-Oriented Services

Now my service is better connected. It’s now possible to get from the list of planets to a description of a diner near Mount Rushmore (assuming there is one). Figure 5-4 illustrates the journey. Starting at the service root (/), the client selects the planet Earth (/Earth). The client uses the HTML form in that representation to search for places called “Mount Rushmore” on Earth (/Earth?show=Mount%20Rushmore). Hopefully the top search result will be Mount Rushmore itself, and the client can follow the first search result link to /Earth/USA/Mount%20Rushmore. The representation of Mount Rushmore has a search form too, and the client enters “diners” in that. Assuming there are any nearby diners, the client can follow the first search result link to find a diner near Mount Rushmore. The search function doesn’t just keep clients from having to mint their own URIs. It resolves human place names, which are always fuzzy, into canonical resource URIs. A client should be also be able to search for “Mount Rushmore National Monument” and get /Earth/USA/Mount%20Rushmore as a search result, just like the client can search for “Springfield” and pick which Springfield they mean. This is a useful feature for any client that lets users type in their own place names.

The HTTP Response I’m almost done with this design. I know what method="POST">

If the user enters the values “blue” and “green” in the text fields color1 and color2 fields, a form-encoded representation of that xml:lang="en"> User homepage for leonardr

You are currently logged in as leonardr.

User homepage for leonardr

Change your password:




Once again I’m using the representation to convey the current resource state, and to help the client drive to other states. I used an HTML form to describe a future PUT request the client might make if it wants to change the user’s password (an item of resource state). Note that there’s no form telling the client how to get a representation, or how to delete this user. It’s taken for granted that you use HTTP GET and DELETE for that. I only need hypermedia for complicated things: links to other resources (so the client knows which URI to GET or DELETE), and descriptions of representations. You may have noticed a problem in Example 6-3. Its form specifies an HTTP method of PUT, but HTML forms only allow GET and POST. As with the “repeat” syntax in Example 5-11, I’m using the as-yet-unreleased XHTML 5 to get around the shortcomings of the current version of HTML. Another way to handle this is to send a WADL snippet instead of an HTML form, or use the trick described in Chapter 8 to run PUT requests over overloaded POST.

If you GET someone else’s user account, you’ll be served a different representation, similar to the one in Example 6-4. Example 6-4. A representation of someone else’s user account

User Accounts as Resources | 153

User homepage for samruby

You are currently logged in as leonardr.

User homepage for samruby



This representation has no controls for altering the state of the resource, because the client isn’t authorized to do that: the client authenticated as leonardr and this is sam ruby’s page. Right now the representation does nothing but confirm to leonardr that a user named samruby exists. If there was no such user, a GET request to /user/samruby would give a status code of 404 (“Not Found”), and the client would be free to create samruby with a PUT request.

Link This Resource to Existing Resources In the previous chapter I defined several classes of resource: the list of maps, individual maps, places, and lists of places (that is, lists of search results). None of these are directly relevant to user accounts, but there are a couple of nice features I can add at this point. One nice feature is to add the “authenticated” message (seen in the two sample representations above) to the representation of every resource. It’ll be displayed whenever the client submits a request with valid credentials. The “authenticated” message is a piece of hypermedia that shows an authenticated client how to retrieve method="PUT" template="/user/{username}">

Username:


Password:


The two deviations from the HTML you’re familiar with are in the method attribute (like Example 6-3, it specifies PUT where HTML 4 allows only GET and POST), and the brand-new template attribute, which inserts a form variable (“username”) into the URI using the URI Templating standard (http://www.ietf.org/internet-drafts/draft-gregoriouritemplate-00.txt). As of the time of writing, URI Templating was a proposed addition to HTML 5, but it hadn’t been approved. It’s possible that it will be rejected, and that Example 6-5 won’t be valid HTML 5 any more than it is valid HTML 4. In that case you can use URI Templating unofficially (forcing your users to write custom clients), or switch to WADL.

The hypermedia form talks about the syntax of the PUT request, but it can’t say much about the semantics. A web service client can read the HTML form in Example 6-5, but its understanding is limited. It knows that the form is labelled “createUser” but it doesn’t know what “createUser” means. It knows that if it PUTs a certain representation to a certain URI, the server will probably accept it. It knows what PUT means, because PUT always means the same thing. It knows that the representation should include a “username,” but it doesn’t know a username from an ostrich. It takes a human being—a programmer—to understand what a user is, that “createUser” means “create a user,” what a username is, and all the rest. A programmer needs to set the rules about when and how user accounts are created. This piece of hypermedia does nothing but tell the client how to structure the PUT request when it comes time to “createUser,” whatever that means. It’s a promise from the web service to the client. Many web services put all of this method="PUT" action="">

Modify this place:

Name:
Type:
Position: ,
Description: We visited on 3/5/2005
Public?



The caveats from earlier apply here too. This isn’t valid XHTML 4, though it is valid XHTML 5, because it specifies PUT as its method. Also, a client doesn’t know what to do with this form unless it’s been programmed in advance. Computers don’t know what “modifyPlace” means or what xml:lang="en"> 43.9°N 103.46°W on Earth

162 | Chapter 6: Designing Read/Write Resource-Oriented Services



You are currently logged in as leonardr.

Welcome to 43.9°N 103.46°W on scenic Earth.

See this location on a map:

• Road
• Satellite
...

Places at this location:

• Mount Rushmore System >The world's largest sculpture
• Mt. Rushmore user1 says: Built on land stolen from the Lakota tribe
• Mount Rushmore Gift Shop user1 says: Best fudge I've ever had
• Mt. Rushmore You said: We visited on 3/5/2005

Show nearby places, features, or businesses:

Custom Places | 163



Create a new place here:

Name:
Type:
Description:
Public?



What’s Supposed to Happen? This new resource, the custom place, mostly works like other resources I’ve already defined. A custom place responds to GET just like a built-in place. It responds to PUT (with a representation consisting of key-value pairs) and DELETE (with no representation) just like “user account” resources do. I only have a couple new edge cases to consider here. When the client creates a custom place, the response code is 201 (“Created”). This works the same way as users. But it was never possible to cause a user’s URI to change, because I prohibited users from changing their usernames. It’s possible to change the name of a place, or to move one (say, a ship) from one point on the map to another. Either of these actions will change the URI. When the client modifies a custom place without changing its location, the response code will be 200 (“OK”). If the location changes, the response code will be 301 (“Moved Permanently”) and the Location header will contain the place’s new URI. The client is responsible for updating its rel="self"/>

Controller Code | 195

leonardr http://localhost:3000/v1/users/leonardr [email protected] REST and WS-*/title> Joe Gregorio's lucid explanation of RESTful principles ... 2007-02-14T02:26:58-05:00

The UsersController Now I’m ready to show you some specific actions. I’ll start with the controller that makes user accounts possible. In the code in Example 7-14, note the call to before_filter that sets up the must_authenticate filter. You don’t need to authenticate to create (POST) a user account (as whom would you authenticate?), but you must authenticate to modify (PUT) or destroy (DELETE) an account. Example 7-14. app/controllers/users_controller.rb class UsersController < ApplicationController # A client must authenticate to modify or delete a user account. before_filter :must_authenticate, :only => [:modify, :destroy] # POST /users def create user = User.find_by_name(params[:user][:name]) if user # The client tried to create a user that already exists. headers['Location'] = user_url(user.name) render :nothing => true, :status => "409 Conflict" else user = User.new(params[:user]) if user.save headers['Location'] = user_path(user.name) render :nothing => true, :status => "201 Created" else # There was a problem saving the user to the href="/road.1/Earth/37.0;-95.8" />Zoom out Zoom in

Instead of providing these links everywhere, the service provider could put up an English document telling the authors of automated clients how to manipulate the zoom level in the first path variable. That would disconnect some related resources (the road map at different zoom levels), but it would save some bandwidth in every representation and it would have little effect on the actual code of any automated client. Personally, if I was writing a client for this service, I’d rather get from zoom level 8 to zoom level 4 by setting road.4 directly, than by following the “Zoom out” link over and over again. My client will break if the URI construction rule ever changes, but maybe I’m willing to take that risk. Now consider this bit of markup from the representation of the planet Earth. It’s reprinted from Example 5-7: name Earth maps
• Road
• Satellite
...


The URIs are technically redundant. The name of the place indicates that these are maps of Earth, and the link text indicate that there’s a satellite and a road map. Given those two pieces of information, a client can construct the corresponding map URI using a rule like the one for S3 objects: slash, map type, slash, planet name. Since the URIs can be replaced by a simple rule, the service might follow the S3 model and save some bandwidth by presenting the representation of Earth in an XML format like this: ...

If I was writing a client for this service, I would rather be given those links than have to construct them myself, but it’s up for debate. Here’s another bit of markup from Example 5-6. These links are to help the client move from one tile on the map to another. Southeast

This Stuff Matters | 225

It’s technically possible for a client to generate these URIs based on rules. After all, the server is generating them based on rules. But the rules involve knowing how latitude and longitude work, the scale of the map at the current zoom level, and the size and shape of the planet. Any client programmer would agree it’s easier to navigate a map by following the links than by calculating the coordinates of tiles. We’ve reached a point at which the relationships between resources are too complex to be expressed in simple rules. Connectedness becomes very important. This is where Google Maps’s tile-based navigation system pays off (I described that system back in “Representing Maps and Points on Maps” in Chapter 5, if you’re curious). Google Maps addresses its tiles by arbitrary X and Y coordinates instead of latitude and longitude. Finding the tile to the north is usually as easy as subtracting one from the value of Y. The relationships between tiles are much simpler. Nobody made me design my tile system in terms of latitude and longitude. If latitude/longitude calculations are why I have to send navigation links along with every map representation, maybe I should rethink my strategy and expose simpler URIs, so that my clients can generate them more easily. But there’s another reason why connectedness is valuable: it makes it possible for the client to handle relationships that change over time. Links not only hide the rules about how to build a URI for a given resource, they embody the rules of how resources are related to each other. Here’s a terrifying example to illustrate the point.

A terrifying example Suppose I get some new map >Checking #11: New balance $150 Savings #55: New balance $250 ...

At this point I can DELETE the transaction and it won’t be rolled back. Or the server might delete it automatically. More likely, it will be archived permanently as part of an audit trail. It’s an addressable resource. Other resources, such as a list of transactions that affected checking account #11, can link to it. The challenge in representing transactions RESTfully is that every HTTP request is supposed to be a self-contained operation that operates on one resource. If you PUT a new balance to /accounts/checking/11, then either the PUT succeeds or it doesn’t. But during a transaction, the state of a resource is in flux. Look at the checking account from inside the transaction, and the balance is $150. Look at it from outside, and the balance is still $200. It’s almost as though there are two different resources. That’s how this solution presents it: as two different resources. There’s the actual checking account, at /accounts/checking/11, and there’s one transaction’s view of the checking account, at /transactions/account-transfer/11a5/accounts/checking/11. When I POSTed to create /transactions/account-transfer/11a5/, the service exposed additional resources beneath the transaction URI: probably one resource for each account on the system. I manipulated those resources as I would the corresponding account resources, but my changes to resource state didn’t go “live” until I committed the transaction. How would this be implemented behind the scenes? Probably with something that takes incoming requests and builds a queue of actions associated with the transaction. When the transaction is committed the server might start a

This is a challenge. The server dares me to repeat my request with the correct credentials. The WWW-Authenticate header gives two clues about what credentials I should send. It identifies what kind of authentication it’s using (in this case, Basic), and it names a realm. The realm can be any name you like, and it’s generally used to identify a collection of resources on a site. In Chapter 7 the realm was “Social bookmarking service” (I defined it in Example 7-11). A single web site might have many sets of protected resources guarded in different ways: the realm lets the client know which authentication credentials it should provide. The realm is the what, and the authentication type is the how. To meet a Basic authentication challenge, the client needs a username and a password. This information might be filed in a cache under the name of the realm, or the client may have to prompt an end user for this information. Once the client has this information, username and password are combined into a single string and encoded with base 64 encoding. Most languages have a standard library for doing this kind of encoding: Example 8-1 uses Ruby to encode a username and password. 238 | Chapter 8: REST and ROA Best Practices

Example 8-1. Base 64 encoding in Ruby #!/usr/bin/ruby # calculate-base64.rb USER="Alibaba" PASSWORD="open sesame" require 'base64' puts Base64.encode64("#{USER}:#{PASSWORD}") # QWxpYmFiYTpvcGVuIHNlc2FtZQ==

This seemingly random string of characters is the value of the Authorization header. Now I can send my request again, using the username and password as Basic auth credentials. GET /resource.html HTTP/1.1 Host: www.example.com Authorization: Basic QWxpYmFiYTpvcGVuIHNlc2FtZQ==

The server decodes this string and matches it against its user and password list. If they match, the response is processed further. If not, the request fails, and once again the status code is 401 (“Unauthorized”). Of course, if the server can decode this string, so can anyone who snoops on your network traffic. Basic authentication effectively transmits usernames and passwords in plain text. One solution to this is to use HTTPS, also known as Transport Level Security or Secure Sockets Layer. HTTPS encrypts all communications between client and server, incidentally including the Authorization header. When I added authentication to my map service in Chapter 6, I switched from plain HTTP to encrypted HTTPS.

Digest authentication HTTP Digest authentication is another way to hide the authorization credentials from network snoops. It’s more complex than Basic authentication, but it’s secure even over unencrypted HTTP. Digest follows the same basic pattern as Basic: the client issues a request, and gets a challenge. Here’s a sample challenge: 401 Unauthorized WWW-Authenticate: Digest realm="My Private , nonce="0cc175b9c0f1b6a831c399e269772661", opaque="92eb5ffee6ae2fec3ad71c777531578f"

This time, the WWW-Authenticate header says that the authentication type is Digest. The header specifies a realm as before, but it also contains three other pieces of information, including a nonce: a random string that changes on every request. The client’s responsibility is to turn this information into an encrypted string that proves the client knows the password, but that doesn’t actually contain the password. First the client generates a client-side nonce and a sequence number. Then the client makes a single “digest” string out of a huge amount of information: the HTTP method Standard Features of HTTP | 239

and path from the request, the four pieces of information from the challenge, the username and password, the client-side nonce, and the sequence number. The formula for doing this is considerably more complicated than for Basic authentication (see Example 8-2). Example 8-2. HTTP digest calculation in Ruby #!/usr/bin/ruby # calculate-http-digest.rb require 'md5' #Information from the original request METHOD="GET" PATH="/resource.html" # Information from the challenge REALM="My Private , OPAQUE="92eb5ffee6ae2fec3ad71c777531578f" QOP="auth" # Information calculated by or known to the client NC="00000001" CNONCE="4a8a08f09d37b73795649038408b5f33" USER="Alibaba" PASSWORD="open sesame" # Calculate the final digest in three steps. ha1 = MD5::hexdigest("#{USER}:#{REALM}:#{PASSWORD}") ha2 = MD5::hexdigest("#{METHOD}:#{PATH}") ha3 = MD5::hexdigest("#{ha1}:#{NONCE}:#{NC}:#{CNONCE}:#{QOP}:#{ha2}") puts ha3 # 2370039ff8a9fb83b4293210b5fb53e3

The digest string is similar to the S3 request signature in Chapter 3. It proves certain things about the client. You could never produce this string unless you knew the client’s username and password, knew what request the client was trying to make, and knew which challenge the server had sent in response to the first request. Once the digest is calculated, the client resends the request and passes back all the constants (except, of course, the password), as well as the final result of the calculation: GET /resource.html HTTP/1.1 Host: www.example.com Authorization: Digest username="Alibaba", realm="My Private , uri="/resource.html", qop=auth, nc=00000001, cnonce="4a8a08f09d37b73795649038408b5f33", response="2370039ff8a9fb83b4293210b5fb53e3", opaque="92eb5ffee6ae2fec3ad71c777531578f"

240 | Chapter 8: REST and ROA Best Practices

The cryptography is considerably more complicated, but the process is the same as for HTTP Basic auth: request, challenge, response. One key difference is that even the server can’t figure out your password from the digest. When a client initially sets a password for a realm, the server needs to calculate the hash of user:realm:password (ha1 in the example above), and keep it on file. That gives the server the information it needs to calculate the final value of ha3, without storing the user’s actual password. A second difference is that every request the client makes is actually two requests. The point of the first request is to get a challenge: it includes no authentication information, and it always fails with a status code of 401 (“Unauthorized”). But the WWW-Authenti cate header includes a unique nonce, which the client can use to construct an appropriate Authorization header. It makes a second request, using this header, and this one is the one that succeeds. In Basic auth, the client can avoid the challenge by sending its authorization credentials along with the first request. That’s not possible in Digest. Digest authentication has some options I haven’t shown here. Specifying qop=authint instead of qop-auth means that the calculation of ha2 above must include the request’s entity-body, not just the HTTP method and the URI path. This prevents a manin-the-middle from tampering with the representations that accompany PUT and POST requests. My goal here isn’t to dwell on the complex mathematics— that’s what libraries are for. I want to demonstrate the central role the WWW-Authenticate and Authorization headers play in this exchange. The WWW-Authenticate header says, “Here’s everything you need to know to authenticate, assuming you know the secret.” The Authorization header says, “I know the secret, and here’s the proof.” Everything else is parameter parsing and a few lines of code.

WSSE username token What if neither HTTP Basic or HTTP Digest work for you? You can define your own standards for what goes into WWW-Authenticate and Authorization. Here’s one real-life example. It turns out that, for a variety of technical reasons, users with low-cost hosting accounts can’t take advantage of either HTTP Basic or HTTP Digest.‡ At one time, this was important to a segment of the Atom community. Coming up with an entirely new cryptographically secure option was beyond the ability of the Atom working group. Instead, they looked to the WS-Security specification, which defines several different ways of authenticating SOAP messages with SOAP headers. (SOAP headers are the “stickers” on the SOAP envelope I mentioned back in Chapter 1.) They took a single idea—WS-Security UsernameToken—from this standard and ported it from SOAP headers to HTTP headers. They defined an extension to HTTP that used WWW-Authenticate and Authorization in a way that people with low-cost hosting accounts could use. We call the resulting extension WSSE UsernameToken, or WSSE for ‡ Documented by Mark Pilgrim in

“Atom Authentication”

(http://www.xml.com/pub/a/2003/12/17/

dive.html) on xml.com.

Standard Features of HTTP | 241

short. (WSSE just means WS-Security Extension. Other extensions would have a claim to the same name, but there aren’t any others right now.) WSSE is like Digest in that the client runs their password through a hash algorithm before sending it across the network. The basic pattern is the same: the client makes a request, gets a challenge, and formulates a response. A WSSE challenge might look like this: HTTP/1.1 401 Unauthorized WWW-Authenticate: WSSE realm="My Private

Instead of Basic or Digest, the authentication type is WSSE. The realm serves the same purpose as before, and the “profile” tells the client that the server expects it to generate a response using the UsernameToken rules (as opposed to some other rule from WSSecurity that hasn’t yet been ported to HTTP headers). The UsernameToken rules mean that the client generates a nonce, then hashes their password along with the nonce and the current date (see Example 8-3). Example 8-3. Calculating a WSSE digest #!/usr/bin/ruby # calculate-wsse-digest.rb require 'base64' require 'sha1' PASSWORD = "open sesame" NONCE = "EFD89F06CCB28C89", CREATED = "2007-04-13T09:00:00Z" puts Base64.encode64(SHA1.digest("#{NONCE}#{CREATED}#{PASSWORD}")) # Z2Y59TewHV6r9BWjtHLkKfUjm2k=

Now the client can send a response to the WSSE challenge: GET /resource.html HTTP/1.1 Host: www.example.com Authorization: WSSE profile="UsernameToken" X-WSSE: UsernameToken Username="Alibaba", PasswordDigest="Z2Y59TewHV6r9BWjtHLkKfUjm2k=", Nonce="EFD89F06CCB28C89", Created="2007-04-13T09:00:00Z"

Same headers. Different authentication method. Same message flow. Different hash algorithm. That’s all it takes to extend HTTP authentication. If you’re curious, here’s what those authentication credentials would look like as a SOAP header under the original WS-Security UsernameToken standard. Alibaba

242 | Chapter 8: REST and ROA Best Practices

Z2Y59TewHV6r9BWjtHLkKfUjm2k= EFD89F06CCB28C89 2007-04-13T09:00:00Z

WSSE UsernameToken authentication has two big advantages. It doesn’t send the password in the clear over the network, the way HTTP Basic does, and it doesn’t require any special setup on the server side, the way HTTP Digest usually does. It’s got one big disadvantage. Under HTTP Basic and Digest, the server can keep a one-way hash of the password instead of the password itself. If the server gets cracked, the passwords are still (somewhat) safe. With WSSE UsernameToken, the server must store the password in plain text, or it can’t verify the responses to its challenges. If someone cracks the server, they’ve got all the passwords. The extra complexity of HTTP Digest is meant to stop this from happening. Security always involves tradeoffs like these.

Compression Textual representations like XML documents can be compressed to a fraction of their original size. An HTTP client library can request a compressed version of a representation and then transparently decompress it for its user. Here’s how it works: along with an HTTP request the client sends an Accept-Encoding header that says what kind of compression algorithms the client understands. The two standard values for AcceptEncoding are compress and gzip. GET /resource.html HTTP/1.1 Host: www.example.com Accept-Encoding: gzip,compresss

If the server understands one of the compression algorithms from Accept-Encoding, it can use that algorithm to compress the representation before serving it. The server sends the same Content-Type it would send if the representation wasn’t compressed. But it also sends the Content-Encoding header, so the client knows the document has been compressed: 200 OK Content-Type: text/html Content-Encoding: gzip [Binary representation goes here]

The client decompresses the

Now the web application can make a web-service call as the end user, without actually knowing anything about the end user. The authentication information never leaves google.com, and the authorization token is only good for one request. Those are the basics. Google’s user-client authorization mechanism has lots of other features. A web service client can use the one-time authorization token to get a “session token” that’s good for more than one request. A client can digitally sign requests, similarly to how I signed Amazon S3 requests back in Chapter 3. These features are different for every user-client authorization mechanism, so I won’t dwell on them here. The point is this flow (shown graphically in Figure 8-3): control moves from the web application’s domain to the web service’s domain. The user authenticates with the web service, and authorizes the foreign web application to act on her behalf. Then control moves back to the web application’s domain. Now the web app has an authorization token that it can use in the Authorization header. It can make web service calls without knowing the user’s username and password.

Applications with No Web Interface For applications that expose a web interface, browser-based user-client authorization makes sense. The user is already in her web browser, and the application she’s using is running on a faraway server. She doesn’t trust the web application with her password, but she does trust her own web browser. But what if the web service client is a standalone application running on the user’s computer? What if it’s got a GUI or commandline interface, but it’s not a web browser?

Why Should a User Trust the HTTP Client? | 255

There are two schools of thought on this. The first is that the end user should trust any client-side application as much as she trusts her web browser. Web applications run on an untrusted computer, but I control every web service client that runs on my computer. I can keep track of what the clients are doing and kill them if they get out of control. If you as a service designer subscribe to this philosophy, there’s no need to hide the end user’s username and password from desktop clients. They’re all just as trustworthy as the web browser. Google takes this attitude. Its authentication mechanism for clientside applications (http://code.google.com/apis/accounts/AuthForInstalledApps.html) is different from the web-based one I described above. Both systems are based on tokens, but desktop applications get an authorization token by gathering the user’s username and password and “logging in” as them—not by redirecting the user’s browser to a Google login page. This token serves little purpose from a security standpoint. The client needs a token to make web service requests, but it can only get one if it knows the user’s username and password—a far more valuable prize. If you don’t like this, then you probably think the web browser is the only client an end user should trust with her username and password. This creates a problem for the programmer of a desktop client. Getting an authentication token means starting up a trusted client—the web browser—and getting the end user to visit a certain URI. For the Flickr service the URI might look like this: http://flickr.com/services/auth/?perms=write&api_sig=925e1&api_key=1234&frob=abcd

The most important query variable here is frob. That’s a predefined ID, obtained through an earlier web service call, and I’ll use it in a moment. The first thing the end user sees is that her browser suddenly pops up and visits this URI, which shows a Flickr login screen. The end user gives her authentication credentials and authorizes the client with api_key=1234 to act on her behalf. In the Google example above, the web service client was the web application at example.com. Here, the web service client is the application running on the end user’s own desktop. Without the frob, the desktop client at this point would have to cajole the end user to copy and paste the authorization token from the browser into the desktop client. But the client and the service agreed on a frob ahead of time, and the desktop client can use this frob to get the authorization token. The end user can close his browser at this point, and the desktop client makes a GET request to a URI that looks like this: http://flickr.com/services/rest/?method=flickr.auth.getToken &api_sig=1f348&api_key=1234&frob=abcd

The eBay and Flickr web services use a mechanism like this: what Flickr calls a frob, eBay calls an runame. The end user can authorize a standalone client to make web service requests on her behalf, without ever telling it her username or password. I’ve diagrammed the whole process in Figure 8-4.

256 | Chapter 8: REST and ROA Best Practices

1

Client-side app

2

Client-side app

Web service request

flickr.auth.getFrob

Flickr auth page ?frob=abcd

Display

Web service request

OK, I trust this app I am running Decision Flickr username/password

User clicks “continue” in client-side app

3

Client-side app

4

Client-side app

flickr.com

flickr.auth.getToken

Authorized web service request

flickr.com

Figure 8-4. How a web application gets authorization to use Flickr

Some mobile devices have network connectivity but no web browser. A web service that thinks the only trusted client is a web browser must make special allowances for such devices, or live with the fact that it’s locking them out.

What Problem Does this Solve? Despite appearances, I’ve gone into very little detail: just enough to give you a feel for the two ways an end user might delegate her authority to make web service calls. Even in the high-level view it’s a complex system, and it’s worth asking what problem it actually solves. After all, the end user still has to type her username and password into a web form, and nothing prevents a malicious application writer from sending the browser to a fake authorization page instead of the real page. Phishers redirect people to fake sign-in pages all the time, and a lot of people fall for it. So what does this additional infrastructure really buy? If you look at a bank or some other web site that’s a common target of phishing attacks, you’ll see a big warning somewhere that looks like this: “Never type in your mybank.com username and password unless you’re using a web browser and visiting a URI that starts with https://www.mybank.com/.” Common sense, right? It’s not the most ironclad guarantee of security, but if you’re careful you’ll be all right. Yet most web services can’t even provide this milquetoast cover. The standalone applications Why Should a User Trust the HTTP Client? | 257

presented throughout this book take your service username and password as input. Can you trust them? If the web site at example.com wants to help you manage your del.icio.us bookmarks, you need to give it your del.icio.us username and password. Do you trust example.com? The human web has a universal client: the web browser. It’s not a big leap of faith to trust a single client that runs on your computer. The programmable web has different clients for different purposes. Should the end user trust all those clients? The mechanisms I described in this section let the end user use her web browser—which she already trusts—as a way of bestowing lesser levels of trust on other clients. If a client abuses the trust, it can be blocked from making future web service requests. These strategies don’t eliminate phishing attacks, but they make it possible for a savvy end user to avoid them, and they allow service providers to issue warnings and disclaimers. Without these mechanisms, it’s technically impossible for the end user to tell the difference between a legitimate client and a phishing site. They both take your password: the only difference is what they do with it.

258 | Chapter 8: REST and ROA Best Practices

CHAPTER 9

The Building Blocks of Services

Throughout this book I’ve said that web services are based on three fundamental technologies: HTTP, URIs, and XML. But there are also lots of technologies that build on top of these. You can usually save yourself some work and broaden your audience by adopting these extra technologies: perhaps a domain-specific XML vocabulary, or a standard set of rules for exposing resources through HTTP’s uniform interface. In this chapter I’ll show you several technologies that can improve your web services. Some you’re already familiar with and some will probably be new to you, but they’re all interesting and powerful.

Representation Formats What representation formats should your service actually send and receive? This is the question of how > home: +1.415.555.1212

Microformat adoption is growing, especially as more special-purpose devices get on the web. Any microformat document can be embedded in an XHTML page, because it is XHTML. A web service can serve an XHTML representation that contains microformat documents, along with links to other resources and forms for creating new ones. This document can be automatically parsed for its microformat rel="license"> Made avaliable under a Creative Commons Attribution-NoDerivs license.

That’s standard XHTML. The only thing the microformat does is define a meaning for the string license when it shows up in the rel attribute. rel-nofollow A new value for the rel attribute, used when linking to URIs without necessarily endorsing them. rel-tag A new value for the rel attribute, used to label a web page according to some external classification system. VoteLinks A new value for the rev attribute, an extension of the idea behind rel-nofollow. VoteLinks lets you say how you feel about the resource you’re linking to by casting a “vote.” For instance: The best webpage ever. A shameless ripoff of www.example.com

XFN Stands for XHTML Friends Network. A new set of values for the rel attribute, for capturing the relationships between people. An XFN value for the rel attribute captures the relationship between this “person” resource and another such resource. To bring back the “Alice” and “Bob” resources from “Relationships Between Resources” in Chapter 8, an XHTML representation of Alice might include this link: Bob

XMDP Stands for XHTML Meta encoding="utf-8"?> RESTful News 2007-04-14T20:00:39Z Leonard Richardson Sam Ruby urn:1c6627a0-8e3f-0129-b1a6-003065546f18 New Resource Will Respond to PUT, City Says urn:239b2f40-8e3f-0129-b1a6-003065546f18 2007-04-14T20:00:39Z After long negotiations, city officials say the new resource being built in the town square will respond to PUT. Earlier criticism of the proposal focused on the city's plan to modify the resource through overloaded POST.

In that example you can see some of the tags that convey the semantics of publishing: author, title, link, summary, updated, and so on. The feed as a whole is a joint project: it has an author tag and a contributor tag. It’s also got a link tag that points to an alternate URI for the underlying “feed” resource: the news site. The single entry has no author tag, so it inherits author information from the feed. The entry does have its own link tag, which points to http://www.example.com/RestfulNews/104. That URI identifies the entry as a resource in its own right. The entry also has a textual summary of the story. To get the remainder, the client must presumably GET the entry’s URI.

264 | Chapter 9: The Building Blocks of Services

An Atom document is basically a directory of published resources. You can use Atom to represent photo galleries, albums of music (maybe a link to the cover art plus one to each track on the album), or lists of search results. Or you can omit the LINK tags and use Atom as a container for original content like status reports or incoming emails. Remember: the two reasons to use Atom are that it represents the semantics of publishing, and that a lot of existing clients can consume it. If your application almost fits in with the Atom schema, but needs an extra tag or two, there’s no problem. You can embed XML tags from other namespaces in an Atom feed. You can even define a custom namespace and embed its tags in your Atom feeds. This is the Atom equivalent of XHTML microformats: your Atom feeds can use conventions not defined in Atom, without becoming invalid. Clients that don’t understand your tag will see a normal Atom feed with some extra mysterious property="dc:title"> RESTful Web Services

There are three parts to an RDF assertion, or triple, as they’re called. There’s the subject, a resource identifier: in this case, isbn:9780596529260. There’s the predicate, which identifies a property of the resource: in this case, dc:title. Finally there’s the object, which is the value of the property: in this case, “RESTful Web Services.” The assertion as a whole reads: “The book with ISBN 9780596529260 has a title of ‘RESTful Web Services.’” I didn’t make up the isbn: URI space: it’s a standard way of addressing books as resources. I didn’t make up the dc:title predicate, either. That comes from the Dublin Core Meta encoding="UTF-8"?>

All but one of my recommended representation formats is based on XML, so that solves most of the problem. But there is an encoding problem with that one outlier, and there’s a further problem in the relationship between XML and HTTP.

XML and HTTP: Battle of the encodings An XML document can and should define a character encoding in its first line, so that the client will know how to interpret the document. An HTTP response can and should specify a value for the Content-Type response header, so that the client knows it’s being given an XML document and not some other kind. But the Content-type can also specify a document character encoding with “charset,” and this encoding might conflict with what it actually says in the document. Content-Type: application/xml; charset="ebcdic-fr-297+euro"

Who wins? Surprisingly, HTTP’s character encoding takes precedence over the encoding in the document itself.†If the document says “UTF-8” and Content-Type says “ebcdic-fr-297+euro,” then extended French EBCDIC it is. Almost no one expects this kind of surprise, and most programmers write code first and check the RFCs later. The result is that the character encoding, as specified in Content-Type, tends to be unreliable. Some

† This is specified, and argued for, in RFC 3023.

Representation Formats | 271

servers claim everything they serve is UTF-8, even though the actual documents say otherwise. When serving XML documents, I don’t recommend going out of your way to send a character encoding as part of Content-type. You can do it if you’re absolutely sure you’ve got the right encoding, but it won’t do much good. What’s really important is that you specify a document encoding. (Technically you can do without a document encoding if you’re using UTF-8, or if you’re using UTF-16 with a byte-order mark. But if you have that much control over the encoding="utf-8"?> New Resource Will Respond to PUT, City Says After long negotiations, city officials say the new resource being built in the town square will respond to PUT. Earlier criticism of the proposal focused on the city's plan to modify the resource through overloaded POST.

Service document This vaguely-named type of resource is just a grouping of collections. A typical move is to serve a single service document, listing all of your collections, as your service’s “home page.” A service document is an XML document written using a particular vocabulary, and its media type is application/atomserv+xml (see Example 9-6). Example 9-6 shows a representation of a typical service document. It describes three collections. One of them is a weblog called “RESTful news,” which accepts a POST request if the representation is an Atom entry document like the one in Example 9-5. 276 | Chapter 9: The Building Blocks of Services

The other two are personal photo albums, which accept a POST request if the representation is an image file. Example 9-6. A representation of a service document that describes three collections Weblogs RESTful News Photo galleries Sam's photos image/* Leonard's photos image/*

How do I know what kind of POST requests a collection will accept? From the accept tags. The accept tag works something like the HTTP Accept header, only in reverse. The Accept header is usually sent by the client with a GET request, to tell the server which representation formats the client understands. The accept tag is the APP server’s way of telling the client which incoming representations a collection will accept as part of a POST request that creates a new member. My two photo gallery collections specify an accept of image/*. Those collections will only accept POST requests where the representation is an image. On the other hand, the RESTful News weblog doesn’t specify an accept tag at all. The APP default is to assume that a collection only accepts POST requests when the representation is an Atom entry document (like the one in Example 9-5). The accept tag defines what the collections are for: the weblog is for textual ?>

278 | Chapter 9: The Building Blocks of Services



The scheme is not fixed, meaning that it’s OK to publish members to the collection even if they belong to categories not listed in this document. This document might be used in an end-user application to show a selectable list of categories for a new “RESTful news” story. As with service documents, the APP defines the representation format for a category document, but says nothing about how category documents are created, modified, or destroyed. It only defines GET on the category document resource. Any other operations (like automatically modifying the category document when someone files an entry under a new category) are up to you to define.

Binary documents as APP members There’s one important wrinkle I’ve glossed over. It has to do with the “photo gallery” collections I described in Example 9-6. I said earlier that a client can create a new member in a photo gallery by POSTing an image file to the collection. But an image file can’t go into an Atom feed: it’s a binary document. What exactly happens when a client POSTs a binary document to an APP collection? What’s in those photo galleries, really? Remember that a resource can have more than one representation. Each photo I upload to a photo collection has two representations. One representation is the binary photo, and the other is an XML document containing meta encoding="utf-8"?> A picture of my guinea pig 2007-01-24T11:52:29Z urn:f1ef2e50-8ec8-0129-b1a7-003065546f18

The actual JPEG I uploaded is at the other end of that link. I can GET it, of course, and I can PUT to it to overwrite it with another image. My POST created a new “member” resource, and my JPEG is a representation of some of its resource state. But there’s also this other representation of resource state: the meta xml:lang="en">

Have you read Great Expectations?



Example 9-8 shows a simple HTML document that contains both sorts of hyperlinks. There are two links that use link to relate the document as a whole to other URIs, and there’s one link that uses a to relate part of the document (the italicized phrase “Great Expectations”) to another URI. The three important attributes of link and a tags are href, rel, and rev. The href attribute is the most important: it gives the URI of the resource that’s being linked to. If you don’t have an href attribute, you don’t have a hyperlink. The rel attribute adds semantics that explain the foreign URI’s relationship to this document. I mentioned this attribute earlier when I was talking about microformats. In Example 9-8, the relationship of the URI atom.xml to this document is “alternate”. The relationship of the URI display.css to this document is “stylesheet”. These particular values for rel are among the 15 defined in the HTML 4 standard. The value “alternate” means that the linked URI is an alternate representation of the resource this document represents. The value “stylesheet” means that the linked URI contains instructions on how to format this document for display. Microformats often define additional values for rel. The rel-nofollow microformat defines the relationship “nofollow”, to show that a document doesn’t trust the resource it’s linking to. The rev attribute is the exact opposite of rel: it explains the relationship of this document to the foreign URI. The VoteLinks microformat lets you express your opinion of a URI by setting rev to “vote-for” or “vote-against”. In this case, the foreign URI probably has no relationship to you, but you have a relationship to it. A simple example illustrates the difference between rel and rev. Here’s an HTML snippet of a user’s home page, which contains two links to his father’s home page.

286 | Chapter 9: The Building Blocks of Services

My father My father

XHTML 4 forms These are the forms that drive the human web. You might not have known about the rel and rev attributes, but if you’ve done any web programming, you should be familiar with the hypermedia capabilities of XHTML forms. To recap what you might already know: HTML forms are described with the form tag. A form tag has a method attribute, which names the HTTP method the client should use when submitting the form. It has an action attribute, which gives the (base) URI of the resource the form is accessing. It also has an enctype attribute, which gives the media type of any representation the client is supposed to send along with the request. A form tag can contain form elements: children like input and select tags. These show up in web browsers as GUI elements: text inputs, checkboxes, buttons, and the like. In application forms, the values entered into the form elements are used to construct the ultimate destination of a GET request. Here’s an application form I just made up: an interface to a search engine.

Since this is an application form, it’s not designed to operate on any particular resource. The point of the form is to use the URI in the action as a jumping-off point to an infinity of resources with user-generated URIs: http://search.example.com/search?q=jellyfish, http://search.example.com/search?q=chocolate, and so on. A resource form in HTML 4 identifies one particular resource, and it specifies an action of POST. The form elements are used to build up a representation to be sent along with the POST request. Here’s a resource form I just made up: an interface to a file upload script.