I'm struggling to determine how to design restful URLs. I'm all for the restful approach of using URLs with nouns and not verbs don't understand how to do this.
We are creating a service to implement a financial calculator. The calculator takes a bunch of parameters that we will upload via a CSV file. The use cases would involve:
Upload new parameters Get the latest parameters Get parameters for a given business date Make a set of parameters active Validate a set of parameters
I gather the restful approach would be to have the following type URLs:
/parameters
/parameters/12-23-2009
You could achieve the first three use cases with:
POST where you include the parameter file in the post request GET of first URL GET of second URL
But how do you do the 4th and 5th use case without a verb? Wouldn't you need URLs like:
/parameters/ID/activate
/parameters/ID/validate
??
General principles for good URI design:
Don't use query parameters to alter state
Don't use mixed-case paths if you can help it; lowercase is best
Don't use implementation-specific extensions in your URIs (.php, .py, .pl, etc.)
Don't fall into RPC with your URIs
Do limit your URI space as much as possible
Do keep path segments short
Do prefer either /resource or /resource/; create 301 redirects from the one you don't use
Do use query parameters for sub-selection of a resource; i.e. pagination, search queries
Do move stuff out of the URI that should be in an HTTP header or a body
(Note: I did not say "RESTful URI design"; URIs are essentially opaque in REST.)
General principles for HTTP method choice:
Don't ever use GET to alter state; this is a great way to have the Googlebot ruin your day
Don't use PUT unless you are updating an entire resource
Don't use PUT unless you can also legitimately do a GET on the same URI
Don't use POST to retrieve information that is long-lived or that might be reasonable to cache
Don't perform an operation that is not idempotent with PUT
Do use GET for as much as possible
Do use POST in preference to PUT when in doubt
Do use POST whenever you have to do something that feels RPC-like
Do use PUT for classes of resources that are larger or hierarchical
Do use DELETE in preference to POST to remove resources
Do use GET for things like calculations, unless your input is large, in which case use POST
General principles of web service design with HTTP:
Don't put metadata in the body of a response that should be in a header
Don't put metadata in a separate resource unless including it would create significant overhead
Do use the appropriate status code
201 Created after creating a resource; resource must exist at the time the response is sent
202 Accepted after performing an operation successfully or creating a resource asynchronously
400 Bad Request when someone does an operation on data that's clearly bogus; for your application this could be a validation error; generally reserve 500 for uncaught exceptions
401 Unauthorized when someone accesses your API either without supplying a necessary Authorization header or when the credentials within the Authorization are invalid; don't use this response code if you aren't expecting credentials via an Authorization header.
403 Forbidden when someone accesses your API in a way that might be malicious or if they aren't authorized
405 Method Not Allowed when someone uses POST when they should have used PUT, etc
413 Request Entity Too Large when someone attempts to send you an unacceptably large file
418 I'm a teapot when attempting to brew coffee with a teapot
Do use caching headers whenever you can
ETag headers are good when you can easily reduce a resource to a hash value
Last-Modified should indicate to you that keeping around a timestamp of when resources are updated is a good idea
Cache-Control and Expires should be given sensible values
Do everything you can to honor caching headers in a request (If-None-Modified, If-Modified-Since)
Do use redirects when they make sense, but these should be rare for a web service
With regard to your specific question, POST should be used for #4 and #5. These operations fall under the "RPC-like" guideline above. For #5, remember that POST does not necessarily have to use Content-Type: application/x-www-form-urlencoded
. This could just as easily be a JSON or CSV payload.
Perhaps something like:
PUT /parameters/activation HTTP/1.1
Content-Type: application/json; encoding=UTF-8
Content-Length: 18
{ "active": true }
POST
vs PUT
is not exactly like insert
vs update
. PUT
updates the resource corresponding to the given path, or creates a new resource corresponding to the given path. POST
creates a new resource somewhere. For example, PUT /blog/posts/3/comments/5
will update the appropriate comment, while POST /blog/posts/3/comments
will create a new comment
resource (and should return the path to the new resource in the response).
PUT
is idempotent while POST
is not. Usually you should put as much constraints on what you provide as the result as possible. Sticking with PUT
gives more information to the client of the service.
Whenever it looks like you need a new verb, think about turning that verb into a noun instead. For example, turn 'activate' into 'activation', and 'validate' into 'validation'.
But just from what you've written I'd say your application has much bigger problems.
Any time a resource called 'parameter' is proposed, it should send up red flags in every project team member's mind. 'parameter' can literally apply to any resource; it's not specific enough.
What exactly does a 'parameter' represent? Probably a number of different things, each of which should have a separate resource dedicated to it.
Another way to get at this - when you discuss your application with end users (those who presumably know little about programming) what are the words they themselves use repeatedly?
Those are the words you should be designing your application around.
If you haven't yet had this conversion with prospective users - stop everything right now and don't write another line of code until you do! Only then will your team have an idea of what needs to be built.
I know nothing about financial software, but if I had to guess, I'd say some of the resources might go by names such as "Report", "Payment", "Transfer", and "Currency".
There are a number of good books on this part of the software design process. Two I can recommend are Domain Driven Design and Analysis Patterns.
The design of your URLs has nothing to do with whether your application is RESTful or not. The phrase "RESTful URLs" is therefore nonsense.
I think you should do some more reading on what REST actually is. REST treats the URLS as opaque, and as such doesn't know what's in them, whether there are verbs or nouns or whatever. You might still want to design your URLs, but that's about UI, not REST.
That said, let's get to your question: The last two cases are not RESTful and don't fit into any kind of restful scheme. Those are what you might call RPC. If you're serious about REST, you'll have to rethink how your application works from the ground up. Either that or abandon REST and just do your app as an RPC app.
Hrmmm maybe not.
The idea here is that you have to treat everything as a resource, so once a set of parameters has a URL you can refer to it from, you just add:
GET [parametersurl]/validationresults
POST [paramatersurl]
body: {command:"activate"}
But again, that activate thing is RPC, not REST.
The activate and validate requirements are situations where you are attempting to change the state of a resource. It is no different that making an order "completed", or some other request "submitted". There are numerous ways to model these kinds of state change but one that I find that often works is to create collection resources for resources of the same state and then to move the resource between the collections to affect the state.
e.g. Create some resources such as,
/ActiveParameters
/ValidatedParameters
If you want to make a set of parameters active, then add that set to the ActiveParameters collection. You could either pass the set of parameters as an entity body, or you could pass an url as a query parameter, as follows:
POST /ActiveParameters?parameter=/Parameters/{Id}
The same thing can be done with the /ValidatedParameters. If the Parameters are not valid then the server can return "Bad Request" to the request to add the parameters to collection of validated parameters.
I would suggest the following Meta resource and methods.
Make parameters active and/or validate them:
> PUT /parameters/<id>/meta HTTP/1.1
> Host: example.com
> Content-Type: application/json
> Connection: close
>
> {'active': true, 'require-valid': true}
>
< HTTP/1.1 200 OK
< Connection: close
<
Check if the parameters are active and valid:
> GET /parameters/<id>/meta HTTP/1.1
> Host: example.com
> Connection: close
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Connection: close
<
< {
< 'active': true,
< 'require-valid': true,
< 'valid': {'status': false, 'reason': '...'}
< }
<
I feel a bit sad to see that after more than 10 years there is no answer really stating how such a thing as requested in the OP could be designed in a REST architecture, hence I feel the need to do this now.
First things first, what is REST?! The acronym REST or ReST stands for "Representational State Transfer" and defines the exchange of a resource's state in a certain representation format. The representation format is tide to the negotiated media type. In the case of application/html
the representation format may be a stream of HTML formatted text content that is rendered in the browser, probably after applying some stylesheet formatting to position certain elements at certain locations.
REST is in principle a generalization of the browsable Web we all know, though targets all kinds of applications and not only browsers. Therefore, by design, the same concepts that apply to the Web also apply to a REST architecture. A question like how to achieve something in a "RESTful" way resolves around answering the question how to achieve something on a Web page and then apply the same concepts onto the application layer.
A Web based calculator may usually start off with some "page" that allows you to input some values to calculate before sending the entered data to the server. In HTML this is usually achieved via HTML <form>
elements that teaches a client on the available parameters to set, the target location to send the request to as well as the representation format to apply upon sending the input data. This can i.e. look like this:
<html>
<head>
...
</head>
<body>
<form action="/../someResource" method="post" enctype="application/x-www-form-urlencoded">
<label for="firstNumber">First number:</label>
<input type="number" id="firstNumber" name="firstNumber"/>
<label for="secondNumber">Second number:</label>
<input type="number" id="secondNumber" name="secondNumber"/>
<input type="submit" value="Add numbers"/>
</form>
</body>
</html>
The sample above i.e. states that there are two input fields that can be filled out either by the user or by some other automata, and that upon invoking the submit input element the browser takes care of formatting the input data into a application/x-www-form-urlencoded
representation format that is sent to the mentioned target location via the specified HTTP request method, POST
in this case. If we enter 1
into the firstNumber
input field and 2
into the secondNumber
input field, the browser will generate a representation of firstNumber=1&secondNumber=2
and send this as the body payload of the actual request to the target resource.
The raw HTTP request issued to the server therefore may look like this:
POST /../someResource
Host: www.acme.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 28
Accept: application/html
firstNumber=1&secondNumber=2
The server may perform the calculation and respond with a further HTML page that contains the result of the calculation, as the request indicated that the client understands this format.
As Breton pointed already out there is no such thing as a "RESTful" URL or URI. A URI/URL is its own kind of thing and should not convey any meaning to a client/user. In the calculator sample above a user simply isn't interested where to send the data to it is just interested in that upon triggering the submit input field the request is sent. All the required information needed to perform the task should already be given by the server.
A browser also might not be aware of that the request is actually feeding a calculator with some input parameters, it could as well be some kind of an order form that returns just the next form representation to continue the ordering process or some totally different kind of resource. It simply performs what the HTML spec demands in such a case and it couldn't care less what the server is actually doing. This concept enables a browser to use the same representation format to do all kind of things such as ordering some stuff from your preferred online shop, chatting with your best friends, signing into an online account and so on.
The affordance of certain elements, such in the submit input field case that is usually rendered as button, defines what you should to with it. In the case of a button or a link it basically tells you to click it. Other elements may convey different affordances. Such an affordance can also be expressed via link-relations as i.e. with preload
annotated links that basically tell a client that it can already load the content of the linked resource in the background as the user will most likely grab this content next. Such link relations should of course be standardized or follow the extension mechanism for relation types as defined by Web linking.
These are the fundamental concept that are used on the Web and that should also be used in a REST architecture. According to "Uncle Bob" Robert C. Martin an architecture is about intent and the intention behind the REST architecture is the decoupling of clients from servers to allow servers to evolve freely in future without having to fear them breaking clients. This unfortunately requires a lot of discipline as it is so easy to introduce coupling or to add quick-fix solutions to get the job done and move on. As Jim Webber pointed out in a REST architecture you, as a service provider, should attempt to design an domain application protocol similar to a text based computer game of the 70s that clients will follow through until they reached the end of a process.
What plenty of so-called "REST" APIs unfortunately do in reality is everything but that. You see the exchange of mostly JSON based data that is specified in an API specific external documentation that is usually hard to dynamically integrate on the fly. The format how a request needs to look like are also hardcoded into the external documentation which lead to plenty of implementation interpreting URIs to return predefined typs instead of using some common representation format that is negotiated upfront. This prevents servers from changing as clients now expect to receive a certain data format (note not representation format!) for predefined URIs. This custom data format exchange furthermore prevents clients from interacting with other APIs as the "data format" is usually tide to a specific API. We know this concept from the past from RPC technologies such as Corba, RMI or SOAP which we condemn as somehow evil, even though Peppol moved to it again by replacing AS2 with AS4 as default transfer protocol as of recently.
In regards to the actual question asked, sending data as csv file is nothing different than using application/x-www-form-urlencoded
representation or similar stuff. Jim Webber made it clear that after all HTTP is just a transport protocol whose application domain is the transfer of documents over the Web. Client and server should at least both support text/csv
as defined in RFC 7111. This CSV file could be generated as a consequence of processing a media type that defines form elements, a target element or attribute to send the request to as well as the HTTP method to perform the upload of the configuration.
There are a couple of media types that support forms such as HTML, HAL Forms, halform, ion or Hydra. I am currently, though, not aware of a media type that automatically can encode the input data into text/csv
directly hence one might need to be defined and registered with IANA's media type registry.
The upload and download of the complete parameter set shouldn't be an issue I guess. As mentioned before, the target URI is not of relevance as a client will just use the URI to retrieve new content to process. Filtering by business date should also not be to difficult. Here the server should however the client with all the possibilities the client simply can chose from. In recent years GraphQL and RestQL evolved which introduce an SQL like language that can be targeted at a certain endpoint to get a filtered response. However, in a true REST sense this violates the idea behind REST as a) GraphQL i.e. only uses a single endpoint which somehow prevents optimal usage of caching and b) requires the knowledge of available fields upfrong, which may lead to introducing a coupling of clients to the base data model of the resource.
Activating or deactivating certain configuration parameters is simply a matter of triggering the hypermedia controls that provide this affordance. In HTML forms this could be a simple checkbox or a multi-line selection in a list or that kind. Depending on the form and what method it defines it could then potentially send the whole configuration via PUT
or be smart about the changes done and only perform a partial update via PATCH
. The latter one requires basically a calculaton of the change representation to the one updated and feed the server with the required steps to tranform the current representation into the desired one. According to the PATH specification this has to be done within a transaction so that either all or none of the steps are applied.
HTTP allows and encourages a server to validate a received request upfront before applying the changes. For PUT the spec states:
An origin server SHOULD verify that the PUT representation is consistent with any constraints the server has for the target resource that cannot or will not be changed by the PUT. This is particularly important when the origin server uses internal configuration information related to the URI in order to set the values for representation metadata on GET responses. When a PUT representation is inconsistent with the target resource, the origin server SHOULD either make them consistent, by transforming the representation or changing the resource configuration, or respond with an appropriate error message containing sufficient information to explain why the representation is unsuitable. The 409 (Conflict) or 415 (Unsupported Media Type) status codes are suggested, with the latter being specific to constraints on Content-Type values. For example, if the target resource is configured to always have a Content-Type of "text/html" and the representation being PUT has a Content-Type of "image/jpeg", the origin server ought to do one of: a. reconfigure the target resource to reflect the new media type; b. transform the PUT representation to a format consistent with that of the resource before saving it as the new resource state; or, c. reject the request with a 415 (Unsupported Media Type) response indicating that the target resource is limited to "text/html", perhaps including a link to a different resource that would be a suitable target for the new representation. HTTP does not define exactly how a PUT method affects the state of an origin server beyond what can be expressed by the intent of the user agent request and the semantics of the origin server response. ...
To sum this post up, you should either use an existing media type that allows you to teach a client about the required or supported input parameters, the target location to send the request to, the operation to use as well as the media-type the request has to be formatted in, or define your own one that you register with IANA. The latter might be necessary if you want to convert the input to text/csv
and then upload the CSV representation to the server. The validation should occur before the changes are applied to the resource. The actual URI should not be of relevance to clients other than to determine where to send the request to and as such can be freely chosen by you, the service implementor. By following these steps you pretty much gain the freedom to change your server side at any time and clients will not break as a consequence if they support the used media-types.
Edit: Indeed the URI would have prevented GET
requests from remaining idempotent.
For the validation however, the use of HTTP status codes to notify the validity of a request (to create a new or modify an existing 'parameter') would fit a Restful model.
Report back with a 400 Bad Request
status code if the data submitted is/are invalid and the request must be altered before being resubmitted (HTTP/1.1 Status Codes).
This relies on validating at submission time though, rather than deferring it as in your use-case. The other answers have suitable solutions to that scenario.
In a REST environment, each URL is a unique resource. What are your resources? A financial calculator really doesn't have any obvious resources. You need to dig into what you are calling parameters and pull out the resources. For example, an amortization calendar for a loan might be a resource. The URL for the calendar might include start date, term (in months or years), period (when interest is compounded), interest rate, and initial principle. With all those values, you have a specific calendar of payments:
http://example.com/amort_cal/2009-10-20/30yrsfixed/monthly/5.00/200000
Now, I don't know what you are calculating, but your concept of a parameter list doesn't sound RESTful. As someone else said, your requirements above sound more XMLRPC. If you are trying for REST, you need nouns. Calculations are not nouns, they are verbs that act on nouns. You need to turn it around to pull the nouns out of your calcs.
Success story sharing