Long running processes and RESTful APIs.

I have to create a RESTful API that can accept a request for a resource that may take some time to find as it’s the result of a complex deterministic model. The ASP.NET Web API is hosted on an Azure Web Role and exposes a RESTful interface to the clients. The clients are not web browsers, in this case, but other applications.

On the initial GET or “fetch” of a particular resource the very long querystring that contains all of the data the model run requires is handled with a 202 Accepted. Then a 304 Not Modified is returned on each subsequent request for the same resource until finally we have the new resource in our server’s cache, placed there by a background process that monitors a queue that in turn is fed by the model engine. Finally we will return the new resource with a 200 OK. As the result already exists, in a way, and we are just having to take some time to fetch it for the client, then the cache is not being updated with a result but rather it is caching the resource from the application.

In REST the resource has an id. In this case the id is the URI or the ETag (the Entity Tag being the hash of the URI). This is not a “process”, like a POST or “insert” of a resource, nothing in the data is being changed, it’s just a very slow request.

Request Response Ping Ping

The initial request immediately returns a 202 Accepted. This is better than holding the connection open for up to the 4 minutes allowed by the Azure Load Balancer as that would be expensive and we would run the risk of overloading our application.

The 202 response carries with it an ETag (Entity Tag) which is the token that the client can use to make another request to the application. An ETag represents a resource and is intended to be used for caching. We are caching our resource and it’s current state is empty.

The client will then present the ETag in the If-None-Match header value. This is the specified way to check for any changes to a resource. If the state of the resource has not changed then the application will return a 304 Not Modified. If the resource has changed, which in practice means the application has completed its run and dumped the result in the cache, then the application will return the current state of the resource. The ETag is also returned, should the client wish to request the resource again, and a 200 OK to indicate the end of the request.

Note that the above is only example code that has been stripped of some conditional checks and guard clauses for readability.