Overview

Request batching is a useful way of minimizing the number of messages that are passed between the client and the server. This reduces network traffic and provides a smoother, less chatty user interface. This feature will enable Web API users to submit multiple service calls in a single HTTP request.

Design

BatchHandler

This is a custom HttpMessageHandler that will be used to handle the batch requests.The BatchHandler takes two arguments in the constructor: an HttpServer and an IBatchProcessor.

• HttpServer will be used to process the individual batch requests.
• IBatchProcessor will be used to parse the request into individual batch requests and submit them to the HttpServer.

namespace System.Web.Http.Batch
{
    publicclass BatchHandler : HttpMessageHandler
    {
        public BatchHandler(HttpServer httpServer, IBatchProcessor batchProcessor);

        public IBatchProcessor BatchProcessor { get; }

        protectedoverride Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken);
    }
}

IBatchProcessor

This is an abstraction for processing the batch requests.

namespace System.Web.Http.Batch
{
    publicinterface IBatchProcessor
    {
        Task<HttpResponseMessage> ExecuteAsync(HttpRequestMessage request, HttpMessageInvoker invoker, CancellationToken cancellationToken);
    }
}

An implementation of IBatchProcessor will do the following:

• Parse the incoming request into batch requests
• Execute the batch requests
• Build the batch response

The idea is to have different IBatchProcessor implementations that understand different batch request/response formats and know how to process them.

Out of the box, we’re providing the DefaultBatchProcessor for Web API Batching

namespace System.Web.Http.Batch
{
    publicclass DefaultBatchProcessor : IBatchProcessor
    {
        publicbool OrderedExecution { get; set; }

        publicvirtual HttpResponseMessage CreateResponseMessage(IList<HttpResponseMessage> responses, HttpRequestMessage request);
        publicvirtual Task<HttpResponseMessage> ExecuteAsync(HttpRequestMessage request, HttpMessageInvoker invoker, CancellationToken cancellationToken);
        publicvirtual Task<IList<HttpResponseMessage>> ExecuteRequestMessagesAsync(IEnumerable<HttpRequestMessage> requests, HttpMessageInvoker invoker, CancellationToken cancellationToken);
        publicvirtual Task<IList<HttpRequestMessage>> ParseBatchRequests(HttpRequestMessage request);
        protectedvirtualvoid ValidateRequest(HttpRequestMessage request);
    }
}

And ODataBatchProcessor/ODataUnbufferedBatchProcessor for OData Batching.

namespace System.Web.Http.OData.Batch
{
    publicabstractclass ODataBatchProcessorBase : IBatchProcessor
    {
        public Uri BaseUri { get; set; }
        public ODataMessageQuotas MessageQuotas { get; }

        publicvirtual HttpResponseMessage CreateResponseMessage(IEnumerable<ODataResponseItem> responseMessages, HttpRequestMessage request);
        publicabstract Task<HttpResponseMessage> ExecuteAsync(HttpRequestMessage request, HttpMessageInvoker invoker, CancellationToken cancellationToken);
        protectedstatic Uri GetRequestBaseUri(HttpRequestMessage request);
        protectedvirtualvoid ValidateRequest(HttpRequestMessage request);
    }

    publicclass ODataBatchProcessor : ODataBatchProcessorBase
    {
        publicoverride Task<HttpResponseMessage> ExecuteAsync(HttpRequestMessage request, HttpMessageInvoker invoker, CancellationToken cancellationToken);
        publicvirtual Task<IList<ODataResponseItem>> ExecuteRequestMessagesAsync(IEnumerable<ODataRequestItem> requests, HttpMessageInvoker invoker, CancellationToken cancellationToken);
        publicvirtual Task<IList<ODataRequestItem>> ParseBatchRequests(HttpRequestMessage request);
    }

    publicclass ODataUnbufferedBatchProcessor : ODataBatchProcessorBase
    {
        publicoverride Task<HttpResponseMessage> ExecuteAsync(HttpRequestMessage request, HttpMessageInvoker invoker, CancellationToken cancellationToken);
        publicvirtual Task<ODataResponseItem> ExecuteRequestAsync(ODataBatchReader batchReader, Guid batchId, HttpMessageInvoker invoker, CancellationToken cancellationToken);
    }
}

Web API Batching

The DefaultBatchProcessor supports the following formats. It basically wraps the request/response messages in a multipart content.

Request Format

POST http://localhost:8080/api/$batch HTTP/1.1
Content-Type: multipart/mixed; boundary="91731eeb-d443-4aa6-9816-560a8aca66b1"
Host: localhost:8080
Content-Length: 390
Expect: 100-continue
Connection: Keep-Alive

--91731eeb-d443-4aa6-9816-560a8aca66b1
Content-Type: application/http; msgtype=request

POST /api/values HTTP/1.1
Host: localhost:8080
Content-Type: application/json; charset=utf-8

"my value"
--91731eeb-d443-4aa6-9816-560a8aca66b1
Content-Type: application/http; msgtype=request

GET /api/values HTTP/1.1
Host: localhost:8080


--91731eeb-d443-4aa6-9816-560a8aca66b1--

Response Format

HTTP/1.1 200 OK
Content-Length: 333
Content-Type: multipart/mixed; boundary="5b2a806d-4040-43f0-8f04-7d4c86793fa7"
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 08 Apr 2013 19:00:26 GMT

--5b2a806d-4040-43f0-8f04-7d4c86793fa7
Content-Type: application/http; msgtype=response

HTTP/1.1 202 Accepted


--5b2a806d-4040-43f0-8f04-7d4c86793fa7
Content-Type: application/http; msgtype=response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

["my value"]
--5b2a806d-4040-43f0-8f04-7d4c86793fa7--

The request can be built using the HttpClient library.

privatestaticvoid CallApiBatch()
{
    string baseAddress = "http://localhost:8080";
    var client = new HttpClient();

    var batchRequest = new HttpRequestMessage(HttpMethod.Post, baseAddress + "/api/$batch");
    var batchContent = new MultipartContent("mixed");
    batchRequest.Content = batchContent;
    batchContent.Add(new HttpMessageContent(new HttpRequestMessage(HttpMethod.Post, baseAddress + "/api/values")
    {
        Content = new ObjectContent<string>("my value", new JsonMediaTypeFormatter())
    }));
    batchContent.Add(new HttpMessageContent(new HttpRequestMessage(HttpMethod.Get, baseAddress + "/api/values")));
    
    HttpResponseMessage batchResponse = client.SendAsync(batchRequest).Result;

    MultipartStreamProvider streamProvider = batchResponse.Content.ReadAsMultipartAsync().Result;
    foreach (var content in streamProvider.Contents)
    {
        HttpResponseMessage response = content.ReadAsHttpResponseMessageAsync().Result;
    }
}

OData Batching

The ODataBatchProcessor/ODataUnbufferedBatchProcessor supports the following formats which are defined by http://www.odata.org/documentation/odata-v3-documentation/batch-processing/.

Request Format

POST /service/$batch HTTP/1.1 
Host: host 
Content-Type: multipart/mixed; boundary=batch_36522ad7-fc75-4b56-8c71-56071383e77b 

--batch_36522ad7-fc75-4b56-8c71-56071383e77b 
Content-Type: multipart/mixed; boundary=changeset_77162fcd-b8da-41ac-a9f8-9357efbbd621 
Content-Length: ###       

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621) 
Content-Type: application/http 
Content-Transfer-Encoding: binary 
Content-ID: 1 

POST /service/Customers HTTP/1.1 
Host: host  
Content-Type: application/atom+xml;type=entry 
Content-Length: ### 

<AtomPubrepresentationofanewCustomer> 

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621) 
Content-Type: application/http 
Content-Transfer-Encoding: binary 

POST $1/Orders HTTP/1.1 
Host: host 
Content-Type: application/atom+xml;type=entry 
Content-Length: ### 

<AtomPubrepresentationofanewOrder> 

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)-- 
--batch(36522ad7-fc75-4b56-8c71-56071383e77b)--

Response Format

HTTP/1.1 202 Accepted
DataServiceVersion: 1.0
Content-Length: ####
Content-Type: multipart/mixed; boundary=batch(36522ad7-fc75-4b56-8c71-56071383e77b)

--batch(36522ad7-fc75-4b56-8c71-56071383e77b)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 Ok
Content-Type: application/atom+xml;type=entry
Content-Length: ###

<AtomPubrepresentationoftheCustomerentitywithEntityKeyALFKI>

--batch(36522ad7-fc75-4b56-8c71-56071383e77b)
Content-Type: multipart/mixed; boundary=changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)
Content-Length: ###      

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 201 Created
Content-Type: application/atom+xml;type=entry
Location: http://host/service.svc/Customer('POIUY')
Content-Length: ###

<AtomPubrepresentationofanewCustomerentity>

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 204 No Content
Host: host

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)--

--batch(36522ad7-fc75-4b56-8c71-56071383e77b)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 404 Not Found
Content-Type: application/xml
Content-Length: ###

<Errormessage>
--batch(36522ad7-fc75-4b56-8c71-56071383e77b)--

The request can be built using “Add Service Reference” (from WCF Data Services) as well as 3rd party JavaScript libraries such as datajs.

WCF Data Services client

privatestaticvoid CallODataBatch()
{
    string baseAddress = "http://localhost:8080/odata";
    Container container = new Container(new Uri(baseAddress));

    Random rand = new Random();
    int id = rand.Next();
    var customer = new Customer { ID = id, Name = "User"+ id };
    var order = new Order { ID = id, Amount = id + 10 };

    // Batch operation.
    container.AddToCustomers(customer);
    container.AddToOrders(order);
    container.AddLink(customer, "Orders", order);

    var batchResponse = container.SaveChanges(SaveChangesOptions.Batch);

    foreach (var response in batchResponse)
    {
        Console.WriteLine(response.StatusCode);
        Console.WriteLine(response.Headers);
    }
}

datajs client

OData.request({
    requestUri: "/odata/$batch",
    method: "POST",
    data: {
        __batchRequests: [
            { __changeRequests: [
                { requestUri: "/odata/Customers", method: "POST", data: customer }
            ] },
            { requestUri: "/odata/Customers", method: "GET" }
        ]
    }
}, function (data, response) {
    //success handler
}, function () {
    alert("request failed");
}, OData.batchHandler);

Scenarios

Web API Batching

Registering default batch endpoint

You can use MapBatchRoute, which is an HttpRouteCollection extension method, to create a batch endpoint.

config.Routes.MapBatchRoute("apiBatch", "api/batch", GlobalConfiguration.DefaultServer);

Under the hood it just uses MapHttpRoute.

config.Routes.MapHttpRoute("apiBatch", "api/batch", null, null, new BatchHandler(GlobalConfiguration.DefaultServer, new DefaultBatchProcessor()));

Ordered Execution

By default each individual batch request is executed asynchronously. Meaning there’s no guarantee that the first batch request will finish executing before kicking off the second one. So if you have a scenario where you want to get the results after all the posts, you can enable the OrderedExecution which will process the requests in order.

var batchProcessor = new DefaultBatchProcessor();
batchProcessor.OrderedExecution = true;
config.Routes.MapBatchRoute("apiBatch", "api/batch", new BatchHandler(GlobalConfiguration.DefaultServer, batchProcessor));

OData Batching

Registering OData batch endpoint

You can use MapODataBatchRoute, which is an HttpRouteCollection extension method, to create an OData batch endpoint.

config.Routes.MapODataBatchRoute("odataBatch", "odata/$batch", GlobalConfiguration.DefaultServer);

Under the hood it just uses MapHttpRoute.

config.Routes.MapHttpRoute("odataBatch", "odata/$batch", null, null, new BatchHandler(GlobalConfiguration.DefaultServer, new ODataBatchProcessor()));

Configuring Batch Quotas

Work in progress…

var odataBatchProcessor = new ODataBatchProcessor();
odataBatchProcessor.MessageQuotas.MaxOperationsPerChangeset = 10;
odataBatchProcessor.MessageQuotas.MaxPartsPerBatch = 50;
config.Routes.MapBatchRoute("odataBatch", "odata/$batch", new BatchHandler(GlobalConfiguration.DefaultServer, odataBatchProcessor));

Order of Execution

According to the OData spec, the operation/ChangeSet within a batch request is executed in ordered manner. Operations within the ChangeSet can be executed regardless of the order but our implementation will execute them in order for simplicity.

Transaction

Work in progress…

Custom Batching

Work in progress…