Chapter 4. AJAX and Server Communication

The common thread of this chapter is server-side communications. Performing asynchronous requests, using the IFRAME transport to submit forms behind the scenes, serializing to and from JavaScript Object Notation (JSON), and using JSONP (JSON with Padding) are a few of the topics that are introduced in this chapter. You'll also learn about Deferred, a class that forms the lynchpin in the toolkit's IO subsystem by providing a uniform interface for handling asynchronous activity.

Quick Overview of AJAX

AJAX[12] (Asynchronous JavaScript and XML) has stirred up considerable buzz and revitalized web design in a refreshing way. Whereas web pages once had to be completely reloaded via a synchronous request to the server to perform a significant update, JavaScript's XMLHttpRequest object allows them to now behave much like traditional desktop applications. XHR is an abbreviation for the XMLHttpRequest object and generally refers to any operation provided the object.

Web pages may now fetch content from the server via an asynchronous request behind the scenes, as shown in Figure 4-1, and a callback function can process it once it arrives. (The image in Figure 4-1 is based on http://adaptivepath.com/ideas/essays/archives/000385.php.) Although a simple concept, this approach has revolutionized the user experience and birthed a new era of Rich Internet Applications.

The difference between synchronous and asynchronous communication for a web application
Figure 4-1. The difference between synchronous and asynchronous communication for a web application

Using JavaScript's XMLHttpRequest object directly isn't exactly rocket science, but like anything else, there are often tricky implementation details involved and boilerplate that must be written in order to cover the common-use cases. For example, asynchronous requests are never guaranteed to return a value (even though they almost always do), so you'll generally need to implement logic that determines when and how to timeout a request; you may want to have some facilities for automatically vetting and transforming JSON strings into JavaScript objects; you'll probably want to have a concise way of separating the logic that handles a successful request versus a request that produces an error; and so forth.

JSON

JSON bears a brief mention before we move on to a discussion of AJAX because it has all but become the universally accepted norm for lightweight data exchange in AJAX applications. You can read about the formalities of JSON at http://json.org, but basically, JSON is nothing more than a string-based representation of JavaScript objects. Base provides two simple functions for converting String values and JavaScript objects back and forth. These functions handle the mundane details of escaping special characters like tabs and new lines, and even allow you to pretty-print if you feel so inclined:

dojo.fromJson(/*String*/ json) //Returns Object
dojo.toJson(/*Object*/ json, /*Boolean?*/ prettyPrint) //Returns String

By default, a tab is used to indent the JSON string if it is pretty-printed. You can change the tab to whatever you'd like by switching the value of the built-in attribute dojo.toJsonIndentStr.

Here's a quick example that illustrates the process of converting an Object to a JSON string that is suitable for human consumption:

var o = {a:1, b:2, c:3, d:4};
dojo.toJson(o, true); //pretty print
/* produces ...
'{
  "a": 1,
  "b": 2,
  "c":3,
  "d":4
}'

AJAX Made Easy

Base provides a small suite of functions suitable for use in a RESTful design that significantly simplifies the process of performing routine AJAX operations. Each of these functions provides explicit mechanisms that eliminate virtually all of the boilerplate you'd normally find yourself writing. Table 4-1 summarizes the property values for args.

Table 4-1. Property values for args

Name

Type (Default)

Comment

url

String

("")

The base URL to direct the request.

content

Object

({})

Contains key/value pairs that are encoded in the most appropriate way for the particular transport being used. For example, they are serialized and appended onto the query string as name1=value2 for a GET request but are included as hidden form fields for the case of an IFRAME transport. Note that even though HTTP allows more than one field with the same name (multivalued fields), this is not possible to achieve via the content property because it is a hash.

timeout

Integer

(Infinity)

The number of milliseconds to wait for the response. If this time passes, then the error callback is executed. Only valid when sync is false.

form

DOMNode | String

The DOM node or id for a form that supplies the key/value pairs that are serialized and provide the query string for the request. (Each form value should have a name attribute that identifies it.)

preventCache

Boolean

(false)

If true, then a special dojo.preventCache parameter is sent in the request with a value that changes with each request (timestamp). Useful only with GET-type requests.

handleAs

String

("text")

Designates the type of the response data that is passed into the load handler. Acceptable values depend on the type of IO transport: "text", "json", "javascript", and "xml".

load

Function

The load function will be called on a successful response and should have the signature function(response, ioArgs) {/*...*/}.

error

Function

The error function will be called in an error case and should have the signature function(response, ioArgs) {/*...*/}.

handle

Function

A function that stands in for both load and error, and thus should be called regardless of whether the request is successful.

sync

Boolean

(false)

Whether to perform a synchronous request.

headers

Object

({})

Additional HTTP headers to include in the request.

postData

String

("")

Raw data to send in the body of a POST request. Only valid for use with rawXhrPost.

putData

String

("")

Raw data to send in the body of a PUT request. Only valid for use with rawXhrPut.

The RESTful XHR functions offered by the toolkit follow; as of Dojo version 1.1, each of these functions sets the X-Requested-With: XMLHttpRequest header to the server automatically. A discussion of the args parameter follows.

All of the XHR functions return a special Object called Deferred, which you'll learn more about in the next section. For now, just concentrate on the discussion at hand.

dojo.xhrGet(/*Object*/args)

Performs an XHR GET request.

dojo.xhrPost(/*Object*/args)

Performs an XHR POST request.

dojo.rawXhrPost(/*Object*/args)

Performs an XHR POST request and allows you to provide the raw data that should be included as the body of the POST.

dojo.xhrPut(/*Object*/args)

Performs an XHR PUT request.

dojo.rawXhrPut(/*Object*/args)

Performs an XHR PUT request and allows you to provide the raw data that should be included as the body of the PUT.

dojo.xhrDelete(/*Object*/args)

Performs an XHR DELETE request.

dojo.xhr(/*String*/ method, /*Object*/ args, /*Boolean?*/ hasBody)

A general purpose XHR function that allows you to define any arbitrary HTTP method to perform asynchronsously.

Although most of the items in the table are pretty straightforward, the arguments that are passed into the load and error functions bear mentioning. The first parameter, response, is what the server returns, and the value for handleAs specifies how the response should be interpreted. Although the default value is "text", specifying "json", for example, results in the response being cast into a JavaScript object so that the response value may be treated as such.

In the load and error functions, you should always return the response value. As you'll learn later in this chapter, all of the various input/output calls such as the XHR facilities return a type called a Deferred, and returning responses so that callbacks and error handlers can be chained together is an important aspect of interacting with Deferreds.

The second parameter, ioArgs, contains some information about the final arguments that were passed to the server in making the request. Although you may not need to use ioArgs very frequently, you may occasionally find it useful—especially in debugging situations. Table 4-2 describes the values you might see in ioArgs.

Table 4-2. Property values for ioArgs

Name

Type

Comment

args

Object

The original argument to the IO call.

xhr

XMLHttpRequest

The actual XMLHttpRequest object that was used for the request.

url

String

The final URL used for the call; often different than the one provided because it is fitted with query parameters, etc.

query

String

Defined only for non-GET requests, this value provides the query string parameters that were passed with the request.

handleAs

String

How the response should be interpreted.

XHR Examples

At an absolute minimum, the arguments for an XHR request should include the URL to retrieve along with the load function; however, it's usually a very good idea to include an error handler, so don't omit it unless there you're really sure you can't possibly need it. Here's an example:

//...snip...
dojo.addOnLoad(function(  ) {
    dojo.xhrGet({

        url : "someText.html",  //the relative URL

        // Run this function if the request is successful
        load : function(response, ioArgs) {
            console.log("successful xhrGet", response, ioArgs);

            //Set some element's content...
            dojo.byId("foo").innerHTML= response;

            return response; //always return the response back
        },

        // Run this function if the request is not successful
        error : function(response, ioArgs) {
            console.log("failed xhrGet", response, ioArgs);

            /* handle the error... */

            return response; //always return the response back
        }
    });
});
//...snip...

You may not necessarily want plain text back; you may want to time out the request after some duration, and you might want to pass in some additional information a query string. Fortunately, life doesn't get any harder. Just add some parameters, like so:

dojo.xhrGet({ 
    url : "someJSON.html", //Something like: {'bar':'baz'}

    handleAs : "json", //Convert to a JavaScript object 

    timeout: 5000, //Call the error handler if nothing after 5 seconds 

    content: {foo:'bar'}, //Append foo=bar to the query string 

    // Run this function if the request is successful 
    load : function(response, ioArgs) { 
        console.log("successful xhrGet", request, ioArgs); 
        console.log(response); 

        //Our handleAs value tells Dojo to 
        //convert the data to an object 

        dojo.byId("foo").innerHTML= response.bar; 
        //Display now updated to say 'baz' 

        return response; //always return the response back 
    }, 

    // Run this function if the request is not successful 
    error : function(response, ioArgs) { 
        console.log("failed xhrGet"); 
        return response; //always return the response back 
    } 
});

Do note that not specifying a proper value for handleAs can produce frustrating bugs that may not be immediately apparent. For example, if you were to mistakenly omit the handleAs parameter, but try to access the response value as a JavaScript object in your load function, you'd most certainly get a nasty error that might lead you to look in a lot of other places before realizing that you are trying to treat a String as an Object—which may not be immediately obvious because logs may display the values nearly identically.

Although applications tend to perform a lot of GET requests, you are bound to come across a circumstance when you'll need to PUT, POST, or DELETE something. The process is exactly the same with the minor caveats that you'll need to include a putData or postData argument for rawXhrPut and rawXhrPost requests, respectively, as a means of providing the data that should be sent to the server. Here's an example of a rawXhrPost:

dojo.rawXhrPost({
    url : "/place/to/post/some/raw/data",
    postData : "{foo : 'bar'}", //a JSON literal
    handleAs : "json",

    load : function(response, ioArgs) {
        /* Something interesting  happens here */
        return response;
    },

    error : function(response, ioArgs) {
        /* Better handle that error */
        return response;
    }
});

General Purpose XMLHttpRequest Calls

Dojo version 1.1 introduced a more general-purpose dojo.xhr function with the following signature:

dojo.xhr(/*String*/ method, /*Object*/ args, /*Boolean?*/ hasBody)

As it turns out, each of the XHR functions from this chapter are actually wrappers around this function. For example, dojo.xhrGet is really just the following wrapper:

dojo.xhrGet  = function(args) {
    return dojo.xhr("GET", args); //Always provide the method name in all caps!
}

Although you'll generally want to use the shortcuts presented in this section, the more general-purpose dojo.xhr function can be useful for some situations in which you need to programmatically configure XHR requests or for times when a wrapper isn't available. For example, to perform a HEAD request for which there isn't a wrapper, you could do the following:

dojo.xhr("HEAD", {
    url : "/foo/bar/baz",
    load : function(response, ioArgs) { /*...*/},
    error : function(response, ioArgs) { /*...*/}
});

Hitching Up Callbacks

Chapter 2 introduced hitch, a function that can be used to guarantee that functions are executed in context. One common place to use hitch is in conjunction with XHR callback functions because the context of the callback function is different from the context of the block that executed the callback function. The following block of code demonstrates the need for hitch by illustrating a common pattern, which aliases this to work around the issue of context in the callback:

//Suppose you have the following addOnLoad block, which could actually be any
//JavaScript Object
dojo.addOnLoad(function(  ) {

        //foo is bound the context of this anonymous function
        this.foo = "bar";

        //alias "this" so that it can be referenced inside of the load callback...
        var self=this;
        dojo.xhrGet({
            url : "./data",
            load : function(response, ioArgs) {
                //you must have aliased "this" to reference foo inside of here...
                console.log(self.foo, response);
            },
            error : function(response, ioArgs) {
                console.log("error", response, ioArgs);
            }
        });

});

While it may not look very confusing for this short example, it can get a bit messy to repeatedly alias this to another value that can be referenced. The next time you encounter the need to alias this, consider the following pattern that makes use of hitch :

dojo.addOnLoad(function(  ) {

        //foo is in the context of this anonymous function
        this.foo = "bar";

        //hitch a callback function to the current context so that foo
        //can be referenced
        var callback = dojo.hitch(this, function(response, ioArgs) {
            console.log("foo (in context) is", this.foo);
            //and you still have response and ioArgs at your disposal...
        });

        dojo.xhrGet({
            url : "./data",
            load : callback,
            error : function(response, ioArgs) {
                console.log("error", response, ioArgs);
            }
        });

});

And don't forget that hitch accepts arguments, so you could just as easily have passed in some parameters that would have been available in the callback, like so:

dojo.addOnLoad(function(  ) {

        //foo is in the context of this anonymous function
        this.foo = "bar";

        //hitch a callback function to the current context so that foo can be
        //referenced
        var callback = dojo.hitch(
            this,
            function(extraParam1, extraParam2, response, ioArgs) {
                console.log("foo (in context) is", this.foo);
                //and you still have response and ioArgs at your disposal...
            },
            "extra", "params"
         );

        dojo.xhrGet({
            url : "./data",
            load : callback,
            error : function(response, ioArgs) {
                console.log("error", response, ioArgs);
            }
        });

});

If you may have a variable number of extra parameters, you can instead opt to use arguments, remembering that the final two values will be response and ioArgs.

Deferreds

JavaScript doesn't currently support the concept of threads, but it does offer the ability to perform asynchronous requests via the XMLHttpRequest object and through delays with the setTimeout function. However, it doesn't take too many asynchronous calls running around before matters get awfully confusing. Base provides a class called Deferred to help manage the complexity often associated with the tedious implementation details of asynchronous events. Like other abstractions, Deferred s allow you to hide away tricky logic and/or boilerplate into a nice, consistent interface.

If the value of a Deferred was described in one sentence, however, it would probably be that it enables you to treat all network I/O uniformly regardless of whether it is synchronous or asynchronous. Even if a Deferred is in flight, has failed, or finished successfully, the process for chaining callbacks and errbacks is the exact same. As you can imagine, this behavior significantly simplifies bookkeeping.

Dojo's implementation of a Deferred is minimally adapted from MochiKit's implementation, which in turn is inspired from Twisted's implementation of the same. Some good background on MochiKit's implementation is available at http://www.mochikit.com/doc/html/MochiKit/Async.html#fn-deferred. Twisted's implementation of Deferred s is available at http://twistedmatrix.com/projects/core/documentation/howto/defer.html.

Some key features of Deferred s are that they allow you to chain together multiple callbacks and errbacks (error-handling routines) so they execute in a predictable sequential order, and Deferred s also allow you to provide a canceling routine that you can use to cleanly abort asynchronous requests. You may not have realized it at the time, but all of those XHR functions you were introduced to earlier in the chapter were returning Deferreds, although we didn't have an immediate need to dive into that just then. In fact, all of the network input/output machinery in the toolkit use and return Deferred s because of the flexibility they offer in managing the asynchronous activity that results from network calls.

Before revisiting some of our earlier XHR efforts, take a look at the following abstract example that directly exposes a Deferred, which forms the basis for some of the concepts that are coming up:

//Create a Deferred
var d = new dojo.Deferred(/* Optional cancellation function goes here */);

//Add a callback
d.addCallback(function(response) {
    console.log("The answer is", response);
    return response;
});

//Add another callback to be fired after the previous one
d.addCallback(function(response) {
    console.log("Yes, indeed. The answer is", response);
    return response;
});

//Add an errback just in case something goes wrong
d.addErrback(function(response) {
    console.log("An error occurred", response);
    return response;
});

//Could add more callbacks/errbacks as needed...

/* Lots of calculations happen */

//Somewhere along the way, the callback chain gets started
d.callback(46);

If you run the example in Firebug, you'd see the following output:

The answer is 46
Yes, indeed. The answer is 46

Before jumping into some more involved examples, you'll probably want to see the API that a Deferred exposes (Table 4-3).

Table 4-3. Deferred functions and properties

Name

Return type

Comment

addCallback(/*Function*/handler)

Deferred

Adds a callback function to the callback chain for successes.

addErrback(/*Function*/handler)

Deferred

Adds a callback function to the callback chain for errors.

addBoth(/*Function|Object*/ context, /*String?*/name)

Deferred

Adds a callback function that acts as both the callback for successes and errors. Useful for adding code that you want to guarantee will run one way or another.

addCallbacks(/*Function*/callback, /*Function*/errback)

Deferred

Allows you to add a callback and an errback at the same time.

callback(/*Any*/value)

N/A

Executes the callback chain.

errback(/*Any*/value)

N/A

Executes the errback chain.

cancel( )

N/A

Cancel the request and execute the cancellation function provided to the constructor, if provided.

Be aware that a Deferred may be in an error state based on one or more combinations of three distinct possibilities:

  • A callback or errback is passed a parameter that is an Error object.

  • A callback or errback raises an exception.

  • A callback or errback returns a value that is an Error object.

Typical use cases normally do not involve the canceller, silentlyCancelled, and fired properties of a Deferred, which provide a reference to the cancellation function, a means of determining if the Deferred was cancelled but there was no canceller method registered, and a means of determining if the Deferred status of the fired, respectively. Values for fired include:

−1: No value yet (initial condition)

0: Successful execution of the callback chain

1: An error occurred

Deferred Examples Via CherryPy

Let's get warmed up with a simple routine on the server that briefly pauses and then serves some content. (The pause is just a way of emphasizing the notion of asynchronous behavior.)

The complete CherryPy file that provides this functionality follows:

import cherrypy
from time import sleep
import os

# a foo.html file will contain our Dojo code performing the XHR request
# and that's all the following config directive is doing

current_dir = os.getcwd()
config = {'/foo.html' :
    {
    'tools.staticfile.on' : True,
    'tools.staticfile.filename' : os.path.join(current_dir, 'foo.html')
    }
}


class Content:

    # this is what actually serves up the content
    @cherrypy.expose
    def index(self):
        sleep(3) # purposefully add a 3 sec delay before responding
        return "Hello"

# start up the web server and have it listen on 8080
cherrypy.quickstart(Content(  ), '/', config=config)

Assuming that the CherryPy content is saved in a file called hello.py, you'd simply type python hello.py in a terminal to startup the server. You should be able to verify that if you navigate to http://127.0.0.1:8080/ that "Hello" appears on your screen after a brief delay.

Using Deferreds returned from XHR functions

Once you have CherryPy up and running save the file below as foo.html and place it alongside the foo.py file you already have running. You should be able to navigate to http://127.0.0.1:8080/foo.html and have foo.html load up without any issues:

<html>
    <head>
        <title>Fun with Deferreds!</title>

        <script type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js">
        </script>

        <script type="text/javascript">
            dojo.addOnLoad(function(  ) {

                 //Fire off an asynchronous request, which returns a Deferred
                 var d = dojo.xhrGet({
                    url: "http://localhost:8080",
                    timeout : 5000,
                    load : function(response, ioArgs) {
                      console.log("Load response is:", response);
                      console.log("Executing the callback chain now...");
                      return response;
                    },
                    error : function(response, ioArgs) {
                      console.log("Error!", response);
                      console.log("Executing the errback chain now...");
                      return response;
                    }
                 });

                console.log("xhrGet fired. Waiting on callbacks or errbacks");

                  //Add some callbacks
                  d.addCallback(
                    function(result) {
                      console.log("Callback 1 says that the result is ", result);
                      return result;
                    }
                  );

                  d.addCallback(
                    function (result) {
                      console.log("Callback 2 says that the result is ", result);
                      return result;
                    }
                  );

                  //Add some errbacks
                  d.addErrback(
                    function(result) {
                      console.log("Errback 1 says that the result is ", result);
                      return result;
                    }
                  );

                  d.addErrback(
                    function(result) {
                      console.log("Errback 2 says that the result is ", result);
                      return result;
                    }
                  );
            });
        </script>
    </head>
    <body>
    Check the Firebug console.
    </body>
</html>

After running this example, you should see the following output in the Firebug console:

xhrGet fired. Waiting on callbacks or errbacks
Load response is: Hello
Executing the callback chain now...
Callback 1 says that the result is Hello
Callback 2 says that the result is Hello

The big takeaway from this example is that the Deferred gives you a clean, consistent interface for interacting with whatever happens to come back from the xhrGet, whether it is a successful response or an error that needs to be handled.

You can adjust the timing values in the dojo.xhrGet function to timeout in less than the three seconds the server will take to respond to produce an error if you want to see the errback chain fire. The errback chain fires if something goes wrong in one of the callback functions, so you could introduce an error in a callback function to see the callback chain partially evaluate before kicking off the errback chain.

Remember to return the value that is passed into callbacks and errbacks so that the chains can execute the whole way through. Inadvertently short-circuiting this behavior causes bizarre results because it inadvertently stops the callback or errback chain from executing—now you know why it is so important to always remember and return a response in your load and error handlers for XHR functions.

Figure 4-2 illustrates the basic flow of events for a Deferred. One of the key points to take away is that Deferred s act like chains.

The basic flow of events through a Deferred
Figure 4-2. The basic flow of events through a Deferred

Injecting Deferreds into XHR functions

Another great feature of a Deferred is that you have a clean way of canceling an asynchronous action before it completes. The following refinement to our previous example illustrates both the ability to cancel an in-flight request as well as "injecting" a Deferred into the load and error handlers of the request:

<html>
    <head>
        <title>Fun with Deferreds!</title>

        <script type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js">
        </script>

        <script type="text/javascript">
        dojo.addOnLoad(function(  ) {
            var d = new dojo.Deferred;

              //Add some callbacks
              d.addCallback(
                function(result) {
                  console.log("Callback 1 says that the result is ", result);
                  return result;
                }
              );

              d.addCallback(
                function (result) {
                  console.log("Callback 2 says that the result is ", result);
                  return result;
                }
              );

              //Add some errbacks
              d.addErrback(
                function(result) {
                  console.log("Errback 1 says that the result is ", result);
                  return result;
                }
              );

              d.addErrback(
                function(result) {
                  console.log("Errback 2 says that the result is ", result);
                  return result;
                }
              );

             //Fire off an asynchronous request, which returns a Deferred
             request = dojo.xhrGet({
                url: "http://localhost:8080",
                timeout : 5000,
                load : function(response, ioArgs) {
                    console.log("Load response is:", response);
                    console.log("Executing the callback chain now...");

                    //inject our Deferred's callback chain
                    d.callback(response, ioArgs);

                    //allow the xhrGet's Deferred chain to continue..
                    return response;
                },
                error : function(response, ioArgs) {
                  console.log("Error!", response);
                  console.log("Executing the errback chain now...");

                  //inject our Deferred's errback chain
                  d.errback(response, ioArgs);

                  //allow the xhrGet's Deferred chain to continue..
                  return response;
                }
             });
     });
    </script>
    </head>
    <body>
        XHR request in progress. You have about 3 seconds to cancel it.
        <button onclick="javascript:request.cancel(  )">Cancel</button>
    </body>
</html>

If you run the example, you'll see the following output:

xhrGet just fired. Waiting on callbacks or errbacks now...
Load response is: Hello
Executing the callback chain now...
Callback 1 says that the result is Hello
Callback 2 says that the result is Hello

Whereas pressing the Cancel button yields the following results:

xhrGet just fired. Waiting on callbacks or errbacks now...
Press the button to cancel...
Error: xhr cancelled dojoType=cancel message=xhr cancelleddojo.xd.js (line 20)
Error! Error: xhr cancelled dojoType=cancel message=xhr cancelled
Executing the errback chain now...
Errback 1 says that the result is Error: xhr cancelled dojoType=cancel
message=xhr cancelled
Errback 2 says that the result is Error: xhr cancelled dojoType=cancel
message=xhr cancelled

Custom canceller

The various XHR functions all have a special cancellation function that is invoked by calling cancel( ), but for custom Deferred s, you can create your own custom canceller, like so:

var canceller = function(  ) {
    console.log("custom canceller...");
    //If you don't return a custom Error, a default "Deferred Cancelled" Error is
    //returned
}
var d = new dojo.Deferred(canceller); //pass in the canceller to the constructor
/* ....interesting stuff happens...*/
d.cancel(  ); // errbacks could be ready to respond to the "Deferred Cancelled" Error
            //in a special way

DeferredList

While Deferred is an innate part of Base, Core provides DeferredList, an additional supplement that facilitates some use cases in which you need to manage multiple Deferred s. Common use cases for DeferredList include:

  • Firing a specific callback or callback chain when all of callbacks for a collection of Deferred s have fired

  • Firing a specific callback or callback chain when at least one of the callbacks for a collection of Deferred s have fired

  • Firing a specific errback or errback chain when at least one of the errbacks for a collection of Deferred s have fired

The API for DeferredList follows:

dojo.DeferredList(/*Array*/list, /*Boolean?*/fireOnOneCallback, /*Boolean?*/
     fireOnOneErrback,  /*Boolean?*/consumeErrors, /*Function?*/canceller)

The signature should be self-descriptive in that calling the constructor with only a single parameter that is an Array of Deferred s produces the default behavior of firing the callback chain when the callback chains for all of the Deferred s have fired; passing in Boolean parameters can control if the callback or errback chain should be fired when at least one callback or errback has fired, respectively.

Setting consumeErrors to true results in errors being consumed by the DeferredList, which is handy if you don't want the errors produced by the individual Deferred s in the list to be directly exposed, and canceller provides a way of passing in custom cancellation function, just like with an ordinary Deferred.

Form and HTTP Utilities

While certain AJAX designs can certainly be breathtaking if implemented properly, let's not forget that certain tried and true elements like plain old HTML forms are far from obsolete and still have prominent roles to play in many modern designs—with or without AJAXification. Three functions that Base provides to transform forms include:

dojo.formToObject(/*DOMNode||String*/ formNode) //Returns Object
dojo.formToQuery(/*DOMNode||String*/ formNode) //Returns String
dojo.formToJson(/*DOMNode||String*/ formNode) //Returns String

To illustrate the effect of each of these functions, let's suppose we have the following form:

<form id="register">

    <input type="text" name="first" value="Foo">
    <input type="button" name="middle" value="Baz" disabled>
    <input type="text" name="last" value="Bar">

    <select type="select" multiple name="favorites" size="5">
        <option value="red">red</option>
        <option value="green" selected>green</option>
        <option value="blue" selected>blue</option>
    </select>

</form>

Here's the effect of running each function. Note that the disabled form element was skipped in the transform.

formToObject produces:

{
    first: "Foo",
    last : "Bar",
    favorites: [
        "green",
        "blue"
    ]
};

formToQuery produces:

"first=Foo&last=Bar&favorites=green&favorites=blue"

formToJson produces:

'{"first": "Foo", "last": "Bar", "favorites": ["green", "blue"]}'

Base provides the following additional convenience functions to you for converting a query string to an object and vice versa. They're just as straightforward as you might imagine with the caveat that the values in query string are converted to strings, even when they are numeric values :

dojo.queryToObject(/*String*/ str) //Returns Object
dojo.objectToQuery(/*Object*/ map) // Returns String

Here's a quick snippet to illustrate:

//produces {foo : "1", bar : "2", baz : "3"}
var o = dojo.queryToObject("foo=1&bar=2&baz=3");

//converts back to foo=1&bar=2&baz=3
dojo.objectToQuery(o);

Cross-Site Scripting with JSONP

While JavaScript's XmlHttpRequest object does not allow you to load data from outside of the page's current domain because of the same origin policy, it turns out that SCRIPT tags are not subject to the "same origin" policy. Consequently, an informal standard known as JSONP has been developed that allows data to be cross-domain loaded. As you might imagine, it is this very capability that empowers web applications[13] to mash up data from multiple sources and present it in a single coherent application.

JSONP Primer

Like anything else, JSONP sounds a bit mysterious at first, but it is pretty simple once you understand it. To introduce the concept, imagine that a SCRIPT tag is dynamically created and appended to the HEAD of a page that was originally loaded from http://oreilly.com. The interesting twist comes in with the source of the tag: instead of loading from the oreilly.com domain, it's perfectly free to load from any domain, say http://example.com?id=23. Using JavaScript, the operation so far is simple:

e = document.createElement("SCRIPT");
e.src="http://example.com?id=23";
e.type="text/javascript";
document.getElementsByTagName("HEAD")[0].appendChild(e);

Although the SCRIPT tag normally implies that you are loading an actual script, you can actually return any kind of content you'd like, including JSON objects. There's just one problem with that—the objects would just get appended to the HEAD of the page and nothing interesting would happen (except that you might wreck the way your page looks).

For example, you might end up with something like the following blurb, where the emphasized text is the result of the previous JavaScript snippet that dynamically added the SCRIPT tag to the HEAD of the page:

<html>
  <head>
    <title>My Page</title>
    <script type="text/javascript" >
      {foo : "bar"}
    </script>
  </head>
  <body>
Some page content.
  </body>
</html>

While shoving a JavaScript object literal into the HEAD is of little use, imagine what would happen if you could somehow receive back JSON data that was wrapped in a function call—to be more precise, a function call that is already defined somewhere on your page. In effect, you'd be achieving a truly marvelous thing because you could now asynchronously request external data whenever you want it and immediately pass it into a function for processing. To accomplish this feat, all that it takes is having the result of inserting the SCRIPT tag return the JSON data padded with an extra function call such as myCallback({foo : "bar"}) instead of just {foo : "bar"}. Assuming that myCallback is already defined when the SCRIPT tag finishes loading, you're all set because the function will execute, pass in the data as a parameter, and effectively provide you with a callback function. (It's worth taking a moment to let this process sink in if it hasn't quite clicked yet.)

But there's still a small problem: how do you get the JSON object to come wrapped with that extra padding that triggers a callback? Easy—all the kind folks at example.com have to do is provide you with an additional query string parameter that allows you to define the name of the function that the result should be wrapped in. Assuming that they've determined that you should pass in your function via the c parameter (a new request that provides c as a query string parameter for you to use), calling http://example.com?id=23&c=myCallback would return myCallback({foo : "bar"}). And that's all there is to it.

Core IO

This section explains the dojo.io facilities that are provided by Core. Injecting dynamic SCRIPT tags to retrieve padded JSON and hacking IFRAME s into a viable transport layer are the central topics of discussion.

Using JSONP with Dojo

You know enough about Dojo by this point that you won't be surprised to know that it streamlines the work involved in implementing JSONP. To accomplish the same functionality as what was described in the primer, you could use dojo.io.script.get, which takes most of the same parameters as the various XHR methods. Notable caveats are that handleAs really isn't applicable for JSONP, and callbackParamName is needed so that Dojo can set up and manage a callback function to be executed on your behalf.

Here's an example of how it's done:

//dojo.io.script is not part of Base, so remember to require it into the page
dojo.require("dojo.io.script");

dojo.io.script.get({
  callbackParamName : "c", //provided by the jsonp service
  url: "http://example.com?id=23",
  load : function(response, ioArgs) {
    console.log(response);
    return response;
  },
  error : function(response, ioArgs) {
    console.log(response);
    return response;
  }
});

To clarify, the callbackParamName specifies the name of the query string parameter that is established by example.com. It is not the name of a function you've defined to act as a callback yourself. Behind the scenes, Dojo manages the callback by creating a temporary function and channeling the response into the load function, following the same conventions as the other XHR functions. So, just allow Dojo to remove that padding for you, and then use the result in the load function and be on your merry way.

If callbackParamName was not specified at all or was incorrectly specified, you'd get a JavaScript error along the lines of "<some callback function> does not exist" because the result of the dynamic SCRIPT tag would be trying to execute a function that doesn't exist.

Connecting to a Flickr data source

The following example illustrates making a JSONP call to a Flickr data source. Try running it in Firebug to see what happens. It is also worthwhile and highly instructive to examine the error that occurs if you don't provide callbackParamName (or misspell it):

dojo.require("dojo.io.script");
dojo.io.script.get({
   callbackParamName : "jsoncallback", //provided by Flickr
   url: "http://www.flickr.com/services/feeds/photos_public.gne",
   content : {format : "json"},
   load : function(response, ioArgs) {
    console.log(response);
    return response;
  },
  error : function(response, ioArgs) {
    console.log("error");
    console.log(response);
    return response;
  }
});

Getting back JavaScript from a JSONP call

As it turns out, you could also use dojo.io.script.get to interact with a server method that returns pure JavaScript. In this case, you'd perform the request in the same manner, except instead of providing a callbackParamName, you'd provide a checkString value. The "check string" value is a mechanism that allows for checking an in-flight response to see if it has completed. Basically, if running the typeof operator on the check string value does not return undefined, the assumption is that the JavaScript has completed loading. (In other words, it's a hack.) Assuming that you had CherryPy set up with the following simple script, you would use a checkString value of o to indicate that the script has successfully loaded, as o is the variable that you're expecting to get back via the JSONP call (and when typeof(o) != undefined, you can assume your call is complete).

First, the CherryPy script that serves up the JavaScript:

import cherrypy

class Content:
    @cherrypy.expose
    def index(self):
        return "var o = {a : 1, b:2}"

cherrypy.quickstart(Content(  ))

Assuming you have CherryPy running on port 8080, here's the corresponding Dojo to fetch the JavaScript:

dojo.require("dojo.io.script");
dojo.io.script.get({
  checkString : "o",
  timeout : 2000,
  url : "http://localhost:8080",
  load : function(response, ioArgs) {
    console.log(o);
    console.log(response)
  },
  error : function(response, ioArgs) {
    console.log("error", response, ioArgs);
    return response;
  }
});

Note that dojo.io.script.get introspects and determines if you're loading JavaScript or JSON based on the presence of either checkString or callbackParamName.

IFRAME Transports

Core provides an IFRAME transport that is handy for accomplishing tasks behind the scenes that would normally require the page to refresh. While XHR methods allow you to fetch data behind the scenes, they don't lend themselves to some tasks very well; form submissions, uploading files, and initiating file downloads are two common examples of when IFRAME transports come in handy.

Following the same pattern that the rest of the IO system has established, using an IFRAME transport requires passing an object containing keyword arguments, and returns a Deferred. IFRAME transports allow using either GET or POST as your HTTP method and a variety of handleAs parameters. In fact, you can provide any of the arguments with the following caveats/additions from Table 4-4.

Table 4-4. IFRAME transport keyword arguments

Name

Type (default)

Comment

method

String ("POST")

The HTTP method to use. Valid values include GET and POST.

handleAs

String ("text")

The format for the response data to be provided to the load or handle callback. Valid values include "text", "html", "javascript", and "json". For any value except "html", the server response should be an HTML file with a textarea element that contains the response.

content

Object

If form is another argument, then the content object produce the same result as if they had been hidden form elements. If there is no form property, the content object is converted to a query string via dojo.objectToQuery( ).

As of version 1.2, XML is also handled by the IFRAME transport.

File downloads with IFRAMEs

Because triggering a file download via an IFRAME is a common operation, let's try it out. Here's a CherryPy file that serves up a local file when you navigate to http://localhost:8080/. We'll use this URL in our dojo.io.frame.send call to the server:

import cherrypy
from cherrypy.lib.static import serve_file
import os

# update this path to an absolute path on your machine
local_file_path="/tmp/foo.html"

class Content:

    #serve up a file...
    @cherrypy.expose
    def download(self):
        return serve_file(local_file_path, "application/x-download", "attachment")

# start up the web server and have it listen on 8080
cherrypy.quickstart(Content(  ), '/')

Here's the HTML file that utilizes the IFRAME. You should be able to load it up, and, assuming you've updated the path in the CherryPy script to point to it, you'll get a download dialog when you click on the button.

The first time a call to dojo.io.iframe.send happens, you may momentarily see the IFRAME get created and then disappear. A common way to work around this problem is to create the IFRAME by sending off an empty request when the page loads, which is generally undetectable. Then, when your application needs to do a send, you won't see the side effect.

<html>
    <head>
        <title>Fun with IFRAME Transports!</title>

        <script type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js">
        </script>

        <script type="text/javascript">
            dojo.require("dojo.io.iframe");

            dojo.addOnLoad(function() {
                download = function(  ) {
                    dojo.io.iframe.send({
                        url : "http://localhost:8080/download/"
                    });
                };
            });
        </script>
    </head>
    <body>
        <button onclick="javascript:download(  )">Download!</button>
    </body>
</html>

In order to use the "Download!" button multiple times, you may need to supply a timeout value for the dojo.io.iframe.send function so that it can eventually time out and make itself available to service another request.

Form submissions with IFRAMEs

Another common use case for IFRAME s is submitting a form behind the scenes—maybe even a form that involves a file upload, which would normally switch out the page. Here's a CherryPy script that handles a file upload:

import cherrypy

# set this to wherever you want to place the uploaded file
local_file_path="/tmp/uploaded_file"

class Content:

    #serve up a file...
    @cherrypy.expose
    def upload(self, inbound):
          outfile = open(local_file_path, 'wb')
          inbound.file.seek(0)
          while True:
              data = inbound.file.read(8192)
              if not data:
                  break
              outfile.write(data)
          outfile.close(  )

          # return a simple HTML file as the response
          return "<html><head></head><body>Thanks!</body></html>"
# start up the web server and have it listen on 8080
cherrypy.quickstart(Content(  ), '/')

And here's the HTML page that performs the upload. If you run the code, any file you upload gets sent in behind the scenes without the page changing, whereas using the form's own submit button POSTs the data and switches out the page. An important thing to note about the example is that the handleAs parameter calls for an HTML response.

<html>
    <head>
        <title>Fun with IFRAME Transports!</title>

        <script type="text/javascript"
          src="http://o.aolcdn.com/dojo/1.1/dojo.dojo.xd.js"
          djConfig="isDebug:true,dojoBlankHtmlUrl:'/path/to/blank.html'">
        </script>

        <script type="text/javascript">
            dojo.require("dojo.io.iframe");

            dojo.addOnLoad(function() {
                upload = function(  ) {
                dojo.io.iframe.send({
                    form : "foo",
                    handleAs : "html", //response type from the server
                    url : "http://localhost:8080/upload/",
                    load : function(response, ioArgs) {
                        console.log(response, ioArgs);
                        return response;
                    },
                    error : function(response, ioArgs) {
                        console.log("error");
                        console.log(response, ioArgs);
                        return response;
                    }
                });
            };
            });
        </script>
    </head>
    <body>
        <form id="foo" action="http://localhost:8080/upload/" method="post"
        enctype="multipart/form-data">
            <label for="file">Filename:</label>
            <input type="file" name="inbound">
            <br />
            <input type="submit" value="Submit Via The Form">
        </form>

        <button onclick="javascript:upload(  );">Submit Via the IFRAME Transport
        </button>
    </body>
</html>

The next section illustrates a caveat that involves getting back a response type that's something other than HTML.

Non-HTML response types

The previous example's server response returned an HTML document that could have been picked out of the response and manipulated. For non-HTML response types, however, there's a special condition that you must fulfill, which involves wrapping the response in a textarea tag. As it turns out, using an HTML document is the only reliable, cross-browser way that this transport could know when a response is loaded, and a textarea is a natural vehicle for transporting text-based content. Internally, of course, Dojo extracts this content and sets it as the response. The following example illustrates the changes to the previous example that would allow the response type to be plain text as opposed to HTML.

Note that while the previous examples for uploading and downloading files did not require the local HTML file to be served up by CherryPy, the following example does. The difference is that the IFRAME transport has to access the DOM of the page to extract the content, which qualifies as cross-site scripting (whereas the previous examples didn't involve any DOM manipulation at all).

The CherryPy script requires only that a configuration be added to serve up the foo.html file and that the final response be changed to wrap the content inside of a textarea like so:

import cherrypy
import os

# a foo.html file will contain our Dojo code performing the XHR request
# and that's all the following config directive is doing

current_dir = os.getcwd()
config = {'/foo.html' :
    {
    'tools.staticfile.on' : True,
    'tools.staticfile.filename' : os.path.join(current_dir, 'foo.html')
    }
}

local_file_path="/tmp/uploaded_file"

class Content:

    #serve up a file...
    @cherrypy.expose
    def upload(self, inbound):
          outfile = open(local_file_path, 'wb')
          inbound.file.seek(0)
          while True:
              data = inbound.file.read(8192)
              if not data:
                  break
              outfile.write(data)
          outfile.close(  )
          return
"<html><head></head><body><textarea>Thanks!</textarea></body></html>"

The only notable change to the request itself is that the handleAs type is different:

dojo.io.iframe.send({
    form : dojo.byId("foo"),
    handleAs : "text", //response type from the server
    url : "http://localhost:8080/upload/",
    load : function(response, ioArgs) {
        console.log(response, ioArgs); //response is "Thanks!"
        return response;
    },
    error : function(response, ioArgs) {
        console.log("error");
        console.log(response, ioArgs);
        return response;
    }
});

Manually creating a hidden IFRAME

As a final consideration, there may be times when you need to create a hidden IFRAME in the page to load in some content and want to be notified when the content finishes loading. Unlike the dojo.io.iframe.send function, which creates an IFRAME and immediately sends some content, the dojo.io.iframe.create function creates an IFRAME and allows you to pass a piece of JavaScript that will be executed when the IFRAME constructs itself. Here's the API:

dojo.io.iframe.create(/*String*/frameName, /*String*onLoadString, /*String?*/url)
//Returns DOMNode

Basically, you provide a name for the frame, a String value that gets evaluated as a callback, and an optional URL, which can load the frame. Here's an example that loads a URL into a hidden IFRAME on the page and executes a callback when it's ready:

<html>
    <head>
        <title>Fun with IFRAME Transports!</title>

        <script type="text/javascript"
            src="http://o.aolcdn.com/dojo/1./dojo/dojo.xd.js"
             djConfig="isDebug:true,dojoBlankHtmlUrl:'/path/to/blank.html'"
        </script>

        <script type="text/javascript">
            dojo.require("dojo.io.iframe");

            function customCallback(  ) {
                console.log("callback!");

                //could refer to iframe content via dojo.byId("fooFrame")...
            }

            create = function(  ) {
                dojo.io.iframe.create("fooFrame", "customCallback(  )",
                  "http://www.exmaple.com");
            }
        </script>
    </head>
    <body>
        <button onclick="javascript:create(  );">Create</button>
    </body>
</html>

Be advised that some pages have JavaScript functions in them that break them out of frames—which renders the previous usage of the transport ineffective.

Although you'll often immediately load something into an IFRAME, there may also be times when you need to create an empty frame. If you are using a locally installed toolkit, just omit the third parameter to dojo.io.iframe.create, and you'll get an empty one. If you are XDomain-loading, however, you'll need to point to a local template that supplies its content. There is a template located in your toolkit's directory at dojo/resources/blank.html that you can copy over to a convenient location. You also need to add an extra configuration parameter to djConfig before you try to create the IFRAME as shown in examples in this section.

In addition to the IO facilities provided by Core, DojoX also provides IO facilities through the dojox.io module. Among other things, you'll find utilities for XHR multipart requests and helpers for proxying.

JSON Remote Procedure Calls

By now, you may have noticed that even after using Dojo's various XHR methods such as dojo.xhrGet to reduce boilerplate, it is still a somewhat redundant and error-prone operation to repeatedly provide content to the call and write a load callback function. Fortunately, you can use Dojo's RPC (Remote Procedure Call) machinery to mitigate some of the monotony via Core's dojo.rpc module. In short, you provide some configuration information via a Simple Method Description (SMD), create an instance of this service by passing in the configuration, and then use the service instead of the xhrGet et al. If your application has a fairly standard way of interacting with the server and responds in very similar ways for error handling, etc., the benefit of using the rpc module is that you'll generally have a cleaner design that's less error-prone.

Currently, Core provides a JsonService and a JsonpService, which both descend from a base class called RpcService.

The dojox.rpc module provides additional RPC capabilities, some of which may soon be migrated to Core.

JSON RPC Example

To illustrate some basic usage of the RPC machinery, let's work through an example that uses JsonService to process a list of numbers, providing the sum of the numbers or the sum of the sum of each number squared. The client consists of an SMD that provides two methods, sum and sumOfSquares, which both take a list of numbers:

<html>
    <head>
        <title>Fun with JSON RPC!</title>

        <script type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js"
            djConfig="isDebug:true">
        </script>
        <script type="text/javascript">
            dojo.require("dojo.rpc.JsonService");
            dojo.addOnLoad(function(  ) {

                //construct the smd as an Object literal...
                var o = {
                    "serviceType": "JSON-RPC",
                    "serviceURL": "/",
                    "methods":[
                        {
                            "name": "sum",
                            "parameters":[{name : "list"}]
                        },
                        {
                            "name": "sumOfSquares",
                           "parameters":[{name : "list"}]
                        }
                    ]
                }

                //instantiate the service
                var rpcObject = new dojo.rpc.JsonService(o);

                //call the service and use the Deferred that is returned to
add a callback
                var sum = rpcObject.sum([4,8,15,16,23,42]);
                sum.addCallback(function(response) {
                    console.log("the answer is ", response);
                });
                //add more callbacks, errbacks, etc.

                //call sumOfSquares the very same way...
            });
        </script>
        <body>
    </body>
</html>

Hopefully, you see the connection that if there were lots of methods communicating with the server in a very standardized way, the general simplicity of calling an RPC client once you've set it up initially declutters the design significantly. Much of the elegance in using the dojo.rpc.JsonService is that it returns a Deferred so you can add callbacks and errbacks as needed.

In case you'd like to interact with the example, here's an example service script. For simplicity, this script purposely doesn't bring in a JSON processing library, but you'd most certainly want to do that for anything much more complicated than this example:

import cherrypy
import os
# a foo.html file will contain our Dojo code performing the XHR request
# and that's all the following config directive is doing

current_dir = os.getcwd()
config = {'/foo.html' :
    {
    'tools.staticfile.on' : True,
    'tools.staticfile.filename' : os.path.join(current_dir, 'foo.html')
    }
}

class Content:

    @cherrypy.expose
    def index(self):
        #############################################################
        # for sheer simplicity, this example does not use a json lib.
        # for anything more sophisticated than this example,
        # get a good json library from http://json.org
        ############################################################

        # read the raw POST data
        rawPost = cherrypy.request.body.read(  )

        # cast to object
        obj = eval(rawPost) #MAJOR security hole! you've been warned...

        # process the data
        if obj["method"] == "sum":
            result = sum(obj["params"][0])
        if obj["method"] == "sumOfSquares":
            result = sum([i*i for i in obj["params"][0]])

        # return a json response
        return str({"result" : result})

# start up the web server and have it listen on 8080
cherrypy.quickstart(Content(  ), '/', config=config)

Using the JsonpService is very similar to using the JsonService. In your Dojo installation, there is an example SMD file for Yahoo! services located at dojox/rpc/yahoo.smd if you want to try it out.

OpenAjax Hub

The OpenAjax Alliance (http://www.openajax.org/) is an organization of vendors and organizations that have committed themselves to interoperable AJAX-based web technologies. One of the key issues of the current era of web development is being able to use multiple JavaScript libraries within a single application. While Dojo and some of the other frameworks take precautions to cover the bare minimums for interoperability such as protecting the global namespace, actually using two libraries concurrently so that they are truly interoperable continues to produce challenges in regards to actually passing data back and forth as well as overall programming style and learning curve.

The OpenAjax Alliance has proposed what is known as the OpenAjax Hub, which is a specification for how libraries should interact. You probably won't be surprised to learn that the basic technique for interoperability is the loosely coupled publish/subscribe idiom. To that end, Core provides an OpenAjax module that implements the specification and exposes the following methods via a global OpenAjax object:

  • registerLibrary

  • unregisterLibrary

  • publish

  • subscribe

  • unsubscribe

As a champion of open standards, you can rest assured that Dojo will strive to stay current with the latest OpenAjax Hub specification, which you can read about at http://www.openajax.org/member/wiki/OpenAjax_Hub_Specification.

Summary

After reading this chapter, you should be able to:

  • Use Dojo's XHR machinery to perform RESTful operations with a web server

  • Understand how Deferred s provide the illusion of threads, even though JavaScript does not support threads

  • Be aware that the toolkit's entire IO subsystem uses and generally returns Deferred s from function calls

  • Be able to use Base's functions for converting forms to and from Objects and JSON

  • Be able to use Core's IFRAME transport layer for common operations such as uploading and downloading files

  • Understand how the RPC machinery can streamline application logic and produce a more maintainable design

  • Be aware of the infrastructure Core provides for implementing the OpenAjax Hub

We'll move on to node manipulation in the next chapter.



[12] Even though the "X" in AJAX specifically stands for XML, the term AJAX now commonly refers to virtually any architecture that employs the XMLHttpRequest object to perform asynchronous requests, regardless of the actual type of data that's returned. Although opting to use the umbrella term XHR would technically be more accurate, we'll follow common parlance and use AJAX in the broader context.

[13] Without loading any external plugins, JSONP is your only means of loading cross-domain data. Plug-ins such as Flash and ActiveX, however, have other ways of working around the "same origin" limitation that is placed on the browser itself.