Chapter 9. Data Abstraction

A common bane of web development is writing routines to parse data that is returned from the server into a conveniently accessible format for the core application logic. While many good routines have been developed to parse common response types such as comma-separated values (CSV) and JSON, a lot of boilerplate is still involved in wiring it all up, issuing updates back to the server, potentially maintaining synchronicity between the local store and the server, and so forth. This chapter introduces Dojo's data APIs, which provide a uniform interface for handling data sources—regardless of where they're located, how they're accessed at the transport level, and what their format may be.

Shifting the Data Paradigm

The toolkit's machinery for managing data sources isn't exactly rocket science, but it does require shifting the paradigm ever so slightly, in that it requires that data can be treated as a local resource that is accessed via a uniform API. Traditional approaches have typically entailed treating data as a remote resource, which necessarily entails acquiring boilerplate to retrieve it, writing updates to the server, maintaining synchronicity with the server, and handling variable formats. One of the central issues, historically speaking, is that the wheel was reinvented far too many times and virtually every application used its own brittle approach to managing the burden of handling data.

Dojo gives a set of APIs via the dojo.data module that provide a standardized means for interacting with arbitrary data sources, shown in Figure 9-1. This allows application programmers to escape entanglement with retrieving, parsing, and managing it. The dojo.data API provides a standardized manner for interacting with data whether it's local or remote, which is a tremendous boon when it comes time to deal with larger and larger data sets as an application scales. Best of all, once you've implemented an interface for a specific data format, it becomes an off-the-shelf resource that you can reuse and distribute at will. Generally speaking, these kinds of off-the-shelf resources allow application developers to be more productive by allowing them to focus on far more interesting tasks at hand than I/O management.

Left: a traditional pattern for accessing arbitrary data sources from an application; right: the toolkit's dojo.data abstraction for accessing arbitrary data sources
Figure 9-1. Left: a traditional pattern for accessing arbitrary data sources from an application; right: the toolkit's dojo.data abstraction for accessing arbitrary data sources

Data API Overview

The most basic atom of the dojo.data API is called an item, which is composed of key/value pairs called attributes and attribute values in dojo.data parlance; conceptually, you can think of an item as a plain old JavaScript Object. However, although the underlying implementation may very well be a JavaScript Object, be careful to use the provided APIs for accessing it, as the internal representation may be something entirely different. For example, some data abstractions may use a DOM model for storing certain types of data for efficiency reasons, or lazy-load data on the fly even though it seems like it's already local. In cases like these, accessing an item like a plain old JavaScript object would likely cause an unexpected error. We'll come back to specific API calls for accessing an item in the next section.

Saying that an item has an attribute—but no value for the attribute—is the same as saying that the item doesn't have the attribute at all. In other words, it's nonsensical to think about having an attribute with no value because attributes inherently have a specific state.

Before getting into the capabilities of any one specific API, it's helpful to survey the landscape. Here's an overview of the various dojo.data APIs with a brief summary of what these APIs provide to the application developer. These APIs are interfaces, not implementations; any concrete dojo.data data store would define one or more of the upcoming APIs:

dojo.data.api.Read

Provides a uniform means of reading, searching, sorting, and filtering data items.

dojo.data.api.Write

Provides a uniform means of creating, deleting, and updating data items.

dojo.data.api.Identity

Provides a uniform means of accessing an item via a unique identifier.

dojo.data.api.Notification

Provides a uniform means of notifying a listener of a change, such as create, delete, or update operation, for a data item.

The remainder of this chapter systematically works through each of these APIs and provides plenty of examples so that you can make the most of dojo.data in your own application.

The APIs

This section provides a summary of the data APIs. If you're just getting started, you may want to skim this section to get a feel for the capabilities the APIs and then come back after you've read the rest of the chapter, which has more concrete examples, to explore it further.

The Read API

All data stores will implement the dojo.data.api.Read API because this API provides the means of retrieving, processing, and accessing the data—clearly a prerequisite for any other operation. The complete API specification follows in Table 9-1. The next section discusses a toolkit-provided implementation: ItemFileReadStore.

The API listings that follow use descriptors like dojo.data.api.Item to convey the notion of a dojo.data item even though an item is somewhat of an abstract concept.

Table 9-1. The dojo.data.api.Read API

Name

Comment

getValue(/*dojo.data.api.Item*/item, /*String*/attribute, /*Any?*/default)

Given an item and an attribute name, returns the value for the item. A value of undefined is returned if the attribute does not exist (whereas null is returned only if null is explicitly set as the value for the attribute). An optional parameter of default can be used to return a default value if one does not exist.

getValues(/*dojo.data.api.Item*/item, /*String*/attribute)

Works just like getValue except that it allows for multivalued attributes. Always returns an array regardless of the number of items returned. You should always use getValues for multivalued attributes.

getAttributes(/*dojo.data.api.Item*/item)

Introspects the item to return an Array of String values corresponding to the item's attributes. Returns an empty Array in the event that an item has no attributes.

hasAttribute(/*dojo.data.api.Item*/item, /*String*/attribute)

Returns true if the item has the attribute.

containsValue(/*dojo.data.api.Item*/item, /*String*/attribute, /*Any*/value)

Returns true if the item has the attribute and the attribute has the value, i.e., getValues would return true.

isItem(/*Any*/item)

Returns true if the parameter is an item and came from the specified store. Returns false if item is a literal or if the item came from any other store. (This call is also especially handy for situations when local variable references to items can become stale, which is quite ordinary for stores that implement the Write API.)

isItemLoaded(/*Any*/item)

Returns true if the item is loaded and available locally.

loadItem(/*Object*/args)

Loads an item to the effect that a subsequent call to isItemLoaded would return true. The args object provides the following keys:

item

An Object providing criteria for the item that is to be loaded (possibly an identifier or subset of identifying data).

onItem(/*dojo.data.api.Item*/item)

A callback to run when the item is loaded; the loaded item is passed as a parameter.

onError(/*Object*/error)

A callback to run when an error occurs loading the item; the error Object is passed as a parameter.

scope

An Object providing the context for callback functions.

fetch(/*Object*/args)

Executes a given query and provides an assortment of asynchronous callbacks for handling events related to the query execution. Returns a dojo.data.api.Request Object, whose primary purpose is supplying an abort( ) method that may be used for aborting the fetch. The arguments may include the following nine options, which should be honored by all implementations:

query

A String or Object providing the query criteria (similar to SELECT in SQL). Note that the query syntax for each store is implementation-dependent.

queryOptions

An Object containing additional options for the query. All stores should attempt to honor the ignoreCase (Boolean, default false ) parameter, which performs a case-insensitive search and the deep (Boolean, default false ) parameter can trigger a query of all items and child items instead of only items at the root level in the store.

onBegin (/*Integer*/size, /*dojo.data.api.Request*/request)

Called before the first onItem callback. size indicates the total number of items and request is the original request for the query. If size is unknown, it will be -1. size may not be the total number of items returned since it may have been pared down with start and count.

onComplete (/*Array*/items, /*dojo.data.api.Request*/request)

Called just after the last onItem callback. If no onItem callback is present, items will be an Array of all items that matched the query; otherwise, it will be null.Request, the original request Object.

onError(/*Object*/error, /*dojo.data.api.Request*/request)

Called if there is an error when executing the query. error contains the error information and request contains the original request Object.

onItem(/*dojo.data.api.Item*/item, /*dojo.data.api.Request*/request)

Called for each item that is returned with each item available as item; request contains the original request Object.

scope (Object)

If provided, executes all of the callback function in this context; otherwise, executes them in the global context.

start (Integer)

Provides a starting offset for the returned results (similar to OFFSET in an SQL query).

count (Integer)

Provides a limit for the items to be returned (similar to LIMIT in a SQL query).

sort (Array)

An Array of JavaScript Objects providing sort criteria for each attribute. Each Object is applied sequentially and must present an attribute key identifying the attribute name and a descending key identifying the sort order.

getFeatures( )

Returns an Object containing key/value pairs that specifies what dojo.data APIs it implements. For example, any API implementing dojo.data.api.Read would necessarily return {'dojo.data.api.Read' : true} for a data store that is read-only.

close(/*dojo.data.api.Request*/request)

Used to close out any information associated with a particular request, which may involve clearing caches, closing connections, etc. The request parameter is expected to be the Object returned from a fetch. For some stores, this may be a no-op.

getLabel(/*dojo.data.api.Item*/item)

Used to return a human-readable label for an item that is generally some kind of identifying description. The label may very well be some combination of attributes.

getLabelAttributes(/*dojo.data.api.Item*/item)

Used to provide an Array of attributes that will generate an item's label. Useful for assisting UI developers in knowing what attributes are useful for display purposes so that redundant information can be hidden when a display includes an item label.

The Identity API

The Identity API, shown in Table 9-2, builds on the Read API to provide a few additional calls for fetching items based upon their identity. Note that the Read API has no stipulations whatsoever that items be unique, and there are certainly use cases where the notion of an identity may not be pertinent; hence the separation between the two of them. With respect to databases, you might think of the Identity API, loosely, as providing a kind of primary key for each item that records can be identified with. It is often the case that data-enabled widgets require the Identity API, particularly when providing Write functionality. (The Write API is coming up next.)

Table 9-2. The dojo.data.api.Identity API

Name

Comment

getFeatures( )

See dojo.data.api.Read. Returns:

{
'dojo.data.api.Read' : true,
'dojo.data.api.Identity : true
}

getIdentity(/*dojo.data.api.Item*/item)

Returns a unique identifier for the item, which will be a String or an Object that has a toString method.

getIdentityAttributes (/*dojo.data.api.Item*/item)

Returns an Array of attribute names that are used to generate the identity. Most of the time, this is a single attribute that expressly provides a unique identifier, but it could be more than one depending on the specifics of the actual data source. This function is often used to optionally hide attributes comprising the identity for display purposes.

fetchItemByIdentity(/*Object*/args)

Uses the identity of an item to retrieve it; conforming implementations should return null if no such item exists. The keyword args may include the following:

identity

A String or Object with a toString function that is uses to provide the reference for the desired item.

onError(/*Object*/error)

Called if there is an error when executing the query. error contains the error information.

onItem(/*dojo.data.api.Item*/item)

Called for each item that is returned with each item available as item.

scope (Object)

If provided, executes all of the callback function in this context; otherwise, executes them in the global context.

The Write API

The Write API, shown in Table 9-3, extends the Read API to include facilities for creating, updating, and deleting items, which necessarily entails managing additional issues such as whether items are dirty —out of sync between the in-memory copy and the server—and handling I/O such as save operations.

Table 9-3. The dojo.data.api.Write API

Name

Comment

getFeatures

See dojo.data.api.Read. Returns:

{
'dojo.data.api.Read' : true,
'dojo.data.api.Write : true
}

newItem(/*Object?*/args, /*Object?*/parentItem)

Returns a newly created item, setting the attributes based on the args Object provided, where generally the key/value pairs in args map directory attributes and attribute values. For stores that support hierarchical item creation, parentItem provides information identifying the parent of the new item and the attribute of the parent that the new item should be assigned (which generally implies that the attribute is multivalued and the new item is appended). Returns the newly created item.

deleteItem(/*dojo.data.api.Item*/item)

Deletes and item from the store. Returns a Boolean value indicating success.

setValue(/*dojo.data.api.Item*/item, /*String*/attribute, /*Any*/value)

Sets an attribute on an item, replacing any previous values. Returns a Boolean value indicating success.

setValues(/*dojo.data.api.Item*/item, /*String*/attribute, /*Array*/values)

Sets values for an attribute, replacing any previous values. Returns a Boolean value indicating success.

unsetAttribute(/*dojo.data.api.Item*/item, /*String*/attribute)

Effectively removes an attribute by deleting all values for it. Returns a Boolean value indicating success.

save(/*Object*/args)

Saves all local changes in memory, and output is passed to a callback function provided in args, which is in the following form:

onError(/*Object*/error)

Called if there is an error; error contains the error information.

onComplete( )

Called to indicate success, usually with no parameters.

scope (Object)

If provided, executes all of the callback functions in this context; otherwise executes them in the global context.

A _saveCustom extension point is available, and if overridden, provides an opportunity to pass the data set back to the server.

revert( )

Discards any local changes. Returns a Boolean value indicating success.

isDirty(/*dojo.data.api.Item?*/)

Returns a Boolean value indicating if a specific item has been modified since the last save operation. If no parameter is provided, returns a Boolean value indicating if any item has been modified.

The Notification API

The Notification API, shown in Table 9-4, is built upon the Read and complements the Write API by providing a unified interface for responding to the typical create, update, and delete events. The Notification API is particularly useful for ensuring visuals properly reflect the state of a store, which may be changing or refreshed via Read and Write operations. (The dijit.Tree and dojox.grid.Grid widgets are great cases in point.)

Table 9-4. The dojo.data.api.Notification API

Name

Comment

getFeatures

See dojo.data.api.Read. Returns:

{
'dojo.data.api.Read' : true,
'dojo.data.api.Notification: true
}

onSet(/*dojo.data.api.Item*/item, /*String*/attribute, /*Object|Array*/old, /*Object|Array*/new)

Called any time an item is modified via setValue, setValues, or unsetAttribute, providing means of monitoring actions on items in the store. The parameters are self-descriptive and provide the item being modified, the attribute being modified, and the old and new value or values, respectively.

onNew(/*dojo.data.api.Item*/item, /*Object?*/parentItem)

Called when a new item is created in the store where item is what was just created; parentItem is not passed if the created item is placed in the root level. parentItem, however, is provided if the item is not a root-level item. (Note that an onSet notification is not generated stating that the parentItem 's attributes have been modified because it is implied and parentItem gives access to that information.)

onDelete(/*dojo.data.api.Item*/item)

Called when an item is deleted from the store where item is what was just deleted.

Core Implementations of Data APIs

The previous section provided a summary of the four primary data APIs available at this time. This section works through the two implementations provided by Core—the ItemFileReadStore and ItemFileWriteStore. As you'll see, the ItemFileReadStore implements the Read and Identity APIs, and the ItemFileWriteStore implements all four APIs discussed. A good understanding of these two stores equips you with enough familiarity to augment these existing stores to suit your own needs—or to implement your own.

Although not explicitly discussed in this book, the dojox.data subproject contains a powerful assortment of dojo.data implementations for common tasks such as interfacing to CSV stores, Flickr, XML, OPML, Picasa, and other handy stores. Since they all follow the same APIs as you're learning about here, picking them up should be a snap.

ItemFileReadStore

Although it is quite likely that your particular situation may benefit from a custom implementation of dojo.data.api.Read to maximize efficiency and impedance mismatching, the toolkit does include the ItemFileReadStore, which implements the Read and Identity interfaces and consumes a flexible JSON representation. For situations in which you need to quickly get something up and running, you need to do little more than have your application's business logic output data in the format that the ItemFileReadStore expects, and voilà, you may use the store as needed.

One thing to know up front is that the ItemFileReadStore consumes the entire data set that backs it into memory the first time a request is made; thus, operations like isItemLoaded and loadItem are fairly useless.

Hierarchical JSON and JSON with references

Although the ItemFileReadStore does implement the Read API, it packs a number of implementation-specific features of its own, including a specific data format, query syntax, a means of deserializing specific attribute values, specific identifiers for the identity of an item, and so on. Before getting into those specifics, however, have a look at some example data that is compliant for the ItemFileReadStore to consume; there are two basic flavors that relate to how nested data is represented: hierarchical JSON and JSON with references. The hierarchical JSON flavor consists of nested references that are concrete item instances, while the JSON with references flavor consists of data that points to actual data items.

To illustrate the difference between the two, first take a look at a short example of the two variations. First, for the hierarchical JSON:

{
    identifier : id,
    items : [
       {
            id : 1, name : "foo", children : [
                {id : 2, name : "bar"},
                {id : 3, name : "baz"}
            ]
        }
        /* more items... */
    ]
}

And now, for the JSON with references:

{
    identifier : id,
    items : [
        {
            id : 1, name : "foo", children : [
              {_reference: 2},
              {_reference: 3}
            ]
        },
        {id : 2, name : "bar"},
        {id : 3, name : "baz"}
        /* more items... */
    ]
k}

To recap, the foo item has two child items in both instances, but the hierarchical JSON explicitly nests the items, while the JSON with references uses pointers keying off of the identifier for the item. The primary advantage to JSON with references is its flexibility; it allows items to appear as the child of more than one parent, as well as the possibility for all items to appear as root-level items. Both possibilities are quite common and convenient for many real-world applications.

The Tree Dijit, introduced in Chapter 15, is a great example that highlights the flexibility and power (as well as some of the shortcomings) of the JSON with references data format.

ItemFileReadStore walkthrough

To get up close and personal with the ItemFileReadStore, consider the data collection represented as hierarchical JSON, shown in Example 9-1, where each item is identified by the name identifier. Note that the identifier, label, and items keys are the only expected values for the outermost level of the store.

Example 9-1. Sample coffee data set
{
    identifier : "name",
    label : "name",

    items : [
        {name : "Light Cinnamon", description : "Very light brown, dry , tastes
like toasted grain with distinct sour tones, baked, bready"},
        {name : "Cinnamon", description : "Light brown and dry, still toasted
grain with distinct sour acidy tones"},
        {name : "New England", description : "Moderate light brown , still sour
but not bready, the norm for cheap Eastern U.S. coffee"},
        {name : "American or Light", description : "Medium light brown, the
traditional norm for the Eastern U.S ."},
        {name : "City, or Medium", description : "Medium brown, the norm for
most of the Western US, good to taste varietal character of a bean."},
        {name : "Full City", description : "Medium dark brown may have some
slight oily drops, good for varietal character with a little bittersweet."},
        {name : "Light French", description : "Moderate dark brown with oily
drops, light surface oil, more bittersweet, caramelly flavor, acidity muted."},
        {name : "French", description : "Dark brown oily, shiny with oil,
also popular for espresso; burned undertones, acidity diminished"},
        {name : "Dark French", description : "Very dark brown very shiny, burned
tones become more distinct, acidity almost gone."},
        {name : "Spanish", description : "Very dark brown, nearly black and
very shiny, charcoal tones dominate, flat."}
    ]
}

Assuming the file was stored on disk as coffee.json, the page shown in Example 9-2 would load the store and make it available via the coffeeStore global JavaScript variable.

Example 9-2. Programmatically loading an ItemFileReadStore
<html>
    <head>
        <title>Fun with ItemFileReadStore!</title>
        <script
            type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js">
        </script>
        <script type="text/javascript">
            dojo.require("dojo.data.ItemFileReadStore");

            dojo.addOnLoad(function(  ) {
                coffeeStore = new dojo.data.ItemFileReadStore({url:"coffee.json"});
            });
        </script>
    </head>
    <body>
    </body>
</html>

Although the parser isn't formally introduced until Chapter 11, using the parser is so common that it's worthwhile to explicitly mention that the markup variation in Example 9-3 would have achieved the very same effect.

Example 9-3. Loading an ItemFileReadStore in markup
<html>
    <head>
        <title>Fun with ItemFileReadStore!</title>
        <script
            type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js"
            djConfig="parseOnLoad:true">
        </script>
        <script type="text/javascript">
            dojo.require("dojo.parser");
            dojo.require("dojo.data.ItemFileReadStore");
        </script>
    </head>
    <body>
        <div dojoType="dojo.data.ItemFileReadStore" url="./coffee.json"
          jsId="coffeeStore"></div>
    </body>
</html>

Regardless of how you declare the store, the API works the same either way. A great exercise is to spend a few minutes in the Firebug console with the existing store. The remainder of this section contains a series of commands and code snippets with the corresponding response values for most of the Read and Identity APIs that you can follow along with and use to accelerate your learning about the ItemFileReadStore.

In addition to using the url parameter to point an ItemFileReadStore at a data set represented as a file, you could also have passed it a variable referencing a JavaScript object that's already in memory via the data parameter.

Fetching an item by identity

Fetching an item from the ItemFileReadStore is generally done in one of two ways, though each way is quite similar. To fetch an item by its identifier, you should use the Identity API's fetchItemByIdentity function, which accepts a collection of named arguments including the identifier, what to do when the item is fetched, and what to do if an error occurs. For example, to query the sample coffeeStore for the Spanish coffee, you could do something like Example 9-4.

Example 9-4. Fetching an item by its identity and then inspecting it
coffeeStore.fetchItemByIdentity({
    identity: "Spanish",
    onItem : function(item, request) {
        var spanishCoffeeItem = item;

        //now do something with the results of the fetch request

        //like get its description...
        coffeeStore.getValue(spanishCoffeeItem, "description"); //Very dark brown...

        //or get its name...
        coffeeStore.getValue(spanishCoffeeItem, "name"); // Spanish

        //in this case, the name and label are the same...
        coffeeStore.getLabel(spanishCoffeeItem); // Spanish

    },
    onComplete(items, request) {
        /* You could access the entire result of a fetch request here,
           which in this case would only be an array of one item
           since a fetch by identity guarantees only one item back */
    },
    onError : function(item, request) {
        /* Handle any error here... */
    }
});

A common mistake when you're just starting out is to accidentally confuse the identity of an item with the item itself, which can be a tricky semantic bug to find because the code "looks right." Finding the Spanish coffee item via var item = coffeeStore.fetchItemByIdentity("Spanish") reads as though it makes sense, but when you take a closer look at the API, you realize that it's wrong in at least two ways: the call doesn't return an item back to you, and you have to provide a collection of named arguments to it—not an identity value.

Fetching an item by arbitrary criteria

If you want to fetch an item by an attribute other than the identity, you could use the more generic fetch function instead of fetchItemByIdentity, like so:

coffeeStore.fetch({
    query: {name : "Spanish"},
    onItem : function(item, request){console.log(item);}
});

However, in addition to accepting fully qualified values for attributes, the fetch function also accepts a small but robust collection of filter criteria that allows for basic regex-style matching. For example, to find any coffee description with the word "dark" in it without regard to case-sensitivity, you follow the process illustrated in Example 9-5.

Example 9-5. Fetching an item by arbitrary criteria
coffeeStore.fetch({
    query: {description : "*dark*"},
    queryOptions:{ignoreCase : true},
    onItem : function(item, request) {
        console.log(coffeeStore.getValue(item, "name"));
    }
    /* include other fetch callbacks here... */
});

Always use the store to access item attributes via getValue. Don't try to access them directly because the underlying implementation of the store may not allow it. For example, you would not want to access an item in the onItem callback as onItem: function(item, request) { console.log(item.name); }. A tremendous benefit from this abstraction is that it gives way to underlying caching mechanisms and other optimizations that improve the efficiency of the store.

If you're designing your own custom implementation of a store, you may find it helpful to know that dojo.data.util.filter is a short mix-in that can give you the same functionality as the regex-style matching that ItemFileReadStore uses for fetch, and dojo.data.util.simpleFetch provides the logic for its eight arguments: onBegin, onItem, onComplete, onError, start, count, sort, and scope.

Querying child items

The existing coffee store is quite primitive in that it is a flat list of items. Example 9-6 spices it up a bit by adding in a few additional items that contain children to produce a nested structure. The ItemFileReadStore expressly uses the children attribute to maintain a list of child items, and we'll use the JSON with references approach to accommodate the task of grouping coffees into different roasts. Note that the Light French roast has been deliberately placed into the Medium Roasts and the Dark Roasts to illustrate the utility of using references. Because each item needs to maintain a unique identity, it wouldn't be possible to include it as a child of two different parents any other way.

Although the remainder of this chapter uses a store that consists of only two levels, there is no reason why you couldn't use a data set with any arbitrary number of levels in it.

Example 9-6. Updated sample coffee data set to reflect hierarchy
{
    identifier : "name",
    items : [
        {
            name : "Light Roasts",
            description : "A number of delicious light roasts",
            children : [
                {_reference: "Light Cinnamon"},
                {_reference: "Cinnamon"},
                {_reference: "New England"}
            ]
        },

        {
            name : "Medium Roasts",
            description : "A number of delicious medium roasts",
            children : [
                {_reference: "American or Light"},
                {_reference: "City, or Medium"},
                {_reference: "Full City"},
                {_reference: "Light French"}
            ]
        },

        {
            name : "Dark Roasts",
            description : "A number of delicious dark roasts",
            children : [
                {_reference: "Light French"},
                {_reference: "French"},
                {_reference: "Dark French"},
                {_reference: "Spanish"}
            ]
        },

        {name : "Light Cinnamon", description : "Very light brown, dry , tastes
like toasted grain with distinct sour tones, baked, bready"},
        ...
    ]
}

A common task you might find yourself needing to accomplish is querying the children of an item. In this case, that amounts to finding the individual names associated with any given roast. Let's try it out in Example 9-7 for the Dark Roasts item to illustrate.

Example 9-7. Fetching an item and iterating over its children
coffeeStore.fetch({
    query: {name : "Dark Roasts"},
    onItem : function(item, request) {
        dojo.forEach(coffeeStore.getValues(item, "children"), function(childItem) {
          console.log(coffeeStore.getValue(childItem, "name"));
        });
    }
});

To recap, we issue a straightforward query for the parent item Dark Roasts, and then once we have the item, we use the getValues function to retrieve the multivalued children attribute and iterate over each with dojo.forEach —all the while remembering to use the getValue function to ultimately access the child item's value.

Note that the whole notion of {_reference: someIdentifier} is simply an implementation detail. There is never a time when you'll want to attempt to query based on the _reference attribute because there really isn't any such thing as a _reference attribute—again, it's just a standardized way of managing the bookkeeping. As far as the dojo.data application programmer is concerned, everything in a dojo.data store should be considered a good old item.

As you hopefully have observed by now, ItemFileReadStore is quite flexible and powerful, which makes it a suitable data format for a variety of situations—especially when you have to prototype an application and get something up and running quickly. As a simple specification, it's not difficult to have a server-side routine spit out data that a web client using dojo.data can digest. At the same time, however, remember that you can always subclass and extend as you see fit—or implement your own.

ItemFileWriteStore

There's no doubt that good abstraction eliminates a lot of cruft when it comes time to serve up data from the server and display it; however, it is quite often the case that you won't have the luxury of not writing data back to the server if it changes—and that's where the ItemFileWriteStore comes in. Just as the ItemFileReadStore provided a nice abstraction for reading a data store, ItemFileWriteStore provides the same kind of abstraction for managing write operations such as creating new items, deleting items, and modifying items. In terms of the dojo.data APIs, the ItemFileWriteStore implements them all—Read, Identity, Write, and Notification.

To get familiar with the ItemFileWriteStore, we'll work through the specifics in much the same way that we did for the ItemFileReadStore using the same coffee.json JSON data. As you'll see, there aren't any real surprises; the API pretty much speaks for itself.

Modifying an existing item

You'll frequently use the setValue function, shown in Example 9-8, to change the value of item's attribute by passing in the item, the attribute you'd like to modify, and the new value for the attribute. If the item doesn't have the named attribute, it will automatically be added.

Example 9-8. Setting an item's attribute
//Fetch an item like usual...
coffeeStore.fetchItemByIdentity({
    identity: "Spanish",
    onItem : function(item, request) {
        var spanishCoffeeItem = item;;

        coffeeStore.setValue(spanishCoffeeItem, "foo", "bar");

        coffeeStore.getValue(spanishCoffeeItem, "foo"); //bar

        //Likewise, you could have changed any other attribute except for the identity
        coffeeStore.setValue(spanishCoffeeItem, "description", "El Matador...?!?");
    }
});

Just like in most other data schemes, it doesn't usually make sense to change an item's identity, as the notion of identity is an immutable characteristic; following suit, the ItemFileWriteStore does not support this operation, nor is it recommended in any custom implementation of your own.

One peculiarity to note is that setting an attribute to be an empty string is not the same thing as removing the attribute altogether; this is especially important to internalize if you find yourself needing to use the Write API's hasAttribute function to check for the existence of an attribute. Example 9-9 illustrates the differences.

Example 9-9. Setting and unsetting attributes on items
coffeeStore.hasAttribute(spanishCoffeeItem, "foo"); //true

coffeeStore.setValue(spanishCoffeeItem, "foo", ""); //foo=""

coffeeStore.hasAttribute(spanishCoffeeItem, "foo"); //true

coffeeStore.unsetAttribute(spanishCoffeeItem, "foo"); //remove it

coffeeStore.hasAttribute(spanishCoffeeItem, "foo"); //false

While the previous examples in this section have demonstrated how to successfully modify an existing item, the changes so far have been incomplete in that an explicit save operation has not occurred. Internally, the ItemFileReadStore keeps track of changes and maintains a collection of dirty items—items that have been modified, but not yet saved. For example, after having modified the spanishCoffeeItem, you could use the isDirty function to learn that it has been modified but not saved, as shown in Example 9-10. After the item is saved, however, it is no longer dirty. For now, saving means nothing more than updating the in memory copy; we'll talk about saving back to the server in just a bit.

Example 9-10. Inspecting an item for dirty status
/* Having first modified the spanishCoffeeItem... */
coffeeStore.isDirty(spanishCoffeeItem); //true
coffeeStore.save(  ); //update in-memory copy of the store
coffeeStore.isDirty(spanishCoffeeItem); //false

Although it might not be immediately obvious, an advantage of requiring an explicit save operation to commit the changes lends the ability to revert the changes in case a later operation that is part of the same macro-level transaction produces an error or any other deal-breaking circumstance occurs. In relational databases, this is often referred to as a rollback. Example 9-11 illustrates reverting a dojo.data store and highlights a very subtle yet quite important point related to local variables that contain item references.

Example 9-11. Reverting changes to an ItemFileWriteStore
coffeeStore.fetchItemByIdentity({
    identity: "Spanish",
    onItem : function(item, request) {
        var spanishCoffeeItem = item;

        coffeeStore.getValue(spanishCoffeeItem, "description"); //Very dark...

        coffeeStore.setValue(spanishCoffeeItem, "description", "El Matador...?!?");

        //Right now, both the spanishCoffeeItem and the store reflect the
        //Udpated description. Let's do another fetch to verify... 

        coffeeStore.fetchItemByIdentity({
            identity: "Spanish",
            onItem : function(item, request) {
                coffeeStore.getValue(item, "description"); //El Matador...?!?
                coffeeStore.isDirty(item); //true

                coffeeStore.revert(  ); //revert the store.

                // Upon revert(  ), the local spanishCoffeeItem variable
                // ceased to be an item in the store
                coffeeStore.isItem(spanishCoffeeItem); //false

                //Fetch out the item again to demonstrate...

                coffeeStore.fetchItemByIdentity({
                    identity: "Spanish",
                    onItem : function(item, request) {
                        coffeeStore.isDirty(item); //false
                        coffeeStore.getValue(item, "description"); //Very dark...
                    }
                });

            }
        });

    }
});

Although it's theoretically possible to implement a custom store that prevents local item references from becoming stale via slick engineering behind the scenes with dojo.connect or pub/sub communication, the ItemFileWriteStore does not go to such lengths, and you should use the isItem function liberally if you are concerned about whether an item reference has become stale.

Creating and deleting items

Once you have a good grasp on the previous section that worked through the various nuances of modifying existing items, you'll have no problem picking up how to add and delete items from a store. All of the same principles apply with respect to saving and reverting—there's really not much to it. First, as shown in Example 9-12, let's add and delete a top-level item from our existing store. Adding an item involves providing a JSON object just like the server would have included in the original data set.

Example 9-12. Adding and deleting an item from an ItemFileWriteStore
var newItem = coffeeStore.newItem({
    name : "Really Dark",
    description : "Left brewing in the pot all day...extra octane."
});

coffeeStore.isItem(newItem); //true
coffeeStore.isDirty(newItem); //true

/* Query the item, save the store, revert the store, etc. */

//Or delete the item...
coffeeStore.deleteItem(newItem);
coffeeStore.isItem(newItem); //false

While adding and removing top-level items from a store is trivial, there is just a little bit more effort involved in adding a top-level item that also needs to a be a child item that is referenced elsewhere. Example 9-13 illustrates how it's done. The basic recipe is that you create it as a top-level item, get the children that you want it to join, and then add it to that same collection of children.

Example 9-13. Adding a child item to a JSON with references store
//Get a reference to the parent with the children
coffeeStore.fetchItemByIdentity({
    identity : "Dark Roasts",
    onItem : function(item, request) {
        var darkRoasts = item;

        //Use getValues to grab the children
        var darkRoastChildren = coffeeStore.getValues(darkRoasts, "children");

        
        //And add it to the children usingsetValues 
        coffeeStore.setValues(darkRoasts, "children", 
            darkRoastChildren.concat(newItem)
        )    
       
        //You could now iterate over those children to see for yourself...
        dojo.forEach(darkRoastChildren, function(x) {
            console.log(coffeeStore.getValue(x, "name"));
        });
    }
});

Remember to use getValues, not getValue, when fetching multivalued attributes.

Deleting items works in much the way you would expect. Deleting a top-level item removes it from the store but leaves its children, if any, in place, as shown in Example 9-14.

Example 9-14. Deleting a top-level item from an ItemFileWriteStore
coffeeStore.fetchItemByIdentity({
    identity : "Dark Roasts",
    onItem : function(item, request) {
        var darkRoasts = item;

        coffeeStore.deleteItem(darkRoasts);

        coffeeStore.fetch({
            query : {name : "*"},
            onItem : function(item, request) {
                //You won't see the "Dark Roasts" item in these results...
                console.log(coffeeStore.getValue(item, "name"));
            },
            onComplete : function(items, request) {
                /* Save the store, or revert the store, or... */
            }
        });
    }
});

Clearly, you could eliminate a top-level item and all of its children by first querying for the children, deleting them, and then deleting the top-level item itself.

Custom saves

You've probably been thinking for a while that saving in memory is great and all—but what about getting data back on the server? As it turns out, the ItemFileWrite store provides a _saveCustom extension point that you can implement to trigger a custom routine that fires anytime you call save ; thus, in addition to updating the local copy in memory and clearing any dirty flags, you can also sync up to the server—or otherwise do anything else that you'd like. You have the very same API available to you that you've been using all along, but in general, a "full save" would probably consist of iterating over the entire data set, serializing into a custom format—quite likely with the help of dojo.toJson —and shooting it off. Just as the Write API states, you provide keyword arguments consisting of optional callbacks, onComplete and onError, which are fired when success or an error occurs. An optional scope argument can be provided that supplies the execution context for either of those callbacks. Those keyword arguments, however, are passed into the save function—not to your _saveCustom extension.

Example 9-15 shows how to implement a _saveCustom handler to pass data back to the server when save() is called. As you'll see, it's pretty predictable.

Example 9-15. Wiring up a custom save handler for an ItemFileWriteStore
<html>
    <head>
        <title>Fun with ItemFileWriteStore!</title>
        <script
            type="text/javascript"
            src="http://o.aolcdn.com/dojo/1.1/dojo/dojo.xd.js">
        </script>
        <script type="text/javascript">
            dojo.require("dojo.data.ItemFileWriteStore");

            dojo.addOnLoad(function(  ) {
                coffeeStore = new dojo.data.ItemFileWriteStore({url:"coffee.json"}
    );
                coffeeStore._saveCustom = function(  ) {
                    /* Use whatever logic you need to save data back to the server.
                       This extension point gets called anytime you call an ordinary
                       save(  ). */
                }
            });
        </script>
    </head>
    <body>
    </body>
</html>

As it turns out, _saveCustom is used less frequently than you might think because it involves passing all of your data back to the server, which is not usually necessary unless you start from a blank slate and need to do that initial batch update. For many use cases—especially ones involving very large data sets—you'll want to use the interface provided by the Notification API that is introduced in the next section to take care of changes when they happen in small bite-size chunks.

Responding to notifications

To round out this section—and the rest of the chapter—we'll briefly review the Notification API that ItemFileWriteStore implements because it is incredibly useful for situations in which you need to respond to specific notifications relating to the creation of a new item, the deletion of an item, or the modification of an item via onNew, onDelete, or onSet, respectively.

As you're probably an expert reading and mapping the APIs back to specific store implementations by now, an example that adds, modifies, and deletes an item from a store is probably self-explanatory. But just in case, Example 9-16 is an adaptation of Example 9-13.

Example 9-16. Using the Notification API to hook events to ItemFileWriteStore
/* Begin notification handlers */
coffeeStore.onNew = function(item, parentItem) {
    var itemName = coffeeStore.getValue(item, "name");
    console.log("Just added", itemName, "which had parent", parentItem);
}

coffeeStore.onSet = function(item, attr, oldValue, newValue) {
    var itemName = coffeeStore.getValue(item, "name");
    console.log("Just modified the ", attr, "attribute for", itemName);

    /* Since children is a multi-valued attribute, oldValue and newValue are
       Arrays that you can iterate over and inspect though often times, you'll 
       only send newValue to the server to log the update */
}

coffeeStore.onDelete = function(item) {
    // coffeeStore.isItem(item) returns false, so don't try to access the item
    console.log("Just deleted", item);
}
/* End notification handlers */


/* Code that uses the notification handlers follows... */

//Add a top level item - triggers a notification
var newItem = coffeeStore.newItem({
    name : "Really Dark",
    description : "Left brewing in the pot all day...extra octane."
});

coffeeStore.fetchItemByIdentity({
    identity : "Dark Roasts",
    onItem : function(item, request) {
        var darkRoasts = item;

        var darkRoastChildren = coffeeStore.getValues(darkRoasts, "children");

        //Modify it - triggers a notification 
        coffeeStore.setValues(darkRoasts, 
            "children",darkRoastChildren.concat(newItem)
        )

        //And now delete it - triggers two notifications
        coffeeStore.deleteItem(newItem)
    }
});

The output you see when you run the example should be something like the following:

Just added Really Dark, which had parent null
Just modified the children attribute for Dark Roasts
Just modified the children attribute for Dark Roasts
Just deleted Object _0=13 name=[1] _RI=true description=[1]

In other words, you get the expected notification when you create the top-level item, a notification for modifying another item's children attribute when you assign the new item as a child, another notification when you remove the child item, and a final notification when you delete the item.

One subtlety to note about Example 9-16 is that the item reference you receive in the onDelete notification has already been removed from the store, so its utility is likely to be somewhat limited since you cannot legally use it in routine store operations.

Serializing and Deserializing Custom Data Types

Although not mentioned until now, you should be aware of one additional feature provided by ItemFileReadStore and ItemFileWriteStore : the ability to pack and unpack custom data types. The motivation for using a type map is that it may often be the case that you need to deal with attributes that aren't primitives, object literals, or arrays. In these circumstances, you're left with manually building up the attributes yourself—introducing cruft in your core logic—or streamlining the situation by tucking away the serialization logic elsewhere.

Implicit type-mapping

Implicit type-mapping for an ItemFileReadStore happens automatically if two special attributes, _type and _value, exist in the data; _type identifies a specific constructor function that should be invoked, which gets passed the _value. JavaScript Date objects are an incredibly common data type that can benefit from being type-mapped; a sample item from our existing data set that has been modified to make use of a date value might look like Example 9-17.

Example 9-17. Using a custom type map to deserialize a value
...
{
    name : "Light Cinnamon",
    description : "Very light brown, dry , tastes like toasted grain with
distinct sour tones, baked, bready"
    lastBrewed : {
        '_type' : "Date",
        '_value':"2008-06-15T00:00:00Z"}
    }
}
...

It almost looks too easy, but assuming that the Date constructor function is defined, that's it! Once the data is deserialized, any values for lastBrewed are honest to goodness Date objects—not just String representations:

var coffeeItem;
coffeeStore.fetchItemByIdentity({
    identity : "Light Cinnamon",
    onItem : function(item, request) {
        coffeeItem = item;
    }
});
coffeeStore.getValue(coffeeItem, "lastBrewed"); //A real Date object

Custom type maps

Alternatively, you can define a JavaScript object and provide a named deserialize function and a type parameter that could be used to construct the value. For ItemFileWriteStore, a serialize function is also available. Following along with the example of managing Date objects, a JavaScript object presenting a valid type map that could be passed in upon construction of the ItemFileWriteStore follows in Example 9-18.

Example 9-18. Passing in a custom type map to an ItemFileReadStore
dojo.require('dojo.date');
dojo.addOnLoad(function(  ) {
    var map = {
        "Date": {
            type: Date,
            deserialize: function(value){
                return dojo.date.stamp.fromISOString(value);
            },
            serialize: function(object){
                return dojo.date.stamp.toISOString(object);
            }
        }
    };

    coffeeStore = new dojo.data.ItemFileReadStore({
        url:"coffee.json",
        typeMap : map
    });
});

Although we intentionally did not delve into dojox.data subprojects in this chapter, it would have been cheating not to at least provide a good reference for using the dojox.data.QueryReadStore, which is the canonical means of interfacing to very large server-side data sources. See http://www.oreillynet.com/onlamp/blog/2008/04/dojo_goodness_part_6_a_million.html for a concise example of using this store along with a custom server routine. This particular example illustrates how to efficiently serve up one million records in the famed DojoX Grid widget.

Summary

After reading this section, you should:

  • Be familiar with the dojo.data APIs and understand the basic value provided by each of them

  • Understand that the Read, Identity, Write, and Notification APIs are abstract, and that any implementation is possible

  • Be aware that the dojox.data subproject provides several really useful custom stores that can save you time accomplishing common tasks such as interfacing to a store of comma-separated values, Flickr, and so on

  • Be aware that the toolkit provides ItemFileReadStore and ItemFileWriteStore as generic yet powerful dojo.data implementations that you may customize or otherwise use as a basis for a custom implementation

  • Understand the value in using custom type maps to save time manually serializing and deserializing data types

Next, we'll move on to simulated classes and inheritance.