Chapter 13. Network Programming

The network is the soul of Java. Most of what is interesting about Java centers on the potential for dynamic, networked applications. As Java’s networking APIs have matured, Java has also become the language of choice for implementing traditional client/server applications and services. In this chapter, we start our discussion of the java.net package, which contains the fundamental classes for communications and working with networked resources (we’ll finish this discussion in Chapter 14). This chapter next discusses the java.rmi package, which provides Java’s native, high-level, Remote Method Invocation (RMI) facilities. Finally, building on the material in Chapter 12, we complete our discussion of the java.nio package, which is highly efficient for implementing large servers.

The classes of java.net fall into two general categories: the Sockets API for working with low-level Internet protocols and higher-level, web-oriented APIs that work with uniform resource locators (URLs). Figure 13-1 shows the java.net package.

Java’s Sockets API provides access to the standard network protocols used for communications between hosts on the Internet. Sockets are the mechanism underlying all other kinds of portable networked communications. Sockets are the lowest-level tool in the general networking toolbox—you can use sockets for any kind of communications between client and server or peer applications on the Net, but you have to implement your own application-level protocols for handling and interpreting the data. Higher-level networking tools, such as remote method invocation, HTTP, and web services are implemented on top of sockets.

Java RMI is a powerful tool that leverages Java object serialization, allowing you to transparently work with Java objects on remote machines almost as if they were local. With RMI, it is easy to write distributed applications in which clients and servers work with each other’s data as full-fledged Java objects rather than raw streams or packets of data.

The java.net package
Figure 13-1. The java.net package

In contrast to RMI, which can only be used between two Java applications, web services is the term for the more general technology that provides platform-independent, loosely coupled invocation of services on remote servers using web standards such as HTTP and XML. We talk about web services in Chapters 14 and 15 when we discuss programming for the Web.

In this chapter, we’ll provide some simple, practical examples of both high- and low-level Java network programming using sockets and RMI. In Chapter 14, we’ll look at the other half of the java.net package, which lets clients work with web servers and services via URLs. Chapter 15 covers Java servlets and the tools that allow you to write web applications and services for web servers.

Sockets

Sockets are a low-level programming interface for networked communications. They send streams of data between applications that may or may not be on the same host. Sockets originated in BSD Unix and are, in some programming languages, hairy, complicated things with lots of small parts that can break off and endanger little children. The reason for this is that most socket APIs can be used with almost any kind of underlying network protocol. Since the protocols that transport data across the network can have radically different features, the socket interface can be quite complex.[37]

The java.net package supports a simplified, object-oriented socket interface that makes network communications considerably easier. If you’ve done network programming using sockets in other languages, you should be pleasantly surprised at how simple things can be when objects encapsulate the gory details. If this is the first time you’ve come across sockets, you’ll find that talking to another application over the network can be as simple as reading a file or getting user input. Most forms of I/O in Java, including most network I/O, use the stream classes described in Chapter 12. Streams provide a unified I/O interface so that reading or writing across the Internet is similar to reading or writing on the local system. In addition to the stream-oriented interfaces, the Java networking APIs can work with the Java NIO buffer-oriented API for highly scalable applications. We’ll see both in this chapter.

Java provides sockets to support three distinct classes of underlying protocols: Sockets, DatagramSockets, and MulticastSockets. In this first section, we look at Java’s basic Socket class, which uses a connection-oriented and reliable protocol. A connection-oriented protocol provides the equivalent of a telephone conversation. After establishing a connection, two applications can send streams of data back and forth and the connection stays in place even when no one is talking. Because the protocol is reliable, it also ensures that no data is lost (resending data as necessary) and that whatever you send always arrives in the order in which you sent it.

In the next section, we look at the DatagramSocket class, which uses a connectionless, unreliable protocol. A connectionless protocol is like the postal service. Applications can send short messages to each other, but no end-to-end connection is set up in advance and no attempt is made to keep the messages in order. It’s not even guaranteed that the messages will arrive at all. A MulticastSocket is a variation of a DatagramSocket that performs multicasting—simultaneously sending data to multiple recipients. Working with multicast sockets is very much like working with datagram sockets. However, because multicasting is currently not widely supported across the Internet, we do not cover it here.

In theory, just about any protocol can be used underneath the socket layer (old-schoolers will remember things like Novell’s IPX, Apple’s AppleTalk, etc.). But in practice, there’s only one important protocol family used on the Internet, and only one protocol family that Java supports: the Internet Protocol (IP). The Socket class speaks TCP, the connection-oriented flavor of IP, and the DatagramSocket class speaks UDP, the connectionless kind.

Clients and Servers

When writing network applications, it’s common to talk about clients and servers. The distinction is increasingly vague, but the side that initiates the conversation is usually considered the client. The side that accepts the request is usually the server. In the case where two peer applications use sockets to talk, the distinction is less important, but for simplicity we’ll use this definition.

For our purposes, the most important difference between a client and a server is that a client can create a socket to initiate a conversation with a server application at any time, while a server must be prepared in advance to listen for incoming conversations. The java.net.Socket class represents one side of an individual socket connection on both the client and server. In addition, the server uses the java.net.ServerSocket class to listen for new connections from clients. In most cases, an application acting as a server creates a ServerSocket object and waits, blocked in a call to its accept() method, until a connection arrives. When it arrives, the accept() method creates a Socket object that the server uses to communicate with the client. A server may carry on conversations with multiple clients at once; in this case, there is still only a single ServerSocket, but the server has multiple Socket objects—one associated with each client, as shown in Figure 13-2.

At the socket level, a client needs two pieces of information to locate and connect to a server on the Internet: a hostname (used to find the host computer’s network address) and a port number. The port number is an identifier that differentiates between multiple clients or servers on the same host. A server application listens on a prearranged port while waiting for connections. Clients use the port number assigned to the service they want to access. If you think of the host computers as hotels and the applications as guests, the ports are like the guests’ room numbers. For one person to call another, he or she must know the other party’s hotel name and room number.

Clients and servers, Sockets and ServerSockets
Figure 13-2. Clients and servers, Sockets and ServerSockets

Clients

A client application opens a connection to a server by constructing a Socket that specifies the hostname and port number of the desired server:

    try {
        Socket sock = new Socket("wupost.wustl.edu", 25);
    } catch ( UnknownHostException e ) {
        System.out.println("Can't find host.");
    } catch ( IOException e ) {
        System.out.println("Error connecting to host.");
    }

This code fragment attempts to connect a Socket to port 25 (the SMTP mail service) of the host wupost.wustl.edu. The client handles the possibility that the hostname can’t be resolved (UnknownHostException) and that it might not be able to connect to it (IOException). In the preceding case, Java used DNS, the standard Domain Name Service, to resolve the hostname to an IP address for us. The constructor can also accept a string containing the host’s raw IP address:

    Socket sock = new Socket("22.66.89.167", 25);

After a connection is made, input and output streams can be retrieved with the Socket getInputStream() and getOutputStream() methods. The following (rather arbitrary) code sends and receives some data with the streams:

    try {
        Socket server = new Socket("foo.bar.com", 1234);
        InputStream in = server.getInputStream();
        OutputStream out = server.getOutputStream();

        // write a byte
        out.write(42);

        // write a newline or carriage return delimited string
        PrintWriter pout = new PrintWriter( out, true );
        pout.println("Hello!");

        // read a byte
        byte back = (byte)in.read();

        // read a newline or carriage return delimited string
        BufferedReader bin =
          new BufferedReader( new InputStreamReader( in ) );
        String response = bin.readLine();

        // send a serialized Java object
        ObjectOutputStream oout = new
        ObjectOutputStream( out );
        oout.writeObject( new java.util.Date() );
        oout.flush();
      
        server.close();
    }
    catch (IOException e ) { ... }

In this exchange, the client first creates a Socket for communicating with the server. The Socket constructor specifies the server’s hostname (foo.bar.com) and a prearranged port number (1234). Once the connection is established, the client writes a single byte to the server using the OutputStream’s write() method. To send a string of text more easily, it then wraps a PrintWriter around the OutputStream. Next, it performs the complementary operations: reading a byte back from the server using InputStream’s read() method and then creating a BufferedReader from which to get a full string of text. Finally, we do something really funky and send a serialized Java object to the server, using an ObjectOutputStream. (We’ll talk in depth about sending serialized objects later in this chapter.) The client then terminates the connection with the close() method. All these operations have the potential to generate IOExceptions; our application will deal with these using the catch clause.

Servers

After a connection is established, a server application uses the same kind of Socket object for its side of the communications. However, to accept a connection from a client, it must first create a ServerSocket, bound to the correct port. Let’s recreate the previous conversation from the server’s point of view:

    // Meanwhile, on foo.bar.com...
    try {
        ServerSocket listener = new ServerSocket( 1234 );

        while ( !finished ) {
            Socket client = listener.accept();  // wait for connection

            InputStream in = client.getInputStream();
            OutputStream out = client.getOutputStream();

            // read a byte
            byte someByte = (byte)in.read();

            // read a newline or carriage-return-delimited string
            BufferedReader bin =
              new BufferedReader( new InputStreamReader( in ) );
            String someString = bin.readLine();

            // write a byte
            out.write(43);

            // say goodbye
            PrintWriter pout = new PrintWriter( out, true );
            pout.println("Goodbye!");
      
            // read a serialized Java object
            ObjectInputStream oin = new ObjectInputStream( in );
            Date date = (Date)oin.readObject();

            client.close();
        }

        listener.close();
    }
    catch (IOException e ) { ... }
    catch (ClassNotFoundException e2 ) { ... }

First, our server creates a ServerSocket attached to port 1234. On some systems, there are rules about which ports an application can use. Port numbers below 1024 are usually reserved for system processes and standard, well-known services, so we pick a port number outside of this range. The ServerSocket is created only once; thereafter, we can accept as many connections as arrive.

Next, we enter a loop, waiting for the accept() method of the ServerSocket to return an active Socket connection from a client. When a connection has been established, we perform the server side of our dialog, then close the connection and return to the top of the loop to wait for another connection. Finally, when the server application wants to stop listening for connections altogether, it calls the close() method of the ServerSocket.

This server is single-threaded; it handles one connection at a time, not calling accept() to listen for a new connection until it’s finished with the current connection. A more realistic server would have a loop that accepts connections concurrently and passes them off to their own threads for processing. There is a lot to be said about implementing multithreaded servers. Later in this chapter, we’ll create a tiny web server that starts a new thread for each connection and also a slightly more complex web server that uses the NIO package to handle many connections with a small number of threads.

Sockets and security

The previous examples presuppose that the client has permission to connect to the server and that the server is allowed to listen on the specified socket. If you’re writing a general, standalone application, this is normally the case (and you can probably skip this section). However, untrusted applications (such as Java applets in a web browser) run under the auspices of a security policy that can impose arbitrary restrictions on what hosts they may or may not talk to and whether or not they can listen for connections.

For example, the security policy imposed on applets by most browsers allow untrusted applets to open socket connections only to the host that served them—that is, they can talk back only to the server from which their class files were retrieved. Untrusted applets are generally not allowed to open server sockets for incoming connections themselves. This doesn’t mean that an untrusted applet can’t cooperate with its server to communicate with anyone, anywhere. The applet’s server could run a proxy that lets the applet communicate indirectly with anyone it likes. What this security policy prevents is malicious applets poking around inside corporate firewalls, making connections to trusted services. It places the burden of security on the originating server, not the client machine. Restricting access to the originating server limits the usefulness of Trojan applications that do annoying things from the client side. (You probably won’t let your proxy spam people, because you’ll be blamed.)

If you are going to run your own application under a security manager, you should be aware that the default security manager dissallows all network access. So in order to make network connections, you would have to modify your policy file to grant the appropriate permissions to your code (see Chapter 3 for details). The following policy file fragment sets the socket permissions to allow connections to or from any host on any nonprivileged port:

    grant {
      permission java.net.SocketPermission
        "*:1024-", "listen,accept,connect";
    };

When starting the Java interpreter, you can install the security manager and use this file (call it mysecurity.policy):

    % java -Djava.security.manager  
    \-Djava.security.policy=mysecurity.policy MyApplication

The DateAtHost Client

In the past, many networked computers ran a simple time service that dispensed their clock’s local time on a well-known port. This was a precursor of NTP, the more general Network Time Protocol. The next example, DateAtHost, includes a subclass of java.util.Date that fetches the time from a remote host instead of initializing itself from the local clock. (See Chapter 11 for a complete discussion of the Date class.)

DateAtHost connects to the time service (port 37) and reads four bytes representing the time on the remote host. These four bytes have a peculiar specification that we decode to get the time. Here’s the code:

    //file: DateAtHost.java
    import java.net.Socket;
    import java.io.*;
      
    public class DateAtHost extends java.util.Date {
        static int timePort = 37;
        // seconds from start of 20th century to Jan 1, 1970 00:00 GMT
        static final long offset = 2208988800L;

        public DateAtHost( String host ) throws IOException {
            this( host, timePort );
        }
      
        public DateAtHost( String host, int port ) throws IOException {
            Socket server = new Socket( host, port );
            DataInputStream din =
              new DataInputStream( server.getInputStream() );
            int time = din.readInt();
            server.close();

            setTime( (((1L << 32) + time) - offset) * 1000 );
        }
    }

That’s all there is to it. It’s not very long, even with a few frills. We have supplied two possible constructors for DateAtHost. Normally we’d expect to use the first, which simply takes the name of the remote host as an argument. The second constructor specifies the hostname and the port number of the remote time service. (If the time service were running on a nonstandard port, we would use the second constructor to specify the alternate port number.) This second constructor does the work of making the connection and setting the time. The first constructor simply invokes the second (using the this() construct) with the default port as an argument. Supplying simplified constructors that invoke their siblings with default arguments is a common and useful pattern in Java; that is the main reason we’ve shown it here.

The second constructor opens a socket to the specified port on the remote host. It creates a DataInputStream to wrap the input stream and then reads a four-byte integer using the readInt() method. It’s no coincidence that the bytes are in the right order. Java’s DataInputStream and DataOutputStream classes work with the bytes of integer types in network byte order (most significant to least significant). The time protocol (and other standard network protocols that deal with binary data) also uses the network byte order, so we don’t need to call any conversion routines. Explicit data conversions would probably be necessary if we were using a nonstandard protocol, especially when talking to a non-Java client or server. In that case, we’d have to read byte by byte and do some rearranging to get our four-byte value. After reading the data, we’re finished with the socket, so we close it, terminating the connection to the server. Finally, the constructor initializes the rest of the object by calling Date’s setTime() method with the calculated time value.

The four bytes of the time value are interpreted as an integer representing the number of seconds since the beginning of the 20th century. DateAtHost converts this to Java’s notion of absolute time—the count of milliseconds since January 1, 1970 (an arbitrary date standardized by C and Unix). The conversion first creates a long value, which is the unsigned equivalent of the integer time. It subtracts an offset to make the time relative to the epoch (January 1, 1970) rather than the century, and multiplies by 1,000 to convert to milliseconds. The converted time is used to initialize the object.

The DateAtHost class can work with a time retrieved from a remote host almost as easily as Date is used with the time on the local host. The only additional overhead is dealing with the possible IOException that can be thrown by the DateAtHost constructor:

    try {
        Date d = new DateAtHost( "someserver.net" );
        System.out.println( "The time over there is: " + d );
    }
    catch ( IOException e ) { ... }

This example fetches the time at the host someserver.net and prints its value.

The TinyHttpd Server

Have you ever wanted to write your very own web server? Well, you’re in luck. In this section, we’re going to build TinyHttpd, a minimal but functional web server. TinyHttpd listens on a specified port and services simple HTTP GET requests. GET requests are simple text commands that look something like this:

    GET /path/filename [ optional stuff ]

Your web browser sends one or more of these requests for each document it retrieves from a web server. Upon reading a request, our server attempts to open the specified file and send its contents. If that document contains references to images or other items to be displayed inline, the browser follows up with additional GET requests. For best performance, TinyHttpd services each request in its own thread. Therefore, TinyHttpd can service several requests concurrently.

This example works, but it’s a bit oversimplified. First, it implements a very old subset of the HTTP protocol, so some browsers may turn their nose up at it. (I tested this in Safari on my Mac at the time of this writing and it worked well enough for this example’s purposes.) Also remember that file pathnames are still somewhat architecture-dependent in Java. This example should work as it is on most systems, but would require some enhancement to be more robust. It’s possible to write slightly more elaborate code that uses the environmental information provided by Java to tailor itself to the local system. (Chapter 12 gives some hints about how.)

Unless you have a firewall or other security in place, the next example serves files from your host without protection. Don’t try this at work!

Now, without further ado, here’s TinyHttpd:

    //file: TinyHttpd.java
    import java.net.*;
    import java.io.*;
    import java.util.regex.*;
    import java.util.concurrent.*;
     
    public class TinyHttpd {
      public static void main( String argv[] ) throws IOException {
        Executor executor = Executors.newFixedThreadPool(3);
        ServerSocket ss = new ServerSocket( Integer.parseInt(argv[0]) );
        while ( true )
          executor.execute( new TinyHttpdConnection( ss.accept() ) );
      }
    }

    class TinyHttpdConnection implements Runnable {
      Socket client;
      TinyHttpdConnection ( Socket client ) throws SocketException {
        this.client = client;
      }
      public void run() {
        try {
          BufferedReader in = new BufferedReader(
            new InputStreamReader(client.getInputStream(), "8859_1" ) );
          OutputStream out = client.getOutputStream();
          PrintWriter pout = new PrintWriter(
            new OutputStreamWriter(out, "8859_1"), true );
          String request = in.readLine();
          System.out.println( "Request: "+request);

          Matcher get = Pattern.compile("GET /?(\\S*).*").matcher( request );
          if ( get.matches() ) {
            request = get.group(1);
            if ( request.endsWith("/") || request.equals("") )
              request = request + "index.html";
            try {
              FileInputStream fis = new FileInputStream ( request );
              byte [] data = new byte [ 64*1024 ];
              for(int read; (read = fis.read( data )) > -1; )
                  out.write( data, 0, read );
              out.flush();
            } catch ( FileNotFoundException e ) {
              pout.println( "404 Object Not Found" ); }
          } else
            pout.println( "400 Bad Request" );
          client.close();
        } catch ( IOException e ) {
          System.out.println( "I/O error " + e ); }
       }
    }

Compile TinyHttpd and place it in your classpath, as described in Chapter 3. Go to a directory with some interesting documents and start the server, specifying an unused port number as an argument. For example:

    % java TinyHttpd 1234

You should now be able to use your web browser to retrieve files from your host. You’ll have to specify the port number you chose in the URL. For example, if your hostname is foo.bar.com, and you started the server as shown, you could reference a file as in:

    http://foo.bar.com:1234/welcome.html

Or, more likely, if you’re running both the server and your web browser on the same machine, you could reference a file as in:

    http://localhost:1234/welcome.html

TinyHttpd looks for files relative to its current directory, so the pathnames you provide should be relative to that location. (Retrieved some files? Did you notice that when you retrieved an HTML file, your web browser automatically generated more requests for items like images that were contained within it?) Let’s take a closer look.

The TinyHttpd application has two classes. The public TinyHttpd class contains the main() method of our standalone application. It begins by creating a ServerSocket, attached to the specified port. It then loops, waiting for client connections and creating instances of the second class, a TinyHttpdConnection, to service each request. The while loop waits for the ServerSocket accept() method to return a new Socket for each client connection. The Socket is passed as an argument to construct the TinyHttpdConnection that handles it. We use an Executor with a fixed pool size of three threads to service all of our connections.

TinyHttpdConnection is a Runnable object. For each connection we start a thread, which lives long enough to handle the single client connection and then dies. The body of TinyHttpdConnection’s run() method is where all the magic happens. First, we fetch an OutputStream for talking back to our client. The second line reads the GET request from the InputStream into the variable request. This request is a single newline-terminated String that looks like the GET request we described earlier. For this, we use a BufferedInputStream wrapped around an InputStreamReader. (We’ll say more about the InputStreamReader in a moment.)

We then parse the contents of request to extract a filename. Here, we are using the Regular Expression API (see Chapter 10 for a full discussion of regular expressions and the Regular Expression API). The pattern simply looks for the "GET" followed by an optional slash and then any string of nonwhitespace characters. We add the ".*" at the end to cause the pattern to match the whole input, so that we can use the Matcher match() method to test if the whole request made sense to us or not. The part that matches the filename is in a capture group: "(\\S*)". This allows us to retrieve that text with the Matcher group() method. Finally, we check to see if the requested filename looks like a directory name (i.e., ends in a slash) or is empty. In these cases, we append the familiar default filename index.html as a convenience.

Once we have the filename, we try to open the specified file and send its contents using a large byte array. Here we loop, reading one buffer at a time and writing to the client via the OutputStream. If we can’t parse the request or the file doesn’t exist, we use the PrintStream to send a textual message. Then we return a standard HTTP error message. Finally, we close the socket and return from run(), completing our task.

Do French web servers speak French?

In TinyHttpd, we explicitly created the InputStreamReader for our BufferedRead and the OutputStreamWriter for our PrintWriter. We do this so that we can specify the character encoding to use when converting to and from the byte representation of the HTTP protocol messages. (Note that we’re not talking about the body of the file to be sent—that is simply a stream of raw bytes to us; rather we’re talking about the GET and response messages.) If we didn’t specify, we’d get the default character encoding for the local system. For many purposes that may be correct, but in this case, we are speaking of a well-defined international protocol and we should be specific. The RFC for HTTP specifies that web clients and servers should use the ISO8859-1 character encoding. We specify this encoding explicitly when we construct the InputStreamReader and OutputStreamWriter. As it turns out, ISO8859-1 is just plain ASCII and conversion to and from Unicode should always leave ASCII values unchanged, so again we would probably not be in any trouble if we did not specify an encoding. But it’s important to think about these things at least once—and now you have.

Taming the daemon

An important problem with TinyHttpd is that there are no restrictions on the files it serves. With a little trickery, the daemon would happily send any file on your computer to the client. It would be nice if we could enforce the restriction that TinyHttpd serve only files that are in the current working directory or a subdirectory, as it normally does. An easy way to do this is to activate the Java Security Manager. Normally, a security manager is used to prevent Java code downloaded over the Net from doing anything suspicious. However, we can press the security manager into service to restrict file access in our application as well.

You can use a policy like the simple one that we provided earlier in this chapter; it allows the server to accept connections on a specified range of sockets. Fortuitously, the default file-access granted by the security policy does just what we‘d like for this example: it allows an application access to files in its current working directory and subdirectories. So simply installing the security manager provides exactly the kind of file protection that we wanted in this case. (It would be easy to add additional permissions if you wish to extend the server’s range to other well-defined areas.)

With the security manager in place, the daemon cannot access anything outside the current directory and its subdirectories. If it tries to, the security manager throws an exception and prevents access to the file. In that case, we should have TinyHttpd catch the SecurityException and return a proper message to the web browser. Add the following catch clause after the FileNotFoundException’s catch clause:

    ...
    } catch ( Security Exception e ) {
        pout.println("403 Forbidden");
    }

Room for improvement

TinyHttpd still has quite a bit of room for improvement. Technically, it implements only an obsolete subset of the HTTP protocol (version 0.9) in which the server expects only the GET request and returns just the content. All modern servers speak HTTP 1.1, which allows for additional metadata in both the HTTP request and response and requires certain data such as version number and content length. You could extend our simple server to add these extra bits of the header without a great deal of effort. HTTP 1.1 also has more sophisticated features such as allowing multiple client requests to be sent over one socket connection, which requires more elaborate code.

Of course, real web servers can do all sorts of other things. For example, you might consider adding a few lines of code to read directories and generate linked HTML listings as most web servers do. Have fun with this example, and you can learn quite a bit.

Socket Options

As we’ve said, the Java Sockets API is a somewhat simplified interface to the general socket mechanisms. In other environments, where all the gory details of the network are visible to you, a lot of complex and sometimes esoteric options can be set on sockets to govern the behavior of the underlying protocols. Java gives us access to a few important ones.

SO_TIMEOUT

The SO_TIMEOUT option sets a timer on all I/O methods of a socket that block so that you don’t have to wait forever if they don’t return. This works for operations such as accept() on server sockets and read() or write() on all sockets. If the timer expires before the operation would complete, an InterruptedIOException is thrown. You can catch the exception and continue to use the socket normally if it is appropriate, or you can take the opportunity to bail out of the operation. Multithreaded, blocking servers, such as TinyHttpd, can use this sort of technique for their shutdown logic:

    serverSocket.setSoTimeout( 2000 ); // 2 seconds

    while ( !shutdown ) {
        try {
            Socket client = serverSocket.accept();
            handleClient( client );
        } catch ( InterruptedIOException e ) {
            // ignore the exception
        }

        // exit
    }

You set the timer by calling the setSoTimeout() method of the Socket class with the timeout period, in milliseconds, as an int argument. This works for regular Sockets, ServerSockets (TCP), and DatagramSockets (UDP), discussed later in this chapter. To find the current timeout value, call getSoTimeout() .

This feature is a workaround for the fact that stream-oriented I/O operations in Java are blocking, and there is no way to test or poll them for activity. Later in this chapter, we’ll complete our discussion of the NIO package, which provides full nonblocking I/O for all types of operations, including sockets.

TCP_NODELAY

This option turns off a feature of TCP called Nagle’s algorithm, which tries to prevent certain interactive applications from flooding the network with very tiny packets. For example, in our very first network example we sent a single byte to the network in one write. With this option on, under certain conditions, the TCP implementation might have decided to hold that byte for a very brief period, hoping for more data to fill the next packet. You can turn this “delay” off if you have a fast network and you want all packets sent as soon as possible. The Socket setTcpNoDelay() method takes a Boolean argument specifying whether the delay is on or off. To find out whether the TCP_NODELAY option is enabled, call getTcpNoDelay(), which returns a boolean.

SO_LINGER

This option controls what happens to any unsent data when you perform a close() on an active socket connection. Normally, the system blocks on the close and tries to deliver any network buffered data and close the connection gracefully. The setSoLinger() method of the Socket class takes two arguments: a boolean that enables or disables the option and an int that sets the time to wait (the linger value), in seconds. If you set the linger value to 0, any unsent data is discarded and the TCP connection is aborted (terminated with a reset). To find the current linger value, call getSoLinger().

TCP_KEEPALIVE

This option can be enabled with the setKeepAlive() method. It triggers a feature of TCP that polls the other side every two hours if there is no other activity. Normally, when no data is flowing on a TCP connection, no packets are sent. This can make it difficult to tell whether the other side is simply being quiet or has disappeared. If one side of the connection closes properly, this is detected. But if the other side simply disappears, we don’t know unless and until we try to talk to it. For this reason, servers often use TCP_KEEPALIVE to detect lost client connections (where they might otherwise only respond to requests, rather than initiate them). Keepalive is not part of the TCP specification; it’s an add-on that’s not guaranteed to be implemented everywhere. If you can, the best way to detect lost clients is to implement the polling as part of your own protocol.

Half-close

In TCP, it is technically possible to close one direction of a stream but not the other. In other words, you can shut down sending but not receiving, or vice versa. A few protocols use this to indicate the end of a client request by closing the client side of the stream, allowing the end of stream to be detected by the server. You can shut down either half of a socket connection with shutdownOutput() or shutdownInput().

Proxies and Firewalls

Most networks today are behind firewalls. Some firewalls not only prevent outsiders from getting in, but by default, prevent applications inside the firewall from opening direct socket-level connections to the outside network. Instead, firewalls that do this often provide a service called SOCKS (named for sockets) that acts as a proxy server for socket connections, giving the administrators more control over what connections are allowed. Firewalls may also be set up with direct proxies for higher-level protocols, such as HTTP and FTP, which allow even greater control and possibly screening of content. It’s all about attempting to control who connects to whom and for what.

If your firewall does not allow any direct outside socket connections, even via SOCKS, your application may still be able to communicate with the outside world by using HTTP to send and receive data in this way. See Chapter 14 for an example of how to perform HTTP POST and GET operations to send and retrieve data through firewalls that allow web traffic.

Java has built-in support for SOCKS as well as HTTP and FTP protocol proxies. All you have to do is set some system properties in your application (in an applet, this should be already taken care of for you through your browser configuration, because you wouldn’t have authority to set those properties). To configure Java to use a SOCKS (version 4 or 5) proxy server, set the following system properties:

socksProxyHost

The SOCKS proxy server name.

socksProxyPort

The SOCKS proxy port number.

If the SOCKS proxy requires a username and password, you can supply them in the additional properties java.net.socks.username and java.net.socks.password.

It’s similar for HTTP and FTP proxies, which are set with separate properties:

http.proxyHost

ftp.proxyHost

The proxy server name.

http.proxyPort

The proxy port number.

http.nonProxyHosts

A list of hosts for which direct connections should always be made. Hosts in the list are separated by vertical bars (|) and may include asterisks (*) as wildcards—for example, myserver|*.mydomain.

You can set these properties on the command line using the Java interpreter’s -D option or by calling the System.setProperty() method. The following command runs MyProgram using the HTTP proxy server at foo.bar.com on port 1234:

    % java -Dhttp.proxyServer=foo.bar.com -Dhttp.proxyPort=1234 MyProgram

ProxySelector

Java also has an API to allow programmatic control of Java’s use of proxies. The java.net.ProxySelector class has a method that takes a uniform resource identifier (URI) object (a generalization of the URLs we use for web addresses; see Chapter 14) and returns a list of java.net.Proxy objects representing the proxies or direct connections to be used for the protocol specified. The default ProxySelector obeys the system properties we listed earlier. If required, you can create and install your own proxy selector to take control of the process and direct different destinations to different proxies. To see what decisions are being made, you can get the default selector with the static method ProxySelector.getDefault() and query it for various protocols with its select() method. The following example prints some string information about the preferred proxy (if any) for a specific HTTP URL:

    ProxySelector ps = java.net.ProxySelector.getDefault();
    List list = ps.select( new URI("http://java.sun.com/") );
    System.out.println( list.get(0) );//e.g. HTTP@myserver:1234

Detailed information can be obtained from the proxy object, which contains a type identifier specifying DIRECT, HTTP, or SOCKS and a proxy address. To query for a SOCKS socket proxy for a given host and port, use a URI string of the form socket://host:port.

Datagram Sockets

TinyHttpd used a Socket to create a connection to the client using the TCP protocol. In that example, the TCP protocol took care of data integrity; we didn’t have to worry about data arriving out of order or incorrect. Now, let’s take a walk on the wild side, building an applet that uses a java.net.DatagramSocket, which uses the UDP protocol. A datagram is sort of like a letter sent via the postal service: it’s a discrete chunk of data transmitted in one packet. Unlike the previous example, where we could get a convenient OutputStream from our Socket and write the data as if writing to a file with a DatagramSocket, we have to work one datagram at a time. (Of course, the TCP protocol was taking our OutputStream and slicing the data into packets, too, but we didn’t have to worry about those details.)

UDP doesn’t guarantee that the data is received. If the data packets are received, they may not arrive in the order in which they were sent; it’s even possible for duplicate datagrams to arrive (under rare circumstances). Using UDP is something like cutting the pages out of the encyclopedia, putting them into separate envelopes, and mailing them to your friend. If your friend wants to read the encyclopedia, it’s his or her job to put the pages in order. If some pages get lost in the mail, your friend has to send you a letter asking for replacements.

Obviously, you wouldn’t use UDP to send a huge amount of data without error correction. However, it’s significantly more efficient than TCP, particularly if you don’t care about the order in which messages arrive or whether 100 percent of their arrival is guaranteed. For example, in a simple periodic database lookup, the client can send a query; the server’s response itself constitutes an acknowledgment. If the response doesn’t arrive within a certain time, the client can just send another query. It shouldn’t be hard for the client to match responses to its original queries. Some important applications that use UDP are the Domain Name System (DNS) and Sun’s Network File System (NFS).

The HeartBeat Applet

In this section, we build a simple applet, HeartBeat, that runs in a web browser and sends a datagram to its server each time it’s started and stopped. We also build a simple standalone server application, Pulse, that receives these datagrams and prints them. Tracking the output could give you a crude measure of who is currently looking at your web page at any given time (assuming that firewalls do not block the UDP packets). This is the kind of thing UDP is good for: we don’t want the overhead of a TCP socket, and if the datagrams get lost, it’s no big deal.

First, the HeartBeat applet:

    //file: HeartBeat.java
    import java.net.*;
    import java.io.*;
      
    public class HeartBeat extends java.applet.Applet {
        String myHost;
        int myPort;
      
        public void init() {
            myHost = getCodeBase().getHost();
            myPort = Integer.parseInt( getParameter("myPort") );
        }
      
        private void sendMessage( String message ) {
            try {
                byte [] data = message.getBytes("UTF-8");
                InetAddress addr = InetAddress.getByName( myHost );
                DatagramPacket packet =
                  new DatagramPacket( data, data.length, addr, myPort );
                DatagramSocket ds = new DatagramSocket();
                ds.send( packet );
                ds.close();
            } catch ( IOException e ) {
                System.out.println( e );  // Error creating socket
            }
        }
      
        public void start() {
            sendMessage("Arrived");
        }
        public void stop() {
            sendMessage("Departed");
        }
    }

Compile the applet and include it in an HTML document with an <applet> tag:

<html><body>
    <h1>Heartbeat!</h1>
    <applet height="1 " width="1 " code="HeartBeat ">
        <param name="myPort" value="1234">
    </applet>
    </body></html>

Make sure to place the compiled HeartBeat.class file in the same directory as the HTML document (which we’ll refer to as heartbeat.html). We talk more about embedding applets in HTML documents in Chapter 23.

The myPort parameter should specify the port number on which our server application listens for data (“1234” as just shown).

Next, the server-side application, Pulse:

    //file: Pulse.java
    import java.net.*;
    import java.io.*;
      
    public class Pulse {
        public static void main( String [] argv ) throws IOException {
            DatagramSocket s =
              new DatagramSocket( Integer.parseInt(argv[0]) );
      
            while ( true ) {
                DatagramPacket packet =
                  new DatagramPacket( new byte [1024], 1024 );
                s.receive( packet );
                String message = new String(
                    packet.getData(), 0, packet.getLength(),"UTF-8" );
                System.out.println( "Heartbeat from: "
                  + packet.getAddress().getHostName()
                  + " - " + message );
            }
        }
    }

Compile Pulse and run it on your web server, specifying the port number as an argument:

    % java Pulse 1234

The port number should be the same as the one you used in the myPort parameter of the <applet> tag for HeartBeat.

Now to run the example we’re going to need a web server. Opening the file directly in your browser will not work here because, as we mentioned earlier, applets are only allowed to talk to the host that served them and when no web server is involved, the security manager doesn’t allow any network communications at all. Fortunately, we wrote a satisfactory web server, TinyHttpd, earlier in this chapter! Just run TinyHttpd in the directory with your heartbeat.html file—being careful to specify a different port number than our HeartBeat client and Pulse server—and use it to serve up the page. We tested this in Safari on a Mac at the time of this writing and it worked as expected. But if you have issues you can try a “real” web server and we won’t be offended.

Now, pull up the web page in your browser. You won’t see much there (a better application might do something visual as well), but you should get a blip from the Pulse application. Leave the page and return to it a few times. Each time the applet is started or stopped, it sends a message that Pulse reports:

    Heartbeat from: foo.bar.com - Arrived
    Heartbeat from: foo.bar.com - Departed
    Heartbeat from: foo.bar.com - Arrived
    Heartbeat from: foo.bar.com - Departed
    ...

Cool, eh? Just remember that the datagrams are not guaranteed to arrive (although it’s highly unlikely you’ll ever see them fail on a normal network), and it’s possible that you could miss an arrival or a departure. Now let’s look at the code.

The HeartBeat applet code

HeartBeat overrides the init(), start(), and stop() methods of the Applet class and implements one private method of its own, sendMessage(), which sends a datagram. (We haven’t covered applets yet, so if you want more details, refer to Chapter 23.) HeartBeat begins its life in init(), where it determines the destination for its messages. It uses the Applet getCodeBase() and getHost() methods to find the name of its originating host and fetches the correct port number from the myPort parameter of the <applet> tag. After init() has finished, the start() and stop() methods are called whenever the applet is started or stopped. These methods merely call sendMessage() with the appropriate message.

sendMessage() is responsible for sending a String message to the server as a datagram. It takes the text as an argument, constructs a datagram packet containing the message, and then sends the datagram. All the datagram information is packed into a java.net.DatagramPacket object, including the destination and port number. The DatagramPacket is like an addressed envelope, stuffed with our bytes. After the DatagramPacket is created, sendMessage() simply has to open a DatagramSocket and send it.

The first five lines of sendMessage() build the DatagramPacket:

    try {
        byte [] data = message.getBytes("UTF-8");
        InetAddress addr = InetAddress.getByName( myHost );
        DatagramPacket pack =
          new DatagramPacket(data, data.length, addr, myPort );

First, the contents of message are placed into an array of bytes called data. Next a java.net.InetAddress object is created by looking up the hostname myHost. An InetAddress holds the network IP address for a host. This class also provides the static getByName() method for looking up an IP address by hostname using the system’s name service. (We’ll say more about InetAddress in the next section.) Finally, we call the DatagramPacket constructor with four arguments: the byte array containing our data, the length of the data, the destination address object, and the port number. We ask for the string to be encoded using the UTF-8 charset; we’ll use the same character set to decode it.

The remaining lines construct a default client DatagramSocket and call its send() method to transmit the DatagramPacket. After sending the datagram, we close the socket:

    DatagramSocket ds = new DatagramSocket();
    ds.send( pack );
    ds.close();

Two operations throw a type of IOException: the InetAddress.getByName() lookup and the DatagramSocket send() method. InetAddress.getByName() can throw an UnknownHostException, which is a type of IOException that indicates the hostname can’t be resolved. If send() throws an IOException, it implies a serious client-side communication problem with the network. We need to catch these exceptions; our catch block simply prints a message telling us that something went wrong. If we get one of these exceptions, we can assume the datagram never arrived. However, we can’t assume the inverse: even if we don’t get an exception, we still don’t know that the host is actually accessible or that the data actually arrived; with a DatagramSocket, we never find out from the API.

The Pulse server code

The Pulse server corresponds to the HeartBeat applet. First, it creates a DatagramSocket to listen on our prearranged port. This time, we specify a port number in the constructor; we get the port number from the command line as a string (argv[0]) and convert it to an integer with Integer.parseInt(). Note the difference between this call to the constructor and the call in HeartBeat. In the server, we need to listen for incoming datagrams on a prearranged port, so we need to specify the port when creating the DatagramSocket. The client just sends datagrams, so we don’t have to specify the port in advance; we build the port number into the DatagramPacket itself.

Second, Pulse creates an empty DatagramPacket of a fixed maximum size to receive an incoming datagram. This form of the constructor for DatagramPacket takes a byte array and a length as arguments. As much data as possible is stored in the byte array when it’s received. (A practical limit on the size of a UDP datagram that can be sent over the Internet is 8K, although datagrams can be larger for local network use—theoretically up to 64K.) Finally, Pulse calls the DatagramSocket’s receive() method to wait for a packet to arrive. When a packet arrives, its contents are printed by turning them to a string using UTF-8 encoding. We determine the actual number of received bytes from the packet’s getLength() method.

As you can see, DatagramSockets are slightly more tedious than regular Sockets. With datagrams, it’s harder to spackle over the messiness of the socket interface. The Java API rather slavishly follows the Unix interface, and that doesn’t help. But all in all, it’s not that hard to use datagrams for simple tasks.

InetAddress

The java.net.InetAddress class is the lowest-level Java API for working with IP addresses. Instances of InetAddress represent individual addresses and the InetAddress class provides the API for using the platform’s name service to map a string hostname to a numeric IP address. Most of our networking examples showed the use of hostnames to identify remote servers, but under the covers, Java utilized the static InetAddress.getByName() method to map the name to a physical IP address. Java normally uses the DNS to perform this lookup (and it caches the results for efficiency). Most Java networking APIs (such as Sockets) will accept either a hostname or an InetAddress as a destination. The InetAddress class can also be used to perform reverse IP lookups (get a name for an IP address) as well as to find the primary address of the local host via the static InetAddress.getLocalHost() method.

A useful feature of InetAddress is the method isReachable(), which attempts to use the ICMP ping protocol to determine whether a remote address can be reached over the network. The ping protocol is the standard mechanism used to check reachability and latency on networks. It is a low-level IP protocol (along with TCP and UDP) and is not guaranteed to be supported everywhere. If isReachable() can’t use ICMP, it attempts to use TCP to reach the echo service (port 7) on the remote host. For example:

    InetAddress server = InetAddress.getByName("myserver");
    If ( !server.isReachable( timeout ) ) // milliseconds
        pageSomeone();

Simple Serialized Object Protocols

Earlier in this chapter, we showed a hypothetical conversation in which a client and server exchanged some primitive data and a serialized Java object. Passing an object between two programs may not have seemed like a big deal at the time, but, in the context of Java as a portable bytecode language, it has big implications. In this section, we show how a protocol can be built using serialized Java objects.

Before we move on, it’s worth considering network protocols. Most programmers would consider working with sockets to be tedious and complex. Even though Java makes sockets much easier to use than many other languages, sockets still provide only an unstructured flow of bytes between their endpoints. If you want to do serious communications using sockets, the first thing you have to do is come up with a protocol that defines the data you are sending and receiving. The most complex part of that protocol usually involves how to marshal (package) your data for transfer over the Net and unpack it on the other side.

As we’ve seen, Java’s DataInputStream and DataOuputStream classes solve this problem for simple data types. We can read and write numbers, Strings, and Java primitives in a standard format that can be understood on any other Java platform. To do real work, however, we need to be able to put simple types together into larger structures. Java object serialization solves this problem elegantly by allowing us to send our data in the state in which we will use it—as Java objects. Serialization can even pack up entire graphs of interconnected objects and put them back together at a later time in another Java VM.

A Simple Object-Based Server

In the following example, a client sends a serialized object to the server, and the server responds in kind. The object sent by the client represents a request and the object returned by the server represents the response. The conversation ends when the client closes the connection. It’s hard to imagine a simpler protocol. All the hairy details are taken care of by object serialization, which allows us to work with standard Java objects as we are used to doing.

To start, we define a class—Request—to serve as a base class for the various kinds of requests we make to the server. Using a common base class is a convenient way to identify the object as a type of request. In a real application, we might also use it to hold basic information, such as client names and passwords, timestamps, serial numbers, and so on. In our example, Request can be an empty class that exists so that others can extend it:

    //file: Request.java
    public class Request implements java.io.Serializable {}

Request implements Serializable, so all its subclasses are serializable by default. Next, we create some specific kinds of Requests. The first, DateRequest, is also a trivial class. We use it to ask the server to send us a java.util.Date object as a response:

    //file: DateRequest.java
    public class DateRequest extends Request {}

Next, we create a generic WorkRequest object. The client sends a WorkRequest to get the server to perform some computation. The server calls the WorkRequest object’s execute() method to do the work on the server side and then returns the resulting object as a response:

    //file: WorkRequest.java
    public abstract class WorkRequest extends Request {
        public abstract Object execute();
    }

For our application, we subclass WorkRequest to create MyCalculation, which adds code that performs a specific calculation; in this case, we just square a number:

    //file: MyCalculation.java
    public class MyCalculation extends WorkRequest {
        int n;
      
        public MyCalculation( int n ) {
            this.n = n;
        }
        public Object execute() {
            return new Integer( n * n );
        }
    }

As far as data content is concerned, MyCalculation really doesn’t do much; it only really transports an integer value for us. But keep in mind that a request object could hold lots of data, including references to many other objects in complex structures, such as arrays or linked lists. The only requirement is that all the objects to be sent must be serializable or must be able to be discarded by marking them as transient (see Chapter 12). Note that MyCalculation also contains behavior—the execute() operation. While Java object serialization sends only the data content of a class, in our discussion of RMI later in this chapter, we’ll see how Java’s ability to dynamically download bytecode for classes can make both the data content and behavior portable over the network.

It’s also important to note that even without dynamically loading classes over the network (which is uncommon in practice), this design pattern, sometimes called the command pattern, is an important one. Using polymorphism to hide behavior details of tasks from the server allows the application to be easily extended. Polymorphism and Java object serialization are a powerful combination.

Now that we have our protocol, we need the server. The following Server class looks a lot like the TinyHttpd server we developed earlier in this chapter:

    //file: Server.java
    import java.net.*;
    import java.io.*;

    public class Server {
      public static void main( String argv[] ) throws IOException {
        ServerSocket ss = new ServerSocket( Integer.parseInt(argv[0]) );
        while ( true )
          new ServerConnection( ss.accept() ).start();
      }
    } // end of class Server
      
    class ServerConnection extends Thread {
      Socket client;
      ServerConnection ( Socket client ) throws SocketException {
        this.client = client;
      }
      
      public void run() {
        try {
          ObjectInputStream in =
            new ObjectInputStream( client.getInputStream() );
          ObjectOutputStream out =
            new ObjectOutputStream( client.getOutputStream() );
          while ( true ) {
            out.writeObject( processRequest( in.readObject() ) );
            out.flush();
          }
        } catch ( EOFException e3 ) { // Normal EOF
          try {
            client.close();
          } catch ( IOException e ) { }
        } catch ( IOException e ) {
          System.out.println( "I/O error " + e ); // I/O error
        } catch ( ClassNotFoundException e2 ) {
          System.out.println( e2 ); // unknown type of request object
        }
      }
      
      private Object processRequest( Object request ) {
        if ( request instanceof DateRequest )
          return new java.util.Date();
        else if ( request instanceof WorkRequest )
          return ((WorkRequest)request).execute();
        else
          return null;
      }
    }

The Server handles each request in a separate thread. For each connection, the run() method creates an ObjectInputStream and an ObjectOutputStream, which the server uses to receive the request and send the response. The processRequest() method decides what the request means and comes up with the response. To figure out what kind of request we have, we use the instanceof operator to look at the object’s type.

Finally, we get to our Client, which is even simpler:

    //file: Client.java
    import java.net.*;
    import java.io.*;
      
    public class Client {
      public static void main( String argv[] ) {
        try {
          Socket server =
            new Socket( argv[0], Integer.parseInt(argv[1]) );
          ObjectOutputStream out =
            new ObjectOutputStream( server.getOutputStream() );
          ObjectInputStream in =
            new ObjectInputStream( server.getInputStream() );
      
          out.writeObject( new DateRequest() );
          out.flush();
          System.out.println( in.readObject() );
      
          out.writeObject( new MyCalculation( 2 ) );
          out.flush();
          System.out.println( in.readObject() );
      
          server.close();
        } catch ( IOException e ) {
          System.out.println( "I/O error " + e ); // I/O error
        } catch ( ClassNotFoundException e2 ) {
          System.out.println( e2 ); // unknown type of response object
        }
      }
    }

Just like the server, Client creates the pair of object streams. It sends a DateRequest and prints the response; it then sends a MyCalculation object and prints the response. Finally, it closes the connection. On both the client and the server, we call the flush() method after each call to writeObject(). This method forces the system to send any buffered data, ensuring that the other side sees the entire request before we wait for a response. When the client closes the connection, our server catches the EOFException that is thrown and ends the session. Alternatively, our client could write a special object, perhaps null, to end the session; the server could watch for this item in its main loop.

The order in which we construct the object streams is important. We create the output streams first because the constructor of an ObjectInputStream tries to read a header from the stream to make sure that the InputStream really is an object stream. If we tried to create both of our input streams first, we would deadlock waiting for the other side to write the headers.

Finally, we run the example, giving it a port number as an argument:

    % java Server 1234

Then we run the Client, telling it the server’s hostname and port number:

    % java Client flatland 1234

The result should look something like this:

    Sun Mar 5 14:25:25 PDT 2006
    4

All right, the result isn’t that impressive, but it’s easy to think of more substantial applications. Imagine that you need to perform a complex computation on many large datasets. Using serialized objects makes maintenance of the data objects natural and sending them over the wire trivial. There is no need to deal with byte-level protocols at all.

Limitations

As we mentioned earlier, there is one catch in this scenario: both the client and server need access to the necessary classes. That is, all the Request classes—including MyCalculation, which is really the property of the Client—must be deployed in the classpath on both the client and the server machines. In the next section, we’ll see that it’s possible to send the Java bytecode along with serialized objects to allow completely new kinds of objects to be transported dynamically over the network. We could create this solution on our own, adding to the earlier example using a network class loader to load the classes for us. But we don’t have to: Java’s RMI facility handles that for us. The ability to send both serialized data and class definitions over the network is not always needed, but it makes Java a powerful tool for prototyping and developing advanced distributed applications.

Remote Method Invocation

The most fundamental means of communication in Java is method invocation. Mechanisms such as the Java event model are built on simple method invocations between objects in the same virtual machine. Therefore, when we want to communicate between virtual machines on different hosts, it’s natural to want a mechanism with similar capabilities and semantics—to run a method “over there.” Java’s RMI mechanism does just that. It lets us get a reference to an object on a remote host and use it almost as if it were in our own virtual machine. RMI lets us invoke methods on remote objects, passing real Java objects as arguments and getting real Java objects as returned values.

Remote invocation is nothing new. For many years, C programmers have used remote procedure calls (RPC) to execute a C function on a remote host and return the results. The primary difference between RPC in other languages and RMI is that RPC is usually primarily concerned with data structures. It’s relatively easy to pack up data and ship it around, but RMI tries to do one better. In Java, we don’t just work with data structures; we work with objects that contain both data and methods for operating on the data. Not only do we have to be able to ship the state of an object (the data) over the wire, but the recipient has to be able to interact with the object (use its methods) after receiving it. With Java RMI, you can work with network services in an object-oriented fashion, using real, extensible types and pass “live” references between client and server.

It should be no surprise that RMI uses object serialization, which allows us to send graphs of objects (objects and the tree of all the connected objects that they reference). When necessary, RMI can also use dynamic class loading and the security manager to transport Java classes safely. In addition to making remote method calls almost as easy to use as local calls, RMI makes it possible to ship both data and behavior (code) around the Net.

Real-World Usage

Now that the introduction has you all excited, we should put things in a little more context. While Java RMI has proven to be very powerful, it has never really caught on as a way to build general applications. Instead, RPC-like web services using XML and HTTP to transfer data using standardized network protocols have ruled for many years. The reason for this is primarily that they are cross-platform and can be easily consumed by JavaScript running within web browsers. Web services that run over HTTP are also generally immune to firewall issues since they use the same mechanism as all web pages. Since the tools to develop applications using web services have become mature and easy to use, developers tend to use them even when building applications purely in Java, where RMI might otherwise be more powerful. In this section we’ll go ahead and show you what can be done with RMI; however, you will definitely want to check out the chapters on web services and web applications later in this book as well.

Remote and Nonremote Objects

Before an object can be used remotely through RMI, it must be serializable. But that’s not sufficient. Remote objects in RMI are real distributed objects. As the name suggests, a remote object can be an object on a different machine or an object on the local host. The term remote means that the object is used through a special kind of object interface that can be passed over the network. Like normal Java objects, remote objects are passed by reference. Regardless of where the reference is used, the method invocation occurs on the original object, which still lives on its original host. If a remote host returns a reference to one of its remote objects to you, you can call the object’s methods; the actual method invocations happen on the remote host where the underlying object resides.

Nonremote objects are simpler; they’re just normal serializable objects. (You can pass these over the network as we did in the previous section.) The catch is that when you pass a nonremote object over the network, it is simply copied, so references to the object on one host are not the same as those on the remote host. Nonremote objects are passed by value (copying) as opposed to by reference. This may be acceptable for many kinds of data holder objects on your host, such as the client requests and server responses in our previous example. These types of objects are sometimes called value objects or data transfer objects (DTOs).

Remote interfaces

Remote objects implement a special remote interface that specifies which of the object’s methods can be invoked remotely. The remote interface is part of the application that you create by extending the java.rmi.Remote interface. Your remote object then implements its remote interface as it would any other Java interface. In your client-side code, you should then refer to the remote object as an instance of the remote interface—not as an instance of its implementation class. Because both the real object and stub that the client receives implement the remote interface, they are equivalent as far as we are concerned (for method invocation); locally, we never have to worry about whether we have a reference to a stub or to an actual object. This type equivalence means that we can use normal language features such as casting with remote objects. Of course, public fields (variables) of the remote object are not accessible through an interface, so you must make accessor methods if you want to manipulate the remote object’s fields.

One additional requirement for remote objects distinguishes them from local objects. All methods in the remote interface must declare that they can throw the exception java.rmi.RemoteException. This exception (or one of its subclasses) is thrown when any kind of networking error happens (for example, a server crash, network failure, or timeout). Some people see this as a limitation and try to paper over it in various ways. However, the RemoteException is there for a reason—remote objects can behave differently from local objects and your code needs to deal with that issue explicitly. There is no magic bullet (automatic retries, transactions) that truly makes the difference go away.

Here’s a simple example of the remote interface that defines the behavior of RemoteObject; we give it two methods that can be invoked remotely, both of which return some kind of Value object:

    import java.rmi.*;

    public interface RemoteObject extends Remote {
        public Value doSomething() throws RemoteException;
        public Value doSomethingElse() throws RemoteException;
    }

Exporting remote objects

You make a remote object available to the outside world by using the java.rmi.server.UnicastRemoteObject class. One way is simply to have the implementation of your remote object extend UnicastRemoteObject. When a subclass of UnicastRemoteObject is constructed, the RMI runtime system automatically “exports” it to start listening for network connections from clients. Like java.lang.Object, this superclass also provides implementations of equals(), hashcode(), and toString() that make sense for a remote object.

Here’s a remote object class that implements the RemoteObject interface we showed earlier and extends UnicastRemoteObject; we haven’t shown implementations for the two methods or the constructor:

    public class MyRemoteObject implements RemoteObject
            extends java.rmi.UnicastRemoteObject
    {
        public MyRemoteObject() throws RemoteException {...}
        public Value doSomething() throws RemoteException {...}
        public Value doSomethingElse() throws RemoteException {...}
        // nonremote methods
        private void doSomethingInternal() { ... }
    }

Note that we have to supply a constructor that can throw a RemoteException (even if it does nothing) because UnicastRemoteObject’s default constructor throws RemoteException and, even if it’s not shown, the Java language always delegates to the superclass constructor. This class can have as many additional methods as it needs (presumably most of them will be private, but that isn’t strictly necessary), but these nonremote methods are not required to throw the remote exception.

Now, what if we can’t or don’t want to make our remote object implementation a subclass of UnicastRemoteObject? Suppose, for example, that it has to be a subclass of BankAccount or some other special base type for our system. Well, we can simply take over the job of exporting the object ourselves, using the static method exportObject() of UnicastRemoteObject. The exportObject() method takes as an argument a Remote interface and accomplishes what the UnicastRemoteObject constructor normally does for us. It returns as a value the remote object’s client stub. However, you will normally not do anything with this directly. In the next section, we’ll discuss how clients actually find your service, through the RMI registry (a lookup service).

Normally, exported objects listen on individual ephemeral (randomly assigned) port numbers by default. (This is implementation-dependent.) You can control the port number allocation explicitly by exporting your objects using another form of UnicastRemoteObject.exportObject(), which takes both a Remote interface and a port number as arguments.

Finally, the name UnicastRemoteObject begs the question, “What other kinds of remote objects are there?” Right now, few. There is another type of object called Activatable that is for RMI objects that require persistence over time. We’ll say a few more words about RMI activation later in this chapter, but it’s not something we will get into in detail.

The RMI registry

The registry is RMI’s phone book. You use the registry to look up a reference to a registered remote object on another host, using an application-specified name. We’ve already described how remote references can be passed back and forth by remote method calls. The registry is needed to bootstrap the process by allowing the client to look up an initial object on the remote host.

The registry is implemented by a class called Naming and an application called rmiregistry. The rmiregistry application must be running on a host before you start a Java program that wants to advertise in the registry. You can then create instances of remote objects and bind them to particular names in the registry. A registry name can be anything you choose; it takes the form of a slash-separated path. When a client object wants to find your object, it constructs a special URL with the rmi: protocol, the hostname, and the object name. On the client, the RMI Naming class then talks to the registry and returns the remote object reference.

So, which objects need to register themselves with the registry? Initially, this can be any object that the client has no other way of finding. After that, a call to a remote method can return another remote object without using the registry. Likewise, a call to a remote method can have another remote object as its argument, without requiring the registry. You could design your system such that only one object registers itself and then serves as a factory for any other remote objects you need. In other words, it wouldn’t be hard to build a simple object request factory that returns references to all the remote objects your application uses. Depending on how you structure your application, this may happen naturally anyway.

The RMI registry is just one implementation of a lookup mechanism for remote objects. It is not very sophisticated, and lookups tend to be slow. It is not intended to be a general-purpose directory service, but simply to bootstrap RMI communications. More generally, the Java Naming and Directory Interface (JNDI) is a Java API allowing access to other widely used name services that can provide this kind of functionality. JNDI is used with RMI as part of the Enterprise JavaBeans APIs.

An RMI Example

In our first example using RMI, we duplicate the simple serialized object protocol from the previous section. We make a remote RMI object called MyServer on which we can invoke methods to get a Date object or execute a WorkRequest object. First, we define our Remote interface:

    //file: ServerRemote.java
    import java.rmi.*;
    import java.util.*;
      
    public interface ServerRemote extends Remote {
        Date getDate() throws RemoteException;
        Object execute( WorkRequest work ) throws RemoteException;
    }

The ServerRemote interface extends the java.rmi.Remote interface, which identifies objects that implement it as remote objects. We supply two methods that take the place of our old protocol: getDate() and execute().

Next, we implement this interface in a class called MyServer that defines the bodies of these methods. (Another common convention for naming the implementation of remote interfaces is to append Impl to the class name. Using that convention, MyServer would instead be named something like ServerImpl.)

    //file: MyServer.java
    import java.rmi.*;
    import java.util.*;

    public class MyServer
        extends java.rmi.server.UnicastRemoteObject
        implements ServerRemote {

        public MyServer() throws RemoteException { }

        // implement the ServerRemote interface
        public Date getDate() throws RemoteException {
            return new Date();
        }

        public Object execute( WorkRequest work )
          throws RemoteException {
            return work.execute();
        }
      
        public static void main(String args[]) {
            try {
                ServerRemote server = new MyServer();
                Naming.rebind("NiftyServer", server);
            } catch (java.io.IOException e) {
                // problem registering server
            }
        }
    }

MyServer extends UnicastRemoteObject so that when we create an instance of MyServer, it is automatically exported and starts listening to the network. We start by providing a constructor that must throw RemoteException, which accommodates errors that might occur in exporting an instance. Next, MyServer implements the methods of the remote interface ServerRemote. These methods are straightforward.

The last method in this class is main(). This method lets the object set itself up as a server. main() creates an instance of the MyServer object and then calls the static method Naming.rebind() to place the object in the registry. The arguments to rebind() include the name of the remote object in the registry (NiftyServer)—which clients use to look up the object—and a reference to the server object itself. We could have called bind() instead, but rebind() handles the case where there’s already a NiftyServer registered by replacing it.

We wouldn’t need the main() method or this Naming business if we weren’t expecting clients to use the registry to find the server—that is, we could omit main() and still use this object as a remote object. We would just be limited to passing the object in method invocations or returning it from method invocations—but that could be part of a factory pattern, as we discussed before.

Now we need our client:

    //file: MyClient.java
    import java.rmi.*;
    import java.util.*;

    public class MyClient {
      
        public static void main(String [] args)
          throws RemoteException {
            new MyClient( args[0] );
        }
      
        public MyClient(String host) {
            try {
                ServerRemote server = (ServerRemote)
                    Naming.lookup("rmi://"+host+"/NiftyServer");
                System.out.println( server.getDate() );
                System.out.println(
                  server.execute( new MyCalculation(2) ) );
            } catch (java.io.IOException e) {
                  // I/O Error or bad URL
            } catch (NotBoundException e) {
                  // NiftyServer isn't registered
            }
        }
    }

When we run MyClient, we pass it the hostname of the server on which the registry is running. The main() method creates an instance of the MyClient object, passing the hostname from the command line as an argument to the constructor.

The constructor for MyClient uses the hostname to construct a URL for the object. The URL looks like this: rmi://hostname/NiftyServer. (Remember, NiftyServer is the name under which we registered our ServerRemote.) We pass the URL to the static Naming.lookup() method. If all goes well, we get back a reference to a ServerRemote (the remote interface). The registry has no idea what kind of object it will return; lookup() therefore returns an Object, which we must cast to ServerRemote, the remote interface type.

Running the example

You can run the client and server on the same machine or on different machines. First, make sure all the classes are in your classpath (or the current directory if there is no classpath) and then start the rmiregistry and MyServer on your server host:

    % rmiregistry &(on Windows:start rmiregistry)
    %java MyServer

Next, run the client, passing the name of the server host (or “localhost” for the local machine):

    % java MyClientmyhost

The client should print the date and the number 4, which the server graciously calculated. Hooray! With just a few lines of code, you have created a powerful client/server application.

Dynamic class loading

Before running the example, we told you to distribute all of the class files to both the client and server machines. However, RMI was designed to ship classes in addition to data around the network; you shouldn’t have to distribute all the classes in advance. Let’s go a step further and have RMI load classes for us as needed. This involves a few extra steps.

First, we need to tell RMI where to find any other classes it needs. We can use the system property java.rmi.server.codebase to specify a URL on a web server (or FTP server) when we run our client or server. This URL specifies the location of a JAR file or a base directory where RMI begins its search for classes. When RMI sends a serialized object (i.e., an object’s data) to a client, it also sends this URL. If the recipient needs the class file in addition to the data, it fetches the file at the specified URL. In addition to stub classes, other classes referenced by remote objects in the application can be loaded dynamically. Therefore, we don’t have to distribute many class files to the client; we can let the client download them as necessary. In Figure 13-3, we see an example of MyClient going to the registry to get a reference to the ServerRemote object. Once there, MyClient dynamically downloads the stub class for MyServer from a web server running on the server object’s host.

We can now split our class files more logically between the server and client machines. For example, we could withhold the MyCalculation class from the server because it really belongs to the client. Instead, we can make the MyCalculation class available via a web server on some machine (probably our client’s) and specify the URL when we run MyClient:

    % java -Djava.rmi.server.codebase='http://myserver/foo/'...

The trailing slash in the codebase URL is important: it says that the location is a base directory that contains the class files. In this case, we would expect that MyCalculation would be accessible at the URL http://myserver/foo/MyCalculation.class.

Next, we have to set up security. Since we are loading class files over the network and executing their methods, we must have a security manager in place to restrict the kinds of things those classes may do, at least when they are not coming from a trusted code source. RMI will not load any classes dynamically unless a security manager is installed. One easy way to meet this condition is to install the RMISecurityManager as the system security manager for your application. It is an example security manager that works with the default system policy and imposes some basic restrictions on what downloaded classes can do. To install the RMISecurityManager, simply add the following line to the beginning of the main() method of both the client and server applications (yes, we’ll be sending code both ways in the next section):

    main() {
        System.setSecurityManager( new RMISecurityManager() );
        ...
RMI applications and dynamic class loading
Figure 13-3. RMI applications and dynamic class loading

The RMISecurityManager works with the system security policy file to enforce restrictions. You have to provide a policy file that allows the client and server to do basic operations like make network connections. Unfortunately, allowing all the operations needed to load classes dynamically requires listing a lot of permission information and we don’t want to get into that here. We suggest that for this example, you simply grant the code all permissions. Here is an example policy file—call it mysecurity.policy:

    grant {
       permission java.security.AllPermission ;
    };

(It’s exceedingly lame, not to mention risky, to install a security manager and then tell it to enforce no real security, but we’re more interested in looking at the networking code at the moment.)

To run our MyServer application, we would use a command such as:

    % java -Djava.rmi.server.codebase='http://myserver/foo/' \
        -Djava.security.policy=mysecurity.policy MyServer

Finally, one last trick is required to enable dynamic class loading. As of the current implementation, the rmiregistry must be run without the classes that are to be loaded in its classpath. If the classes are in the classpath of rmiregistry, it does not annotate the serialized objects with the URLs of their class files, and no classes are dynamically loaded. This limitation is really annoying; all we can say is to heed the warning for now.

If you follow these directions, you should be able to run our client with only the MyClient class and the ServerRemote remote interface in its classpath. All the other classes are loaded dynamically from the specified server as needed.

Passing remote object references

So far, we haven’t done anything that we couldn’t have done with the simple object protocol. We used only one remote object, MyServer, and we got its reference from the RMI registry. Now we extend our example to pass some remote references between the client and server, allowing additional remote calls in both directions. We’ll add two methods to our remote ServerRemoteinterface:

    public interface ServerRemote extends Remote {
        ...
        StringIterator getList() throws RemoteException;
        void asyncExecute( WorkRequest work, WorkListener
        listener )
            throws RemoteException;
    }

getList() retrieves a new kind of object from the server: a StringIterator. The StringIterator we’ve created is a simple list of strings with some methods for accessing the strings in order. We make it a remote object so that implementations of StringIterator stay on the server.

Next, we spice up our work request feature by adding an asyncExecute() method. asyncExecute() lets us hand off a WorkRequest object as before, but it does the calculation on its own time. The return type for asyncExecute() is void because it doesn’t actually return a value; we get the result later. Along with the request, our client passes a reference to a WorkListener object that is to be notified when the WorkRequest is done. We’ll have our client implement WorkListener itself.

Because this is to be a remote object, our interface must extend Remote and its methods must throw RemoteExceptions:

    //file: StringIterator.java
    import java.rmi.*;

    public interface StringIterator extends Remote {
        public boolean hasNext() throws RemoteException;
        public String next() throws RemoteException;
    }

Next, we provide a simple implementation of StringIterator, called MyStringIterator:

    //file: MyStringIterator.java
    import java.rmi.*;

    public class MyStringIterator
      extends java.rmi.server.UnicastRemoteObject
      implements StringIterator {
      
        String [] list;
        int index = 0;

        public MyStringIterator( String [] list )
          throws RemoteException {
            this.list = list;
        }
        public boolean hasNext() throws RemoteException {
            return index < list.length;
        }
        public String next() throws RemoteException {
            return list[index++];
        }
    }

MyStringIterator extends UnicastRemoteObject. Its methods are simple: it can give you the next string in the list, and it can tell you if there are any strings you haven’t seen yet.

Next, we discuss the WorkListener remote interface that defines how an object should listen for a completed WorkRequest. It has one method, workCompleted(), which the server executing a WorkRequest calls when the job is done:

    //file: WorkListener.java
    import java.rmi.*;

    public interface WorkListener extends Remote {
        public void workCompleted(WorkRequest request, Object result )
            throws RemoteException;
    }

Let’s add the new features to MyServer. We need to add implementations of the getList() and asyncExecute() methods, which we just added to the ServerRemote interface:

    public class MyServer extends java.rmi.server.UnicastRemoteObject
                          implements ServerRemote {
      ...
      public StringIterator getList() throws RemoteException {
        return new MyStringIterator(
            new String [] { "Foo", "Bar", "Gee" } );
      }

      public void asyncExecute(
         final WorkRequest request, final WorkListener listener )
         throws java.rmi.RemoteException
      {
        new Thread() {
          public void run() {
            Object result = request.execute();
            try {
              listener.workCompleted( request, result );
            } catch ( RemoteException e ) {
              System.out.println( e ); // error calling client
            }
        }}.start();
      }
    }

getList() just returns a StringIterator with some stuff in it. asyncExecute() calls a WorkRequest’s execute() method and notifies the listener when it’s done. asyncExecute() runs the request in a separate thread, allowing the remote method call to return immediately. Later, when the work is done, the server uses the client’s WorkListener interface to return the result.

We have to modify MyClient to implement the remote WorkListener interface. This turns MyClient into a remote object, so we will have it extend UnicastRemoteObject. We also add the workCompleted() method the WorkListener interface requires. Finally, we want MyClient to exercise the new features. We’ve put all of this in a new version of the client called MyClientAsync:

    //file: MyClientAsync.java
    import java.rmi.*;
    import java.util.*;
     
    public class MyClientAsync
        extends java.rmi.server.UnicastRemoteObject implements WorkListener
    {
     
        public MyClientAsync(String host) throws RemoteException
        {
            try {
                ServerRemote server = (ServerRemote)
                    Naming.lookup("rmi://"+host+"/NiftyServer");
     
                server.asyncExecute( new MyCalculation( 100 ), this );
                System.out.println("call done...");
            } catch (java.io.IOException e) {
                // I/O Error or bad URL
            } catch (NotBoundException e) {
                // NiftyServer isn't registered
            }
        }
     
        public void workCompleted( WorkRequest request, Object result )
            throws RemoteException
        {
            System.out.println("Async result: "+result );
        }
     
        public static void main(String [] args) throws RemoteException {
            new MyClientAsync( args[0] );
        }
     
    }
     

We use getList() to get the iterator from the server and then loop, printing the strings. We also call asyncExecute() to perform another calculation; this time, we square the number 100. The second argument to asyncExecute() is the WorkListener to notify when the data is ready; we pass a reference to ourselves (this).

Restart the RMI registry and MyServer on your server, and run the client somewhere. You should get the following:

    Foo
    Bar
    Gee
    Async result = 10000

We hope that this introduction has given you a feel for the tremendous power that RMI offers through object serialization and dynamic class loading. Java is one of the first programming languages to offer this kind of powerful framework for distributed applications. Although some of the advanced features are not used widely in business applications, RMI was the underpinning for the very widely used J2EE Enterprise JavaBeans architecture and is an important technology. For more information on RMI and J2EE, see Java Enterprise in a Nutshell (O’Reilly).

RMI and CORBA

Java supports an important alternative to RMI, called CORBA (Common Object Request Broker Architecture). We won’t say much about CORBA here, but you should know that it exists. CORBA is an older distributed object standard developed by the Object Management Group (OMG), of which Sun Microsystems was one of the founding members. Its major advantage is that it works across languages: a Java program can use CORBA to talk to objects written in other languages, like C or C++. This may be a considerable advantage if you want to build a Java frontend for an older program that you can’t afford to reimplement. CORBA also provides other services similar to those in the Java Enterprise APIs. CORBA’s major disadvantages are that it’s complex, inelegant, and somewhat arcane.

Scalable I/O with NIO

We’ll now conclude the discussion of the NIO package we began in Chapter 12 by talking about nonblocking and selectable network communications. All our server examples in this chapter thus far have used a thread-bound pattern (one thread per I/O operation). In Java, this is very natural because of the ease with which we can create threads. It’s also very efficient, within limits. Problems arise when you try to build very large-scale servers using this style of client handling. While on a large machine it’s certainly possible to have hundreds or even thousands of threads (especially if they’re mostly idle, waiting for I/O), this is a resource-hungry solution. Every thread you start in Java consumes memory for its internal stack, and the performance of managing this number of threads is highly system-dependent.

An alternate approach is to take a lesson from the old, dark days before threading was available and use nonblocking I/O operations to manage numerous communications from a single thread. Better yet, our server uses a configurable pool of threads, taking advantage of machines with many processors.

At the heart of this process is the concept of selectable I/O. It’s not good enough to simply have nonblocking I/O operations if you have no way to efficiently poll for work to be done. The NIO package provides for efficient polling using selectable channels. A selectable channel allows for the registration of a special kind of listener called a selector that can check the readiness of the channel for operations, such as reading and writing or accepting or creating network connections.

The selector and the selection process are not typical Java listeners of the kind we’ll see elsewhere in this book, but instead rather slavishly follow the conventions of C language systems. This is mainly for performance reasons; because this API is primarily intended for high-volume servers, it is bound very tightly to the traditional, underlying operating system facilities with less regard for ease of use. This, combined with the other details of using the NIO package, mean that this section is somewhat dense and the server we create here is one of the longer and more complex examples in the book. Don’t be discouraged if you are a bit put off by this section. You can use the general techniques earlier in this chapter for most applications and reserve this knowledge for creating services that handle the very highest volumes of simultaneous client requests.

Selectable Channels

A selectable channel implements the SelectableChannel interface, which specifies that the channel can be set to a nonblocking mode and that it supports the select API that makes efficient polling possible. The primary implementations of selectable channels are those for working with the network: SocketChannel, ServerSocketChannel, and DatagramChannel. The only other selectable channel is the Pipe (which can be used in an analogous way for intra-VM communication).

At the heart of the process is the Selector object, which knows about a particular set of selectable channels and provides a select() method for determining their readiness for I/O operations. Conceptually, the process is simple; you register one or more channels with a selector and then poll it, asking it to tell you which set of channels is ready to go. In actuality, there are a few additional pieces involved.

First, the Selector does not work directly with channels but instead operates on SelectionKey objects. A SelectionKey object is created implicitly when the channel is registered with the Selector. It encapsulates the selectable channel as well as information about what types of operations (e.g., read, write) we are interested in waiting for. That information is held in the SelectionKey in a set of flags called the interest set, which can be changed by the application at any time. SelectionKeys are also used to return the results of a select operation. Each call to select() returns the number of SelectionKeys that are ready for some type of I/O. The keys are then retrieved with the selectedKeys() method. Each key also has a set of flags called the ready set that indicates which operation of interest is actually ready (possibly more than one). For example, a SelectionKey interest set might indicate that we want to know when its channel is ready for reading or writing. After a select operation, if that key is in the set returned by the selector, we know that it is ready for one or more of those operations, and we can check the key’s ready set to find out which one.

Before we go on, we should say that although we have been saying that channels are registered with selectors, the API is (confusingly) the other way around. Selectors are actually registered with the one or more channels they manage, but it’s better to mentally spackle over this and think of them the other way around.

Using Select

A Selector object is created using the Selector.open() method (Selector uses a factory pattern):

    Selector selector = Selector.open();

To register one or more channels with the selector, set them to nonblocking mode:

    SelectableChannel channelA = // ...
    channelA.configureBlocking( false );

Next, register the channels:

    int interestOps = SelectionKey.OP_READ | SelectionKey.OP_WRITE;
    SelectionKey key = channelA.register( selector, interestOps );

When we register the channel, we have an opportunity to set the initial interest operations (or “interest ops”). These are defined by constant fields in the SelectionKey class:

OP_READ

Ready to read

OP_WRITE

Ready to write

OP_CONNECT

Client-socket connection ready

OP_ACCEPT

Server-socket connection ready

These fields are bit flags; you can logically OR them together as in this example to express interest in more than one type of operation.

The result of the register() method is a SelectionKey object. We can use the key to change the interest ops at any time with the SelectionKey interestOps() method or to unregister the channel from the Selector with the key’s cancel() method.

This same key is also returned as the result of selection operations when its channel is ready. When the SelectionKey is returned, its ready set holds flags for the operations that do not block if called. We can retrieve the value of the flags with the readySet() method. Convenience methods are available to test for each operation in the ready set: isReadable(), isWritable(), isConnectable(), and isAcceptable().

Depending on how you structure your application, it may not be necessary to save the SelectionKey at registration time. In our example, we let the Selector keep track of the keys for us, simply using them when they are ready. In fact, we go even further and put the SelectionKey to work by asking it to hold a reference for us! The SelectionKey attach() method is a convenience method that can attach an arbitrary object to the key for use by our application. We show how this can be useful in a bit.

After one or more channels are registered with the Selector, we can perform a select operation using one of its select() methods:

    int readyCount = selector.select();

Without arguments, the method blocks until at least one channel is ready for some operation or until the Selector’s wakeup() method is called. Alternatively, you can use the form of select() that takes a timeout (in milliseconds) to wait for a ready channel before returning. There is also selectNow(), which always returns immediately. Both of these return methods count the number of ready channels.

You can use select() and wakeup() somewhat like wait() and notify(). The wakeup is necessary because once a selection is started, it will not see any changes to its key’s interest ops until the next invocation. If another thread changes the interest ops, it must use wakeup() to prompt the selecting thread to select() again. The Selector is also heavily synchronized; for example, calls to register new channels block until the select is finished. Often it’s much easier to simply use select with a short timeout and a loop, like this:

    while ( selector.select( 50 ) == 0 );

However, if another thread is allowed to change the interest ops, you still need to use wakeup() to maximize throughput. Otherwise, in the worst case, you could end up waiting the full select wait period on every iteration, even when there is work to be done.

Next, we can get the set of ready channels from the Selector with the selectedKeys() method and iterate through them, doing whatever our application dictates:

    Set readySet = selector.selectedKeys();
    for( Iterator it = readySet.iterator(); it.hasNext(); ) {
        SelectionKey key = (SelectionKey)it.next();
        it.remove();  // remove the key from the ready set
     // use the key
    }

The ready set is returned to us as a java.util.Set, which we walk through with an Iterator (see Chapter 1). One important thing to note is that we’ve used the Iterator’s remove() method to remove the key from the ready set. The select() methods add keys only to the ready set or add flags to keys already in the set; they never remove them, so we must clear the keys when we handle them. You can get the full set of keys a Selector is managing with the keys() method, but you should not attempt to remove keys from that set; use the cancel() method on individual keys instead. Or you can close the entire Selector with its close() method, unregistering all its keys.

LargerHttpd

Let’s put this information to use. In this section, we’ll create the big brother of TinyHttpd (our minimalist web server) called LargerHttpd. The LargerHttpd server is a nonblocking web server that uses SocketChannels and a pool of threads to service requests. In this example, a single thread executes a main loop that accepts new connections and checks the readiness of existing client connections for reading or writing. Whenever a client needs attention, it places the job in a queue where a thread from our thread pool waits to service it. As we said, this example is a bit longer than we would like, but it is really the minimum that is necessary to show a realistic usage of the APIs:

import java.io.*;
import java.util.*;
import java.util.concurrent.*;
import java.net.*;
import java.nio.*;
import java.nio.channels.*;
import java.nio.charset.*;
import java.util.regex.*;

public class LargerHttpd
{
    Selector clientSelector;

    public void run( int port, int threads ) throws IOException 
    {
        clientSelector = Selector.open();
        ServerSocketChannel ssc = ServerSocketChannel.open();
        ssc.configureBlocking(false);
        InetSocketAddress sa =  new InetSocketAddress( InetAddress
            .getLoopbackAddress(), port );
        ssc.socket().bind( sa );
        ssc.register( clientSelector, SelectionKey.OP_ACCEPT );
    
        Executor executor = Executors.newFixedThreadPool( threads );

        while ( true ) {
            try {
                while ( clientSelector.select(100) == 0 );
                Set<SelectionKey> readySet = clientSelector.selectedKeys();
                for(Iterator<SelectionKey> it=readySet.iterator();
                    it.hasNext();)
                {
                    final SelectionKey key = it.next();
                    it.remove();
                    if ( key.isAcceptable() ) {
                        acceptClient( ssc );
                    } else {
                        key.interestOps( 0 );
                        executor.execute( new Runnable() {
                          public void run() {
                            try {
                                handleClient( key );
                              } catch ( IOException e) {
                                System.out.println(e);
                              }
                          }
                        } );
                    }
                }
            } catch ( IOException e ) { System.out.println(e); }
        }
    }

    void acceptClient( ServerSocketChannel ssc ) throws IOException
    {
        SocketChannel clientSocket = ssc.accept();
        clientSocket.configureBlocking(false);
        SelectionKey key =  clientSocket.register( clientSelector,
            SelectionKey.OP_READ );
        HttpdConnection client = new HttpdConnection( clientSocket );
        key.attach( client );
    }

    void handleClient( SelectionKey key ) throws IOException
    {
        HttpdConnection client = (HttpdConnection)key.attachment();
        if ( key.isReadable() ) {
            client.read( key );
        } else {
            client.write( key );
        }
        clientSelector.wakeup();
    }

    public static void main( String argv[] ) throws IOException {
        //new LargerHttpd().run( Integer.parseInt(argv[0]), 3/*threads*/ );
        new LargerHttpd().run( 1235, 3/*threads*/ );
    }
}

class HttpdConnection 
{
    static Charset charset = Charset.forName("8859_1");
    static Pattern httpGetPattern = Pattern.compile("(?s)GET /?(\\S*).*");
    SocketChannel clientSocket;
    ByteBuffer buff = ByteBuffer.allocateDirect( 64*1024 );
    String request;
    String response;
    FileChannel file;
    int filePosition;

    HttpdConnection ( SocketChannel clientSocket ) {
        this.clientSocket = clientSocket;
    }

    void read( SelectionKey key ) throws IOException {
        if ( request == null && (clientSocket.read( buff ) == -1 
                || buff.get( buff.position()-1 ) == '\n' ) )
            processRequest( key );
        else
            key.interestOps( SelectionKey.OP_READ );
    }

    void processRequest( SelectionKey key ) {
        buff.flip();
        request = charset.decode( buff ).toString();
        Matcher get = httpGetPattern.matcher( request );
        if ( get.matches() ) {
            request = get.group(1);
            if ( request.endsWith("/") || request.equals("") )
                request = request + "index.html";
            System.out.println( "Request: "+request);
            try {
                file = new FileInputStream ( request ).getChannel();
            } catch ( FileNotFoundException e ) {
                response = "404 Object Not Found";
            }
        } else
            response = "400 Bad Request" ;

        if ( response != null ) {
            buff.clear();
            charset.newEncoder().encode( 
                CharBuffer.wrap( response ), buff, true );
            buff.flip();
        }
        key.interestOps( SelectionKey.OP_WRITE );
    }

    void write( SelectionKey key ) throws IOException {
        if ( response != null ) {
            clientSocket.write( buff );
            if ( buff.remaining() == 0 ) 
                response = null;
        } else if ( file != null ) {
            int remaining = (int)file.size()-filePosition;
            long sent = file.transferTo( filePosition, remaining,
                clientSocket);
            if ( sent >= remaining || remaining <= 0 ) {
                file.close();
                file = null;
            } else
                filePosition += sent;
        } 
        if ( response == null && file == null ) {
            clientSocket.close();
            key.cancel();        
        } else 
            key.interestOps( SelectionKey.OP_WRITE );
    }
}

From a bird’s-eye view, the structure of LargerHttpd is the same as TinyHttpd. The main class, LargerHttpd, accepts connections, and a connection class, HttpdConnection, encapsulates a socket and handles the conversation with the client. However, this time, instead of each connection object being a Runnable serviced in its own thread, its functionality is broken into two primary methods called read() and write(). The job of our LargerHttpd is to accept new client socket connections, wrap them in an instance of HttpdConnection, and then watch the client’s status with a Selector. Whenever we detect that a client is ready to send or receive data, we hand off a Runnable task to our Executor. The task calls read() or write() on the corresponding client, based on the operation that is is ready.

The HttpConnection object encapsulates the state of the conversation with the client. Because its interface is rather coarse, it must keep track of whether it is waiting to read more input, generate a response, or write file output. The HttpdConnection also manages the interest set of its key so that it can effectively schedule itself to be woken up when it’s ready for reading or writing. The association between the HttpdConnection and the key is made by using the key’s attach() and attachment() methods.

LargerHttpd’s acceptClient() method does several things. First, it accepts the new socket connection. Next, it configures and registers it with the selector with an initial interest set for reading. Finally, it creates the HttpdConnection for the socket, and attaches the HttpdConnection object to the key for later retrieval.

The main loop of LargerHttpd is fairly straightforward. First, we set up the ServerSocketChannel. This is similar to setting up a plain ServerSocket, except that we must first create an InetSocketAddress object to hold the local loopback address and port combination of our server socket and then explicitly bind our socket to that address with the ServerSocketChannel bind() method. We also configure the server socket to nonblocking mode and register it with our main Selector so that we can select for client connections in the same loop that we use to select for client read and write readiness.

In the main select loop, we check to see whether the key is ready for an accept operation and if so, we call acceptClient(); if not, we set the key’s interest set to zero with the interestOps() method and dispatch the key to our handleClient() method via a Runnable task. It’s important that we change the interest set to zero to clear it before the next loop; otherwise, we’d be in a race to see whether the thread pool performed its maximum work before we detected another ready condition. Setting the interest ops to 0 and resetting it in the HttpdConnection object upon completion ensures that only one thread is handling a given client at a time.

For each operation that is ready, we dispatch a task to our Executor. The task calls handleClient(), passing it the selection key. From the key, we retrieve the associated HttpdConnection object and call the appropriate service method based on whether the key is ready for reading or writing. After that, it’s up to the connection object to do its job. Each call to the read() method simply does what would be one iteration of a read loop in a thread-bound application. Each read gets as much data as available and checks to see whether we’ve reached the end of a line (a \n newline character). Upon reaching the end of a line, we dispatch the call to the processRequest() method, which turns the byte buffer into text and uses the same techniques as our TinyHttpd to parse the request into a file pathname. On each incomplete call to read(), we set the interest ops of our key back to OP_READ. Upon completing the read and processing the request, we switch to using OP_WRITE because we are now ready to send a response.

The write() method keeps track of whether it’s sending a text response (error message) or a file by using the response and file instance variables. When sending a file, we use the FileChannel’s transferTo() method to transfer bytes from the file directly to the network socket without copying them into Java’s memory space. (This is indeed an efficient little web server.) And that’s about it. When we’re done, we close the client socket and cancel our key, which causes it to be removed from the Selector’s key set during the next select operation (discarding our HttpdConnection object with it).

Nonblocking Client-Side Operations

Our example showed SocketChannel used for nonblocking, selectable I/O in a typical server application. It’s less common to need nonblocking I/O from a client, but there is certainly no reason you can’t do it. Perhaps you’re writing a peer-to-peer (P2P) application that manages many connections from both sides.

For the client side of communications, one additional tool is provided: a nonblocking socket-connect operation. The process of creating a TCP connection from the client side involves contacting the remote host in a two-phase acknowledgment. This process normally blocks until the connection is established. However, the NIO package provides an alternative that allows you to initiate the connection and then poll for its status. When set to nonblocking mode, a call to a SocketChannel’s connect() method returns immediately. The connection is then attempted (and possibly succeeds or fails) in the background. Later, a Selector can be used, checking for the OP_CONNECT flag to see when the socket is ready to “finish connecting.” The connection is finished by invoking the SocketChannel’s finishConnect() method, which either returns or throws an IOException indicating the failure. The process of finishing the connection is really more about collecting the results of the asynchronous connection—acknowledging its success or failure—than about doing work.



[37] For a discussion of sockets in general, see Unix Network Programming by Richard Stevens (Prentice-Hall).