Managing a Dynamic Java Trust Store

This is the latest (and probably last) in my series of client-side Java key and trust store management articles, and a good summary article for the topic, I hope.

It’s clear from the design of SSLContext in the JSSE that Java key and trust stores are meant to contain static data. Yet browsers regularly display the standard security warning dialog when connecting to sites whose certificates have expired or whose administrators haven’t bothered to purchase a CA-signed certificate. This dialog generally offers you three choices:

  • Get me out of here!
  • I understand the risks: add certificate for this session only
  • I understand the risks: add certificate permanently

In this article, I’d like to elaborate on what it means to “add certificate” – either temporarily or permanently.

Let’s start with a simple Java http(s) client:

public byte[] getContentBytes(URI uri, SSLContext ctx)
    throws Exception {
  URL url = uri.toURL();
  URLConnection conn = url.openConnection();
  if (conn instanceof HttpsURLConnection && ctx != null) {
    ((HttpsURLConnection)conn).setSSLSocketFactory(
        ctx.getSocketFactory());
  }
  InputStream is = conn.getInputStream();
  int bytesRead, bufsz = Math.max(is.available(), 4096);
  ByteArrayOutputStream os = new ByteArrayOutputStream(bufsz);
  byte[] buffer = new byte[bufsz];
  while ((bytesRead = is.read(buffer)) > 0)
    os.write(buffer, 0, bytesRead);
  byte[] content = os.toByteArray();
  os.close(); is.close();
  return content;
}

This client opens a URLConnection, reads the input stream into a byte buffer, and then closes the connection. If the connection is https – that is, an instance of HttpsURLConnection – it applies the SocketFactory from the supplied SSLContext.

NOTE: I’m purposely ignoring exception managment in this article to keep it short.

This code is simple and concise, but clearly there’s no way to affect what happens during application of the SSL certificates and keys at this level of the code. Certificate and key management is handled by the SSLContext so if we want to modify the behavior of the SocketFactory relative to key management, we’re going to have to do something with SSLContext before we pass it to the client. The simplest way to get an SSLContext is to call SSLContext.getDefault in this manner:

byte[] bytes = getContentBytes(
    URI.create("https://www.example.com/"), 
    SSLContext.getDefault());

The default SSLContext is fairly limited in functionality. It uses either default key and trust store files (and passwords!) or else ones specified in system properties – often via the java command line in this manner:

$ java -Djavax.net.ssl.keyStore=/path/to/keystore.jks \
 -Djavax.net.ssl.keyStorePassword=changeit \
 -Djavax.net.ssl.trustStorePath=/path/to/truststore.jks \
 -Djavax.net.ssl.trustStorePassword=changeit ...

In reality, there is no default keystore, which is fine for normal situations, as most websites don’t require X.509 client authentication (more commonly referred to as mutual auth). The default trust store is $JAVA_HOME/jre/lib/security/cacerts, and the default trust store password is changeit. The cacerts file contains several dozen certificate authority (CA) root certificates and will validate any server whose public key certificate is signed by one of these CAs.

More importantly, however, the default SSLContext simply fails to connect to a server in the event that a trust certificate is missing from the default trust store. But that’s not what web browsers do. Instead, they display the aforementioned dialog presenting the user with options to handle the situation in the manner that suits him or her best.

Assume the simple client above is a part of a larger application that adds certificates to the trust store during execution of other code paths and then expects to be able to use this updated trust store later during the same session. This dynamic reload functionality requires some SSLContext customization.

Let’s explore. SSLContext is a great example of a composite design. It’s built from several other classes, each of which may be specified by the user when initializing a context object. This practically eliminates the need to sub-class SSLContext in order to define custom behavior. The default context is eschewed in favor of a user-initialized instance of SSLContext like this:

public SSLContext getSSLContext(String tspath) 
    throws Exception {
  TrustManager[] trustManagers = new TrustManager[] { 
    new ReloadableX509TrustManager(tspath) 
  };
  SSLContext sslContext = SSLContext.getInstance("SSL");
  sslContext.init(null, trustManagers, null);
  return sslContext;
}

At the heart of this method is the instantiation of a new ReloadableX509TrustManager. The init method of SSLContext accepts a reference to an array of TrustManager objects. Passing null tells the context to use the default trust manager array which exihibits the default behavior mentioned above.

The init method also accepts two other parameters, to which I’ve passed null. The first parameter is a KeyManager array and the third is an implementation of SecureRandom. Passing null for any of these three parameters tells SSLContext to use the default. Here’s one implementation of ReloadableX509TrustManager:

class ReloadableX509TrustManager 
    implements X509TrustManager {
  private final String trustStorePath;
  private X509TrustManager trustManager;
  private List tempCertList 
      = new List();

  public ReloadableX509TrustManager(String tspath)
      throws Exception {
    this.trustStorePath = tspath;
    reloadTrustManager();
  }

  @Override
  public void checkClientTrusted(X509Certificate[] chain, 
      String authType) throws CertificateException {
    trustManager.checkClientTrusted(chain, authType);
  }

  @Override
  public void checkServerTrusted(X509Certificate[] chain, 
      String authType) throws CertificateException {
    try {
      trustManager.checkServerTrusted(chain, authType);
    } catch (CertificateException cx) {
      addServerCertAndReload(chain[0], true);
      trustManager.checkServerTrusted(chain, authType);
    }
  }

  @Override
  public X509Certificate[] getAcceptedIssuers() {
    X509Certificate[] issuers 
        = trustManager.getAcceptedIssuers();
    return issuers;
  }

  private void reloadTrustManager() throws Exception {

    // load keystore from specified cert store (or default)
    KeyStore ts = KeyStore.getInstance(
	    KeyStore.getDefaultType());
    InputStream in = new FileInputStream(trustStorePath);
    try { ts.load(in, null); }
    finally { in.close(); }

    // add all temporary certs to KeyStore (ts)
    for (Certificate cert : tempCertList) {
      ts.setCertificateEntry(UUID.randomUUID(), cert);
    }

    // initialize a new TMF with the ts we just loaded
    TrustManagerFactory tmf 
	    = TrustManagerFactory.getInstance(
            TrustManagerFactory.getDefaultAlgorithm());
    tmf.init(ts);

    // acquire X509 trust manager from factory
    TrustManager tms[] = tmf.getTrustManagers();
    for (int i = 0; i < tms.length; i++) {
      if (tms[i] instanceof X509TrustManager) {
        trustManager = (X509TrustManager)tms[i];
        return;
      }
    }

    throw new NoSuchAlgorithmException(
        "No X509TrustManager in TrustManagerFactory");
  }

  private void addServerCertAndReload(Certificate cert, 
      boolean permanent) {
    try {
      if (permanent) {
        // import the cert into file trust store
        // Google "java keytool source" or just ...
        Runtime.getRuntime().exec("keytool -importcert ...");
      } else {
        tempCertList.add(cert);
      }
      reloadTrustManager();
    } catch (Exception ex) { /* ... */ }
  }
}

NOTE: Trust stores often have passwords but for validation of credentials the password is not needed because public key certificates are publicly accessible in any key or trust store. If you supply a password, the KeyStore.load method will use it when loading the store but only to validate the integrity of non-public information during the load – never during actual use of public key certificates in the store. Thus, you may always pass null in the second argument to KeyStore.load. If you do so, only public information will be loaded from the store.

A full implementation of X509TrustManager is difficult and only sparsely documented but, thankfully, not necessary. What makes this implementation simple is that it delegates to the default trust manager. There are two key bits of functionality in this implementation: The first is that it loads a named trust store other than cacerts. If you want to use the default trust store, simply assign $JAVA_HOME/jre/lib/security/cacerts to trustStorePath.

The second bit of functionality is the call to addServerCertAndReload during the exception handler in the checkServerTrusted method. When a certificate presented by a server is not found in the trust manager’s in-memory database, ReloadableX509TrustManager assumes that the trust store has been updated on disk, reloads it, and then redelegates to the internal trust manager.

A more functional implementation might display a dialog box to the user before calling addServerCertAndReload. If the user selects Get me out of here!, the method would simply rethrow the exception instead of calling that routine. If the user selects It’s cool: add permanently, the method would add the certificate to the file-based trust store, reload from disk, and then reissue the delegated request. If the user selects I’ll bite: add temporarily, the certificate would be added to a list of temporary certificates in memory.

The way I’ve implemented the latter case is to add the certificate to a temporary list and then reload from disk. Strictly speaking, reloading from disk isn’t necessary in this case since no changes were made to the disk file but the KeyStore built from the disk image would have to be kept around for reloading into the trust manager (after the new cert was added to it), so some modifications would have to be made to avoid reloading from disk.

This same code might as well be used in a server-side setting but the checkClientTrusted method would have to be modified instead of the checkServerTrusted method as in this example.

Advertisements

RESTful Authentication

My last post on RESTful transactions sure seemed to attract a lot of attention. There are a number of REST discussion topics that tend to get a lot of hand-waving by the REST community, but no real concrete answers seem to be forthcoming. I believe the most fundamental reasons for this include the fact that the existing answers are unpalatable – both to the web services world at large, and to REST purists. Once in a while when they do mention a possible solution to a tricky REST-based issue, the web services world responds violently – mostly because REST purists give answers like “just don’t do that” to questions like “How do I handle session management in a RESTful manner?”

I recently read an excellent treatise on the subject of melding RESTful web services concepts with enterprise web service needs. Benjamin Carlyle’s Sound Advice blog entry, entitled The REST Statelessness Constraint hits the mark dead center. Rather than try to persuade enterprise web service designers not to do non-RESTful things, Benjamin instead tries to convey the purposes behind REST constraints (in this case, specifically statelessness), allowing web service designers to make rational tradeoffs in REST purity for the sake of enterprise goals, functionality, and performance. Nice job Ben!

The fact is that the REST architectural style was designed with one primary goal in mind: to create web architectures that would scale well to the Internet. The Internet is large, representing literally billions of clients. To make a web service scale to a billion-client network, you have to make hard choices. For instance, http is connectionless. Connectionless protocols scale very well to large numbers of clients. Can you imagine a web server that had to manage 500,000 simultaneous long-term connections?

Server-side session data is a difficult concept to shoehorn into a RESTful architecture, and it’s the subject of this post. Lots of web services – I’d venture to say 99 percent of them – manage authentication using SSL/TLS and the HTTP “basic auth” authentication scheme. They use SSL/TLS to keep from exposing a user’s name and password over the wire, essentially in clear text. They use basic auth because it’s trivial. Even banking institutions use this mechanism because, for the most part, it’s secure. Those who try to go beyond SSL/TLS/basic auth often do so because they have special needs, such as identity federation of disparate services.

To use SSL/TLS effectively, however, these services try hard to use long-term TCP connections. HTTP 1.0 had no built-in mechanism for allowing long-term connections, but NetScape hacked in an add-on mechanism in the form of the “connection: keep-alive” header, and most web browsers support it, even today. HTTP 1.1 specifies that connections remain open by default. If an HTTP 1.1 client sends the “connection: close” header in a request then the server will close the connection after sending the response, but otherwise, the connection remains open.

This is a nice enhancement, because it allows underlying transport-level security mechanisms like SSL/TLS to optimize transport-level session management. Each new SSL/TLS connection has to be authenticated, and this process costs a few round-trips between client and server. By allowing multiple requests to occur over the same authenticated sesssion, the cost of transport-level session management is amortized over several requests.

In fact, by using SSL/TLS mutual authentication as the primary authentication mechanism, no application state need be maintained by the server at all for authentication purposes. For any given request, the server need only ask the connection layer who the client is. If the service requires SSL/TLS mutual auth, and the client has made a request, then the server knows that the client is authenticated. Authorization (resource access control) must still be handled by the service, but authorization data is not session data, it’s service data.

However, SSL/TLS mutual auth has an inherent deployment problem: key management. No matter how you slice it, authentication requires that the server know something about the client in order to authenticate that client. For SSL/TLS mutual auth, that something is a public key certificate. Somehow, each client must create a public key certificate and install it on the server. Thus, mutual auth is often reserved for the enterprise, where key management is done by IT departments for the entire company. Even then, IT departments cringe at the thought of key management issues.

User name and password schemes are simpler, because often web services will provide users a way of creating their account and setting their user name and password in the process. Credential management done. Key management can be handled in the same way, but it’s not as simple. Some web services allow users to upload their public key certificate, which is the SSL/TLS mutual-auth equivalent of setting a password. But a user has to create a public/private key pair, and then generate a public key certificate from this key pair. Java keytool makes this process as painless as possible, but it’s still far from simple. No – user name and password is by far the simpler solution.

As I mentioned above, the predominant solution today is a combination of CA-based transport-layer certificate validation for server authentication, and HTTP basic auth for client authentication. The web service obtains a public/private key pair that’s been generated by a well-known Certificate Authority (CA). This is done by generating a certificate signing request using either openssl or the Java keytool utility (or by using less mainstream tools provided by the CA). Because most popular web browsers today ship well-known CA certificates in their truststores, and because clients implicitly trust services that provide certificates signed by these well-known CA’s, people tend to feel warm and fuzzy because no warning messages pop up on the screen when they connect to one of these services. Should they fear? Given the service verification process used by CAs like Entrust and Verisign, they probably should, but that problem is very difficult to solve, so most people just live with this stop-gap solution.

On the server side, the web service needs to know the identity of the client in order to know what service resources that client should have access to. If a client requests a protected resource, the server must be able to validate that client’s right to the resource. If the client hasn’t authenticated yet, the server challenges the client for credentials using a response header and a “401 Unauthorized” response code. Using the basic auth scheme, the client base64-encodes his user name and password and returns this string in a response header. Now, base64 encoding is not encrytion, so the client is essentially passing his user name and password in what amounts to clear text. This is why SSL/TLS is used. By the time the server issues the challenge, the SSL/TLS encrypted channel is already established, so the user’s credentials are protected from even non-casual snoopers.

When the proper credentials arrive in the next attempt to request the protected resource, the server decodes the user name and password, verifies them against its user database, and either returns the requested resource, or fails the request with “401 Unauthorized” again, if the user doesn’t have the requisite rights to the requested resource.

If this was the extent of the matter, there would be nothing unRESTful about this protocol. Each subsequent request contains the user’s name and password in the Authorization header, so the server has the option of using this information on each request to ensure that only authorized users can access protected resources. No session state is managed by the server here. Session or application state is managed by the client, using a well-known protocol for passing client credentials on each request – basic auth.

But things don’t usually stop there. Web services want to provide a good session experience for the user – perhaps a shopping cart containing selected items. Servers typically implement shopping carts by keeping a session database, and associating collections of selected items with users in this database. How long should such session data be kept around? What if the user tires of shopping before she checks out, goes for coffee, and gets hit by a car? Most web services deal with such scenarios by timing out shopping carts after a fixed period – anywhere from an hour to a month. What if the session includes resource locks? For example, items in a shopping cart are sometimes made unavailable to others for selection – they’re locked. Companies like to offer good service to customers, but keeping items locked in your shopping cart for a month while you’re recovering in the hospital just isn’t good business.

REST principles dictate that keeping any sort of session data is not viable for Internet-scalable web services. One approach is to encode all session data in a cookie that’s passed back and forth between client and server. While this approach allows the server to be completely stateless with respect to the client, it has its flaws. First, even though the data is application state data, it’s still owned by the server, not the client. Most clients don’t even try to interpret this data. They just hand it back to the server on each successive request. But this data is application state data, so the client should manage it, not the server.

There’s no good answers to these questions yet. What it comes down to is that service design is a series of trade-offs. If you really need your web service to scale to billions of users, then you’d better find ways to make your architecture compliant with REST principles. If you’re only worried about servicing a few thousand users at a time, then perhaps you can relax the constraints a bit. The point is that you should understand the constraints, and then make informed design decisions.

Java Https Key Setup

In my last article, I showed how to remove all security from a secure web (https) transaction by installing dummy trust manager and host name verifier objects into an SSLSocketFactory. Today, I’m going to take it to the next level by demonstrating how to create a private key and self-signed certificate in a JKS keystore, exporting the public key certificate to a client-side trust store, and configuring our client to use the trust store to verify our server.

I’ll be using a Tomcat 6 server – mainly because it’s almost trivial to install and configure for SSL traffic. On my OpenSuSE 11.1 64-bit GNU/Linux machine, I’ve installed the tomcat6 package, and then I’ve gone into YaST’s service management panel and enabled the tomcat6 service.

Self-Signed Certificates

Let’s start by generating the proper keys. First, we’ll generate the server’s self-signed certificate, with embedded public/private key pair. For the common name (CN) field, I’ll make sure to enter the fully qualified domain name of my server (jmc-linux-64.provo.novell.com). This will ensure that my Java client code will properly compare the hostname used in my URL with the server’s certificate. Using any other value here would cause my client to fail with an invalid hostname exception. Here’s the Java keytool command line to create a self-signed certificate in a JKS key store called jmc-linux-64.keystore.jks:

$ keytool -genkey -alias jmc-linux-64 \
 -keyalg RSA -keystore jmc-linux-64.keystore.jks
Enter keystore password: password
Re-enter new password: password
What is your first and last name?
  [Unknown]:  jmc-linux-64.provo.novell.com
What is the name of your organizational unit?
  [Unknown]:  Engineering
What is the name of your organization?
  [Unknown]:  Novell, Inc.
What is the name of your City or Locality?
  [Unknown]:  Provo
What is the name of your State or Province?
  [Unknown]:  Utah
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=jmc-linux-64.provo.novell.com, OU=Engineering,
 O="Novell, Inc.", L=Provo, ST=Utah, C=US correct?
  [no]:  yes

Enter key password for 
         (RETURN if same as keystore password): <CR>
		
$

To view the new certificate and key pair, just use the -list option, along with the -v (verbose) option, like this:

$ keytool -list -v -keystore jmc-linux-64.keystore.jks
Enter keystore password: password

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: jmc-linux-64
Creation date: Jun 19, 2009
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Issuer: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Serial number: 4a3c006f
Valid from: Fri Jun 19 15:17:35 MDT 2009 until: Thu Sep 17 15:17:35 MDT 2009
Certificate fingerprints:
         MD5:  E5:37:9F:85:C9:76:60:FC:DC:01:81:AD:5F:FC:F4:9A
         SHA1: FD:E3:47:6C:AE:9B:75:3B:9C:6C:05:7B:C9:A4:B4:E6:07:F6:B5:FB
         Signature algorithm name: SHA1withRSA
         Version: 3


*******************************************
*******************************************

$

Server Configuration

Okay, now we have a server certificate with public and private key pair in a JKS keystore. The next step is to configure Tomcat to listen for https requests. The default configuration for Tomcat is to run a bare http server on port 8080. To enable the https server on port 8443, I edited the /usr/share/tomcat6/conf/server.xml file and uncommented the default entry for SSL that was already in place as a comment:

...
<!-- Define a SSL HTTP/1.1 Connector on port 8443
     This connector uses the JSSE configuration, when using APR, the
     connector should be using the OpenSSL style configuration
     described in the APR documentation -->

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
           maxThreads="150" scheme="https" secure="true"
           keystoreFile="/jmc-linux-64.keystore.jks" 
           keystorePass="password"
           clientAuth="false" sslProtocol="TLS" />
...

Make sure the sslProtocol is set to at least “SSLv3” – I just used “TLS” here. The important fields, however, are the keystoreFile and keystorePass fields, which I’ve set to the keystore we created in the previous step, and its password. You can put the keystore file anywhere on your file system accessible by the user running the tomcat service. On my system, the tomcat6 service is executed as root by default, so I just copied my keystore to the root of my file system.

After editing the file, I had to restart the tomcat6 service:

# rctomcat6 restart
Shutting down Tomcat (/usr/share/tomcat6)	... done
Starting Tomcat (/usr/share/tomcat6)		... done
#

Client-Side Trust Store

So much for server configuration. Now we have to configure the client’s trust store with the server’s self-signed certificate. This is done by exporting the certificate and public key from the server’s keystore, and then importing it into a client trust store. A trust store is just a JKS keystore that contains only trust certificates:

$ keytool -export -alias jmc-linux-64 \
 -keystore jmc-linux-64.keystore.jks -rfc \
 -file jmc-linux-64.cert
Enter keystore password: password
Certificate stored in file 
$
$ cat jmc-linux-64.cert
-----BEGIN CERTIFICATE-----
MIICezCCAeSgAwIBAgIESjwAbzANBgkqhkiG9w0BAQUFADCBgTELMAkGA1UEBhMCVVMxDTALBgNV
BAgTBFV0YWgxDjAMBgNVBAcTBVByb3ZvMRUwEwYDVQQKEwxOb3ZlbGwsIEluYy4xFDASBgNVBAsT
C0VuZ2luZWVyaW5nMSYwJAYDVQQDEx1qbWMtbGludXgtNjQucHJvdm8ubm92ZWxsLmNvbTAeFw0w
OTA2MTkyMTE3MzVaFw0wOTA5MTcyMTE3MzVaMIGBMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRh
aDEOMAwGA1UEBxMFUHJvdm8xFTATBgNVBAoTDE5vdmVsbCwgSW5jLjEUMBIGA1UECxMLRW5naW5l
ZXJpbmcxJjAkBgNVBAMTHWptYy1saW51eC02NC5wcm92by5ub3ZlbGwuY29tMIGfMA0GCSqGSIb3
DQEBAQUAA4GNADCBiQKBgQCOwb5migz+c1mmZS5eEhBQ5wsYFuSmp6bAL7LlHARQxhZg62FEVBFL
Y2klPoCGfUoXUFegnhCV5I37M0dAQtNLSHiEPj0NjAvWuzagevE6Tq+0zXEBw9fKoVV/ypEsAxEX
6JQ+a1WU2W/vdL+x0lEbRpRCk9t6yhxLw16M/VD/GwIDAQABMA0GCSqGSIb3DQEBBQUAA4GBAC5E
kniYYFxwZUqg9ToFlF0LKjGZfttkXJoTMfOFwA6OXrO6cKdzS04srxhoDzkD8V4RskPxttt0pbKr
iAoGKT/9P4hpDb0Ej4urek9TxlrnoC8g0rOYaDfE57SMStDrCg2ha4IuJFtJOh1aMcl4pm/sk+JW
7U/cWyW9B7InJinZ
-----END CERTIFICATE-----

$
$ keytool -import -alias jmc-linux-64 \
 -file jmc-linux-64.cert \
 -keystore jmc-linux-64.truststore.jks
Enter keystore password: trustpass
Re-enter new password: trustpass
Owner: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Issuer: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Serial number: 4a3c006f
Valid from: Fri Jun 19 15:17:35 MDT 2009 until: Thu Sep 17 15:17:35 MDT 2009
Certificate fingerprints:
         MD5:  E5:37:9F:85:C9:76:60:FC:DC:01:81:AD:5F:FC:F4:9A
         SHA1: FD:E3:47:6C:AE:9B:75:3B:9C:6C:05:7B:C9:A4:B4:E6:07:F6:B5:FB
         Signature algorithm name: SHA1withRSA
         Version: 3
Trust this certificate? [no]:  yes
Certificate was added to keystore

$

We now have a file called jmc-linux-64.truststore.jks, which contains only the server’s public key and certificate. You can show the contents of the truststore JKS file with the -list option, like this:

$ keytool -list -v -keystore jmc-linux-64.truststore.jks
Enter keystore password: trustpass

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: jmc-linux-64
Creation date: Jun 19, 2009
Entry type: trustedCertEntry

Owner: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Issuer: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Serial number: 4a3c006f
Valid from: Fri Jun 19 15:17:35 MDT 2009 until: Thu Sep 17 15:17:35 MDT 2009
Certificate fingerprints:
         MD5:  E5:37:9F:85:C9:76:60:FC:DC:01:81:AD:5F:FC:F4:9A
         SHA1: FD:E3:47:6C:AE:9B:75:3B:9C:6C:05:7B:C9:A4:B4:E6:07:F6:B5:FB
         Signature algorithm name: SHA1withRSA
         Version: 3


*******************************************
*******************************************

$

A Simple Https Client

We have several options for how to consume this trust store in client code. I’ll take the easy route today, but watch for another article that describes more complex mechanisms that provide more flexibility. Today, I’ll just show you how to set system properties on our client application. This client is very simple. All it does is connect to the server and display the contents of the web page in raw html to the console:

import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

public class HttpsClient
{
  private final String serverUrl;

  public HttpsClient(String serverUrl) 
  {
    this.serverUrl = serverUrl;
  }

  public void connect() 
  {
    try
    {
      HttpURLConnection conn = null;
      URL url = new URL(serverUrl);

      try
      {
        conn = (HttpURLConnection)url.openConnection();
        conn.setRequestMethod("GET");
        conn.setDoOutput(false);
        conn.setDoInput(true);
        conn.connect();
        InputStream is = conn.getInputStream();

        Integer bytes;
        byte [] buffer = new byte[512];
        while ((bytes = is.read(buffer, 0, 512)) > 0)
          System.out.write(buffer, 0, bytes);
      }
      catch (IOException e) { e.printStackTrace(); }
    }
    catch(MalformedURLException e) { e.printStackTrace(); }
  }

  public static void main(String[] args) 
  {
    HttpsClient client = new HttpsClient(
        "https://jmc-linux-64.provo.novell.com:8443");
    client.connect();
  }
}

Executing this client as is, without an assigned trust store will cause it to use the default trust store ($JAVA_HOME/lib/security/cacerts), which doesn’t contain our server’s public certificate, so it will fail with an exception:

javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: 
PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
... stack trace ...
Caused by: 
sun.security.validator.ValidatorException: 
PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
... stack trace ...
Caused by: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
... stack trace ...  

Configuring the Client Trust Store

The quick way to get this client to work properly is to assign our client’s trust store (containing the server’s public key and self-signed certificate) to JSSE system properties in this manner:

$ java -Djavax.net.ssl.trustStore=jmc-linux-64.truststore.jks \
  -Djavax.net.ssl.trustStorePassword=trustword

If you get the path to the trust store file wrong, you’ll get a different cryptic exception:

javax.net.ssl.SSLException: 
java.lang.RuntimeException: Unexpected error: 
java.security.InvalidAlgorithmParameterException: 
the trustAnchors parameter must be non-empty
... stack trace ...
Caused by: java.lang.RuntimeException: Unexpected error: 
java.security.InvalidAlgorithmParameterException: 
the trustAnchors parameter must be non-empty
... stack trace ...
Caused by: 
java.security.InvalidAlgorithmParameterException: 
the trustAnchors parameter must be non-empty
... stack trace ...

And if you get the password wrong, you’ll get yet another (somewhat less) cryptic exception:

java.net.SocketException: 
java.security.NoSuchAlgorithmException: 
Error constructing implementation 
(algorithm: Default, provider: SunJSSE, 
class: com.sun.net.ssl.internal.ssl.DefaultSSLContextImpl)
... stack trace ...
Caused by: java.security.NoSuchAlgorithmException: 
Error constructing implementation 
(algorithm: Default, provider: SunJSSE, 
class: com.sun.net.ssl.internal.ssl.DefaultSSLContextImpl)
... stack trace ...
Caused by: java.io.IOException: 
Keystore was tampered with, or password was incorrect
... stack trace ...
Caused by: java.security.UnrecoverableKeyException: 
Password verification failed
... stack trace ...

In these examples, my client is using my server’s fully qualified domain name in the URL, which is the common name we used when we created the self-signed certificate:

  ...
  public static void main(String[] args) 
  {
    HttpsClient client = new HttpsClient(
        "https://jmc-linux-64.provo.novell.com:8443");
    client.connect();
  }
}

This is the only name that will work with this trust store. In my next article I’ll show you how to generate certificates that work with aliases like the IP address. I’ll also show you how to add a hostname verifier to allow our client code to be a bit more intelligent about which aliases it rejects out of hand.

Java HTTPS Client Issues

I’ve written in the last several months about creating a client for a RESTful web-based auditing service. In that client, I had to implement client-side authentication, which is much more involved (or it should be anyway) than writing a simple secure web client that accesses content from secure public web servers.

Such a simple secure web client has only a little more functionality than a non-secure (http) web client. Essentially, it must perform a check after each connection to the secure web server to ensure that the server certificate is valid and trustworthy. This involves basically two steps:

  1. Verifying the server’s certificate chain.
  2. Verifying the server’s host name against that certificate.

Verifying the Certificate

The purpose of step 1 is to ensure that the service you’re attempting to use is not trying
to pull something shady on you. That is, the owner of the service was willing to put his or her name on the line with a Certificate Authority (CA) like Entrust or VeriSign. When you purchase a CA-signed certificate, you have to follow various procedures that document who you are, and why you’re setting up the service. But don’t worry – the CA doesn’t get to determine if your service is worthy of public consumption. Rather, only that you are who you say you are. The CA verifies actual existence, names, addresses, phone numbers, etc. If there’s any question about the service later, a consumer may contact that CA to find out the details of the service provider. This is dangerous for scam artists because they can be tracked and subsequently prosecuted. Thus, they don’t want to deal with Certificate Authorities if they don’t have to.

The client’s verification process (step 1) usually involves following the certificates in the certificate chain presented by the server back to a CA-signed certificate installed in its own trust store. A normal Sun JRE comes with a standard JKS truststore in $JAVA_HOME/lib/security/cacerts. This file contains a list of several dozen world-renowned public Certificate Authority certificates. By default, the SSLContext object associated with a normal HTTPSURLConnection object refers to a TrustManager object that will compare the certificates in the certificate chain presented by servers with the list of public CA certificates in the cacerts trust store file.

If you have an older cacerts file that doesn’t happen to contain a certificate for a site to which you are connecting, or if you’ve set up the site yourself using a self-signed certificate, then you’ll encounter an exception when you attempt to connect:

javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: 
PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target

Ouch! Does this mean you can’t connect to your test server while writing your client code? Can you only test against public servers? No, of course not, but unfortunately, it does mean a bit more work for you. You have basically two options. The first is to install your test server’s self-signed certificate into your default trust store. I first learned about this technique from a blog entry by Andreas Sterbenz in October of 2006. Nice article, Andreas. Thanks!

However, there is another way. You can write some temporary code in the form of your own sort of dumb trust manager that accepts any certificate from any server. Of course, you don’t want to ship your client with this code in place, but for testing and debugging, it’s nice not to have to mess up your default trust store with temporary certs that you may not want in there all the time. Writing DumbX509TrustManager is surprisingly simple. As with most well-considered Java interfaces, the number of required methods for the X509TrustManager interface is very small:

public class MyHttpsClient
{
  private Boolean isSecure;
  private String serverURL;

  private class DumbX509TrustManager 
      implements X509TrustManager 
  {
    public void checkClientTrusted(X509Certificate[] chain, 
        String authType) throws CertificateException {}

    public void checkServerTrusted(X509Certificate[] chain, 
        String authType) throws CertificateException {}

    public X509Certificate[] getAcceptedIssuers() 
        { return new X509Certificate[] {}; }
  }
  ...

To make use of this trust manager, simply obtain an SSLSocketFactory object in your client’s constructor that you can configure with your dumb trust manager. Then, as you establish https connections to your test server, install your preconfigured SSLSocketFactory object, like this:

  ...
  private SSLSocketFactory getSocketFactory()
  {
    SSLSocketFactory socketFactory = null;
    try
    {
      SSLContext context = SSLContext.getInstance("SSLv3");
      context.init(null, new X509TrustManager[] 
          { new DumbX509TrustManager() }, null);
      socketFactory = context.getSocketFactory();
    }
    catch (Exception e) { e.printstacktrace(); }
    return socketFactory;
  }

  public MyHttpsClient(String serverURL)
  {
    this.serverURL = serverURL;
    if (isSecure = serverURL.startsWith("https:"))
      sslSocketFactory = getSocketFactory();
  }

  public void process() 
  {
    try
    {
      HttpURLConnection conn = null;
      URL url = new URL(serverURL);
      try
      {
        conn = (HttpURLConnection)url.openConnection();
        if (isSecure)
        {
          HttpsURLConnection sconn = (HttpsURLConnection)conn;
              sconn.setSSLSocketFactory(sslSocketFactory);
        }
        conn.setRequestMethod(verb);
        conn.setDoOutput(false);
        conn.setDoInput(true);
        conn.connect();
        ...

That’s it. Warning: Don’t ship your client with DumbX509TrustManager in place. You don’t need it for public secure web servers anyway. If you know your client will only ever be used against properly configured public secure web servers, then you can rely on the default trust manager in the default socket factory associated with HttpsURLConnection.

If you think your client may be expected to work with non-public secure web servers with self-signed, or company-signed certificates, then you have more work to do. Here, you have two options. You can write some code similar to that found in browsers, wherein the client displays a dialog box upon connection, asking if you would like to connect to this “unknown” server just this once, or forever (where upon, the client then imports the server’s certificate into the default trust store). Or you can allow your customer to pre-configure the default trust store with certificates from non-public servers that he or she knows about in advance. But these are topics for another article.

Verifying the Server

Returning to the original two-step process, the purpose of step 2 (host name verification) is to ensure that the certificate you received from the service to which you connected was not stolen by a scammer.

When a CA-signed certificate is generated, the information sent to the Certificate Authority by the would-be service provider includes the fully qualified domain name of the server for which the new cert is intended. This FQDN is embedded in a field of the certificate, which the client uses to ensure that the server is really the owner of the certificate that it’s presenting.

As I mentioned in a previous article, Java’s keytool utility won’t let you generate self-signed certs containing the FQDN in the proper field, thus the default host name verification code will always fail with self-signed certs generated by keytool. Again, a simple dummy class comes to the rescue in the form of the DumbHostnameVerifier class. Just implement the HostnameVerifier interface, which has one required method, verify. Have it return true all the time, and you won’t see anymore Java exceptions like this:

HTTPS hostname wrong:  
should be <jmc-linux-64.provo.novell.com>

Here’s an example:

  ...
  private class DumbHostnameVerifier 
      implements HostnameVerifier
  {
    public boolean verify(String arg0, SSLSession arg1) 
        { return true; }
  }
  ...
  public void process() 
  {
        ...
        if (isSecure)
        {
          HttpsURLConnection sconn = (HttpsURLConnection)conn;
          sconn.setSSLSocketFactory(sslSocketFactory);
          sconn.setHostnameVerifier(new DumbHostnameVerifier());
        }
        ...

Scoping the Changes

A final decision you should make is the proper scope for setting the dummy trust manager and hostname verifier objects. The JSSE framework is extremely flexible. You can set these on a per-request basis, or as the class defaults, so that whenever a new HttpsURLConnection object is created, your objects are automatically assigned to them internally. For instance, you can use the following code to setup class default values:

public class MyHttpsClient
{
  private static class DumbX509TrustManager 
      implements X509TrustManager 
  {
    public void checkClientTrusted(X509Certificate[] chain, 
        String authType) throws CertificateException {}

    public void checkServerTrusted(X509Certificate[] chain, 
        String authType) throws CertificateException {}

    public X509Certificate[] getAcceptedIssuers() 
        { return new X509Certificate[] {}; }
  }

  private static class DumbHostnameVerifier 
      implements HostnameVerifier
  {
    public boolean verify(String arg0, SSLSession arg1) 
        { return true; }
  }

  private static SSLSocketFactory getSocketFactory()
  {
    SSLSocketFactory socketFactory = null;
    try
    {
      SSLContext context = SSLContext.getInstance("SSLv3");
      context.init(null, new X509TrustManager[] 
          { new DumbX509TrustManager() }, null);
      socketFactory = context.getSocketFactory();
    }
    catch (Exception e) { e.printstacktrace(); }
    return socketFactory;
  }

  static
  {
    HttpsURLConnection.setDefaultHostnameVerifier(
        new DumbHostnameVerifier());
    HttpsURLConnection.setDefaultSSLSocketFactory(
        getSocketFactory());
  }

  private String serverURL;
  
  public MyHttpsClient(String serverURL)
  {
    this.serverURL = serverURL;
  }
  ...

You can now remove the isSecure check in the process routine, because new instances of HttpsURLConnection will automatically be assigned objects of your new trust manager and hostname verifier classes – the default objects you stored in the classes with the HttpsClient class’s static initializer.

With that, you’re set to connect to any https server. Here’s a little insight for you: The difficult part – the real work – of writing https clients involves writing real code for these classes. I’ll write a future article that provides details on these processes. Again, I remind you: Don’t accidentally ship your clients with DumbHostnameVerifier in place! (Unless, of course, you want to. After all, it’s your code…)

Java Secure HTTP Keys, Part II

In my last article, I described the process of configuring client-side key and trust stores within a Java web client application. To keep it simple, I purposely used the built-in functionality of HttpsURLConnection to read certain System properties to obtain references to these credential stores, along with their passwords.

However, for an embedded client–as would be the case with library code–you’d not want to rely on any System properties, because these belong to your user and her application, not to your library. But, manually configuring the key and trust stores for a client-side https connection is a little more involved.

In this article, I’d like to show you how it’s done, and I’d like to begin by suggesting some required reading for a solid understanding of the way it all works. I’m referring to the Java Secure Socket Extensions (JSSE) Reference Guide. Since JSSE was introduced in Java 1.4, and hasn’t really changed much since then, this document is officially up to date–even in Java SE 6.

Getting Started…

Note that the process for setting up the key and trust stores hasn’t changed, so I’ll simply refer you to my previous article for this information.

To summarize, the goal here is to associate our key and trust stores with our client-side connections without specifying them in System properties. And it’s amazing the amount of extra work we have to go through in order to accomplish this seemingly simple task.

The first thing we’ll do is remove the calls to System.setProperty in our AuditRestClient constructor. We still need the values we wrote to those properties, so we’ll just convert them to constants in the AuditRestClient class. At some later point, these should undoubtedly be converted to properties that we read from our own configuration file, but for now, these constants will do:

  public class AuditRestClient
  {
    // URL components (should be configured variables)
    private static final String HTTP = "HTTP";
    private static final String HTTPS = "HTTPS";
    private static final String HOSTNAME = "10.0.0.1";
    private static final Integer PORT = 9015;

    // secure channel key material stores (should be configured)
    private static final String keystore = "/tmp/keystore.jks";
    private static final String truststore = "/tmp/truststore.jks";
    private static final String keypass = "changeit";
    private static final String trustpass = "changeit";

    // secure channel variables
    private Boolean isSecure = true;
    private SSLSocketFactory sslSocketFactory = null;

    public AuditRestClient()
    {
      setupSocketFactory();
    }
    ...

Building Your Own Socket Factory

The new version of the AuditRestClient constructor calls a private method called setupSocketFactory, which configures an SSLSocketFactory object for use later when we configure our HttpsURLConnection object. Here’s the code:

    ...
    private void setupSocketFactory()
    {
      try
      {
        String protocol = "TLS";
        String type = "JKS";
        String algorithm = KeyManagerFactory.getDefaultAlgorithm();
        String trustAlgorithm =
            TrustManagerFactory.getDefaultAlgorithm();

        // create and initialize an SSLContext object
        SSLContext sslContext = SSLContext.getInstance(protocol);
        sslContext.init(getKeyManagers(type, algorithm),
            getTrustManagers(type, trustAlgorithm),
            new SecureRandom());

        // obtain the SSLSocketFactory from the SSLContext
        sslSocketFactory = sslContext.getSocketFactory();
      }
      catch (Exception e) { e.printStackTrace(); }
    }
    ...

This private helper method calls two other private methods, getKeyManagers and getTrustManagers to configure the key and trust stores. Each of these two routines also call a routine named getStore to obtain the key and trust stores from the configured key and trust managers. Again, here’s the code for all three of these methods:

    ...
    private KeyStore getStore(String type,
        String filename, String pwd) throws Exception
    {
      KeyStore ks = KeyStore.getInstance(type);
      InputStream istream = null;

      try
      {
        File ksfile = new File(filename);
        istream = new FileInputStream(ksfile);
        ks.load(istream, pwd != null? pwd.toCharArray(): null);
      }
      finally { if (istream != null) istream.close(); }

      return ks;
    }

    private KeyManager[] getKeyManagers(String type,
        String algorithm) throws Exception
    {
      KeyStore ks = getStore(type, keyStore, keyPass);
      KeyManagerFactory kmf =
          KeyManagerFactory.getInstance(algorithm);

      kmf.init(ks, keypass.toCharArray());

      return kmf.getKeyManagers();
    }

    private TrustManager[] getTrustManagers(String type,
        String algorithm) throws Exception
    {
      KeyStore ts = getStore(type, trustStore, trustPass);
      TrustManagerFactory tmf =
          TrustManagerFactory.getInstance(algorithm);

      tmf.init(ts);

      return tmf.getTrustManagers();
    }
    ...

The getStore method calls KeyStore.getInstance to obtain an instance of the key store associated with the specified type–in this case, “JKS”. It should be noted that if you wish to specify your own provider, you may do so by calling the other version of KeyStore.getInstance, which accepts a string provider name, as well.

Using Your New Socket Factory

Now that you have your socket factory built (whew!), it’s time to look at how it’s used by the rest of the AuditRestClient code. Here’s the context for the use of the new object:

    public void send(JSONObject event)
    {
      byte[] bytes = event.toString().getBytes();
      HttpURLConnection conn = null;

      try
      {
        URL url = new URL(isSecure? HTTPS: HTTP,
            HOSTNAME, PORT, "/audit/log/test");
        conn = (HttpURLConnection)url.openConnection();
        if (isSecure)
        {
          HttpsURLConnection sconn = (HttpsURLConnection)conn;
          sconn.setSSLSocketFactory(sslSocketFactory);
        }
        conn.setRequestMethod("POST");
        conn.addRequestProperty("Content-Length", "" + bytes.length);
        conn.addRequestProperty("Content-Type", CTYPE);
        conn.addRequestProperty("Connection", "Keep-Alive");
        conn.setDoOutput(true);
        conn.setDoInput(true);
        conn.connect();
        ...

Now, this code is completely independent of application owned System properties. Additionally, it’s portable between secure and non-secure HTTP channels. This protocol portability requires a type cast of the connection from HttpURLConnection to HttpsURLConnection in one place (as highlighted in the example above in bold text).

You may have also noticed that I converted the previous version of send to use the other popular form of the URL constructor. This form accepts constituent parts of the URL as separate parameters, rather than as a single string. It’s a bit more efficient under the covers, as the constructor doesn’t need to parse these components from the URL string. It made more sense on my end, as well since I’m parameterizing several of these parts now anyway. Attributes like HOSTNAME and PORT will eventually be read from a library configuration file.

Java Secure HTTP Client Key Management

My current project at Novell involves the development of a ReSTful web service for submission of audit records from security applications. The server is a Jersey servlet within an embedded Tomcat 6 container.

One of the primary reasons for using a ReSTful web service for this purpose is to alleviate the need to design and build a heavy-weight audit record submission client library. Such client libraries need to be orthogonally portable across both hardware platforms and languages in order to be useful to Novell’s customers. Just maintaining the portability of this client library in one language is difficult enough, without adding multiple languages to the matrix.

Regardless of our motivation, we still felt the need to provide a quality reference implementation of a typical audit client library to our customers. They may incorporate as much or as little of this code as they wish, but a good reference implementation is worth a thousand pages of documentation. (Don’t get me wrong, however–this is no excuse for not writing good documentation! The combination of quality concise documentation and a good reference implementation is really the best solution.)

The idea here is simple: Our customers won’t have to deal with difficulties that we stumble upon and then subsequently provide solutions for. Additionally, it’s just plain foolish to provide a server component for which you’ve never written a client. It’s like publishing a library API that you’ve never written to. You don’t know if the API will even work the way you originally intended until you’ve at least tried it out.

Since we’re already using Java in the server, we’ve decided that our initial client reference implementation should also be written in Java. Yesterday found my code throwing one exception after another while simply trying to establish the TLS connection to the server from the client. All of these problems ultimately came down to my lack of understanding of the Java key store and trust store concepts.

You see, the establishment of a TLS connection from within a Java client application depends heavily on the proper configuration of a client-side trust store. If you’re using mutual authentication, as we are, then you also need to properly configure a client-side key store for the client’s private key. The level at which we are consuming Java network interfaces also demands that we specify these stores in system properties. More on this later…

Using Curl as an Https Client

We based our initial assumptions about how the Java client needed to be configured on our use of the curl command line utility in order to test the web service. The curl command line looks something like this:

  curl -k --cert client.cer --cert-type DER --key client-key.pem
    --key-type PEM --header "Content-Type: application/audit+json"
    -X POST --data @test-event.json https://10.0.0.1:9015/audit/log/test

The important aspects of this command-line include the use of the –cert, –cert-type, –key and –key-type parameters, as well as the fact that we specified a protocol scheme of “https” in the URL.

With one exception, the remaining options are related to which http method to use (-X), what data to send (–data), and which message properties to send (–header). The exception is the -k option, and therein lay most of our problems with this Java client.

The curl man-page indicates that the -k/–insecure option allows the TLS handshake to succeed without verifying the server certificate in the client’s CA (Certificate Authority) trust store. The reason this option was added was because several releases of the curl package shipped with a terribly out-dated trust store, and people were getting tired of having to manually add certificates to their trust stores everytime they hit a newer site.

Doing it in Java

But this really isn’t the safe way to access any secure public web service. Without server certificate verification, your client can’t really know that it’s not communicating with a server that just says it’s the right server. (“Trust me!”)

During the TLS handshake, the server’s certificate is passed to the client. The client should then verify the subject name of the certificate. But verify it against what? Well, let’s consider–what information does the client have access to, outside of the certificate itself? It has the fully qualified URL that it used to contact the server, which usually contains the DNS host name. And indeed, a client is supposed to compare the CN (Common Name) portion of the subject DN (Distinguished Name) in the server certificate to the DNS host name in the URL, according to section 3.1 “Server Identity” of RFC 2818 “HTTP over TLS”.

Java’s HttpsURLConnection class strictly enforces the advice given in RFC 2818 regarding peer verification. You can override these constraints, but you have to basically write your own version of HttpsURLConnection, or sub-class it and override the methods that verify peer identity.

Creating Java Key and Trust Stores

Before even attempting a client connection to our server, we had to create three key stores:

  1. A server key store.
  2. A client key store.
  3. A client trust store.

The server key store contains the server’s self-signed certificate and private key. This store is used by the server to sign messages and to return credentials to the client.

The client key store contains the client’s self-signed certificate and private key. This store is used by the client for the same purpose–to send client credentials to the server during the TLS mutual authentication handshake. It’s also used to sign client-side messages for the server during the TLS handshake. (Note that once authentication is established, encryption happens using a secret or symetric key encryption algorithm, rather than public/private or asymetric key encryption. Symetric key encryption is a LOT faster.)

The client trust store contains the server’s self-signed certificate. Client-side trust stores normally contain a set of CA root certificates. These root certificates come from various widely-known certificate vendors, such as Entrust and Verisign. Presumably, almost all publicly visible servers have a purchased certificate from one of these CA’s. Thus, when your web browser connects to such a public server over a secure HTTP connection, the server’s certificate can be verified as having come from one of these well-known certificate vendors.

I first generated my server key store, but this keystore contains the server’s private key also. I didn’t want the private key in my client’s trust store, so I extracted the certificate into a stand-alone certificate file. Then I imported that server certificate into a trust store. Finally, I generated the client key store:

  $ keytool -genkey -alias server -keyalg RSA \
  > -storepass changeit -keystore server-keystore.jks
  What is your first and last name?
    [Unknown]:  audit-server
  What is the name of your organizational unit?
    [Unknown]:  Eng
  What is the name of your organization?
    [Unknown]:  Novell
  What is the name of your City or Locality?
    [Unknown]:  Provo
  What is the name of your State or Province?
    [Unknown]:  Utah
  What is the two-letter country code for this unit?
    [Unknown]:  US
  Is CN=audit-server, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US correct?
    [no]:  yes

  Enter key password for <server>
          (RETURN if same as keystore password):  
  $
  $ keytool -exportcert -keystore server-keystore.jks \
  > -file server.der -alias server -storepass changeit
  Certificate stored in file <server.der>
  $
  $ keytool -importcert -trustcacerts -alias server \
  > -keystore server-truststore.jks -storepass changeit \
  > -file server.der
  Owner: CN=audit-server, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Issuer: CN=audit-server, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Serial number: 491cad67
  Valid from: Thu Nov 13 15:42:47 MST 2008 until: Wed Feb 11 15:42:47 MST 2009
  Certificate fingerprints:
           MD5:  EE:FA:EE:78:A8:42:2B:F2:3A:04:50:37:D3:94:B3:C0
           SHA1: 4E:BA:9B:2F:FC:84:10:5A:2E:62:D2:5B:B3:70:70:B5:2F:03:E1:CD
	   Signature algorithm name: SHA1withRSA
           Version: 3
  Trust this certificate? [no]:  yes
  Certificate was added to keystore
  $
  $ keytool -genkey -alias client -keyalg RSA -storepass changeit \
  > -keystore client-keystore.jks
  What is your first and last name?
    [Unknown]:  audit-client
  What is the name of your organizational unit?
    [Unknown]:  Eng
  What is the name of your organization?
    [Unknown]:  Novell
  What is the name of your City or Locality?
    [Unknown]:  Provo
  What is the name of your State or Province?
    [Unknown]:  Utah
  What is the two-letter country code for this unit?
    [Unknown]:  US
  Is CN=audit-client, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US correct?
    [no]:  yes

  Enter key password for <client>
          (RETURN if same as keystore password):  
  $
  $ ls -1
  client-keystore.jks
  server.der
  server-keystore.jks
  server-truststore.jks
  $

Telling the Client About Keys

There are various ways of telling the client about its key and trust stores. One method involves setting system properties on the command line. This is commonly used because it avoids the need to enter absolute paths directly into the source code, or to manage separate configuration files.

  $ java -Djavax.net.ssl.keyStore=/tmp/keystore.jks ...

Another method is to set the same system properties inside the code itself, like this:

  public class AuditRestClient
  {
    public AuditRestClient() 
    {
      System.setProperty("javax.net.ssl.keyStore", 
          "/tmp/keystore.jks");
      System.setProperty("javax.net.ssl.keyStorePassword", 
          "changeit");
      System.setProperty("javax.net.ssl.trustStore", 
          "/tmp/truststore.jks");
      System.setProperty("javax.net.ssl.trustStorePassword", 
          "changeit");
    }
    ...

I chose the latter, as I’ll eventually extract the strings into property files loaded as needed by the client code. I don’t really care for the fact that Java makes me specify these stores in system properties. This is especially a problem for our embedded client code, because our customers may have other uses for these system properties in the applications in which they will embed our code. Here’s the rest of the simple client code:

    ...
    public void send(JSONObject event) 
    {
      byte[] bytes = event.toString().getBytes();
      HttpURLConnection conn = null; 
		
      try
      {
        // establish connection parameters
        URL url = new URL("https://10.0.0.1:9015/audit/log/test");
        conn = (HttpURLConnection)url.openConnection();
        conn.setRequestMethod("POST");
        conn.addRequestProperty("Content-Length", "" + bytes.length);
        conn.addRequestProperty("Content-Type", "application/audit1+json");
        conn.setDoOutput(true);
        conn.setDoInput(true);
        conn.connect();

        // send POST data
        OutputStream out = (OutputStream)conn.getOutputStream();
        out.write(bytes);
        out.flush();
        out.close(); 		

        // get response code and data
        System.out.println(conn.getResponseCode());
        BufferedReader read = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        String query = null;
        while((query = read.readLine()) != null)
          System.out.println(query);
      }
      catch(MalformedURLException e) { e.printStackTrace(); }
      catch(ProtocolException e) { e.printStackTrace(); }
      catch(IOException e) { e.printStackTrace(); }
      finally { conn.disconnect(); }
    }
  }

Getting it Wrong…

I also have a static test “main” function so I can send some content. But when I tried to execute this test, I got an exception indicating that the server certificate didn’t match the host name. I was using a hard-coded IP address (10.0.0.1), but my certificate contained the name “audit-server”.

It turns out that the HttpsURLConnection class uses an algorithm to determine if the server that sent the certificate really belongs to the server on the other end of the connection. If the URL contains an IP address, then it attempts to locate a matching IP address in the “Alternate Names” portion of the server certificate.

Did you notice a keytool prompt to enter alternate names when you generated your server certificate? I didn’t–and it turns out there isn’t one. The Java keytool utility doesn’t provide a way to enter alternate names–a standardized extension of the X509 certificate format. To enter an alternate name containing the requisite IP address, you’d have to generate your certificate using the openssl utility, or some other more functional certificate generation tool, and then find a way to import these foreign certificates into a Java key store.

…And then Doing it Right

On the other hand, if the URL contains a DNS name, then HttpsURLConnection attempts to match the CN portion of the Subject DN with the DNS name. This means that your server certificates have to contain the DNS name of the server as the CN portion of the subject. Returning to keytool, I regenerated my server certificate and stores using the following commands:

  $ keytool -genkey -alias server -keyalg RSA \
  > -storepass changeit -keystore server-keystore.jks
  What is your first and last name?
    [Unknown]:  jmc-test.provo.novell.com

  ... (the rest is the same) ...
  
  $ keytool -exportcert -keystore server-keystore.jks \
  > -file server.der -alias server -storepass changeit
  Certificate stored in file <server.der>
  $
  $ keytool -importcert -trustcacerts -alias server \
  > -keystore server-truststore.jks -storepass changeit \
  > -file server.der
  Owner: CN=jmc-test.provo.novell.com, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Issuer: CN=jmc-test.provo.novell.com, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Serial number: 491cad67
  Valid from: Thu Nov 13 15:42:47 MST 2008 until: Wed Feb 11 15:42:47 MST 2009
  Certificate fingerprints:
           MD5:  EE:FA:EE:78:A8:42:2B:F2:3A:04:50:37:D3:94:B3:C0
           SHA1: 4E:BA:9B:2F:FC:84:10:5A:2E:62:D2:5B:B3:70:70:B5:2F:03:E1:CD
	   Signature algorithm name: SHA1withRSA
           Version: 3
  Trust this certificate? [no]:  yes
  Certificate was added to keystore
  $

Of course, I also had to change the way I was specifying my URL in the client code:

  ...
  URL url = new URL("https://jmc-test.provo.novell.com:9015/audit/log/test");
  conn = (HttpURLConnection)url.openConnection();
  ...

At this point, I was finally able to connect to my server and send the message. Is this reasonable? Probably not for my case. Both my client and server are within a corporate firewall, and controlled by the same IT staff, so to force this sort of gyration is really unreasonable. Can we do anything about it? Well, one thing that you can do is to provide a custom host name verifier like this:

  ...
  URL url = new URL("https://jmc-sentinel.dnsdhcp.provo.novell.com:9015/audit/AuditLog/test");
  conn = (HttpsURLConnection)url.openConnection();
  conn.setHostnameVerifier(new HostnameVerifier()
  { 
    public boolean verify(String hostname, SSLSession session) 
        { return true; }
  });
  conn.setRequestMethod("POST");
  ...

When you do this, however, you should be aware that you give up the right to treat the connection as anything but an https connection. Note that we had to change the type of “conn” to HttpsURLConnection from its original type of HttpURLConnection. This means, sadly, that this code will now only work with secure http connections. I chose to use the DNS name in my URL, although a perfectly viable option would also have been the creation of a certificate containing the IP address as an “Alternate Name”.

Is This Okay?!

Ultimately, our client code will probably be embedded in some fairly robust and feature-rich security applications. Given this fact, we can’t really expect our customers to be okay with our sample code taking over the system properties for key and trust store management. No, we’ll have to rework this code to do the same sort of thing that Tomcat does–manage our own lower-level SSL connections, and thereby import certificates and CA chains ourselves. In a future article, I’ll show you how to do this.

On Personal Internet Security…

I have over a hundred accounts on the Internet. I do a lot of online shopping for everything from books to computer hardware to toys for my kids. I also have accounts on various social networking sites like linked-in, facebook, myspace, plaxo, pulse and naymes. I use online authoring sites like wordpress, freesoftwaremagazine, digg, technorati and others. I like to personalize google news and various product support sites to my own tastes. I like it when sites like this allow me to create a profile – essentially a login account.

I also work in the software industry and write a fair amount of open source code, so I have accounts at locations like sourceforge.net, which manages separate authentication materials for mailing list accounts, primary site access, shell access, etc – and often for each project they support. This means literally dozens of passwords for a single site.

These places are all pretty benign as far as security issues are concerned. Frankly, I don’t really care if someone knows my middle name, or my phone number for that matter. But I do most of my banking online , and some web-based store fronts keep track of my credit card information these days. I have the option of not giving it to them, but if I trust them, I like to use this feature, and that presents a real security problem for me. Some of these sites have fairly good identity security–others do not. I don’t know which ones do and which ones don’t.

I used to use the same password everywhere – so I wouldn’t forget it. When I started doing online banking and storing credit card information at various store fronts, I used one password for these places, and another one for everywhere else. But lately the number of security classifications I use has increased significantly, making it difficult to remember all of the passwords I use.

If a hacker can break into one of these weaker sites, and capture account information and passwords, they can then access more sensitive personal information at many other sites where I have accounts. Now, I’m not a conspiracy theorist. I don’t believe there are groups of people out to get me personally. But I do believe in bad guys. And I know for a fact that there are bad guys out there “phishing” for random authentication materials. If they find a way to access one (like mine) and if they then find that I use the same password at my bank, I really do believe they’ll go after my cash. After all, they don’t really care whose money they take.

KeePass

Recently, I was introduced to the KeePass project on SourceForge.net. What a gem of a little project! KeePass allows you to store passwords and other account information in an easily accessible hierarchical format within an encrypted database on your hard drive. You only need to remember a single master password to get into the database.

Some people might balk at the idea of another layer of indirection between themselves and their online banking web site. I’d agree myself, if it weren’t for some of the really cool usability features in KeePass. For instance, KeePass can copy a password to the clipboard from an entry in its database, which means you need only click on the password entry field and press Ctrl-V to paste it in. If you care to take this to the next level, KeePass will also fill in login forms automatically with a configurable hot-key press on the login page of your sites.

KeePass also contains a small area in each password entry for notes and such. I have an AT&T cell phone account which allows me to connect to the Internet on my laptop through my phone over a high-speed connection. But configuring this connection initially was a real pain in the neck! Once I got it figured out, I wrote down the steps for configuration so I wouldn’t forget them. The next time I needed to reconfigure my laptop, I forgot where I’d written down these instructions. Now, I have them in the notes section for my AT&T wireless account in KeePass.

Another nice feature is that KeePass will automatically generate a high-security password for you, with a single click. When I create a new account on a web-site these days, I just pull up KeePass and create the account and the KeePass entry at the same time. When the site asks me for a password, I don’t waste time thinking about what I should use–I just tell KeePass to give me a good one, then cut and paste it in.

Finally, KeePass will stay resident on your Windows machine, adding a little icon to the system tray while it’s running. Click the icon and you have instant access to your password database. With highly configurable security policy tailored to your personal tastes, you can decide how often you want to type in your master password: Once at login, each time you click the system tray icon, only when you lock it, when you lock your computer screen, etc. You can also configure it to minimize to the tray, or to close to the tray.

Taking It With You

This is all well and good if you only work on one machine. I work on multiple machines. I have one at home where I spend time shopping, and I have one at work where I access my sourceforge.net accounts. I have a laptop that I take with me to sneak in some work or play while I’m waiting at the repair shop for my car to be fixed. Sometimes I use my wife’s laptop–just because it’s handy. Sometimes I use a kiosk computer at the airport or at the library. Sometimes I use a colleague’s computer in another office at work.

KeePass has a solution for this problem as well. If you wish, you can store the database on a removable media device, like a USB drive. You can pick up a 1G USB drive these days for 10 to 20 bucks. And this is 100 times as much memory as you need for a password store.

But the database does you little good if you can’t access it with the KeePass program when you need a password. The designers of KeePass understood this. You can store a portable version of the program itself on the USB drive. Portable, in this context, means programming in such a way that the software requires no explicit installation. It creates no registry entries, or special file system objects. This means you can access your password database from any Windows machine with a USB port. Just plug it in and run the program right from the USB drive.

What, Now Linux Too?!

What more could I ask for? Well…recently, I installed Linux on my desktop machine at work. Since moving to OpenSUSE 10.3, I’ve been very satisfied with what I’ve been able to accomplish using only free software. It’s been a whirlwind romance, and I’ve loved every minute of it, but it’s the first time I’ve been without a Windows machine handy to…you know, do the stuff I can only do on Windows. Sad to consider it that way, but it’s been true for me, so I’m guessing it’s true for most everyone else, as well.

Unfortunately, KeePass is a Windows program. “Well, I’m in love with the concept, not the program”, I told myself. So I went looking for a more portable alternative. One that was perhaps not as functional as KeePass, but at least ran on Windows and Linux. And I found it–KeePassX. This is a spin off of the original Windows open source program found on SourceForge.net.

KeePassX is written using QT and compiled under mingw on Windows, so its interfaces on both platforms are nearly identical. The people who did the port stayed true to the original KeePass look and feel as much as they could in this portable version. I’m very pleased, because now I can carry copies of KeePassX for Windows and Linux, as well as the database which, of course, both versions will open and process.

The only glitch I ran into with KeePassX was that it requires the mingwm10.dll, which fact is not advertised anywhere on the KeePassX web site that I could find, and the win32 package didn’t ship with this library. In fact, the only reference to it that I could find was an entry by a user in their forums indicating that they should probably mention the requirement somewhere. Personally, I think it’s an oversight, and that the Windows bundle should just install it.

To get the library, I just did a Google search for mingwm10 and found a myriad of places from which I could download it. I did that, placed the library in the same directory as the executable and all was well again.

Setting It All Up

To set all this up, I first formatted the USB key under Windows (because Linux has no problem reading FAT-formatted drives, and typically Windows only does Windows). Then I created a directory structure like this on the USB key:

Win32\
   ...unpacked files from KeePassX Win32 bundle
Linux\
   ...unpacked files from KeePassX Linux bundle
Install\
   ...bundles for both platforms, plus mingwm10 bundle, still packed
Passwords.kdb
Autorun.inf

Now, I like to do things up right. On Windows XP, when you insert a USB key, it acts like a removable drive–a CDROM or a USB hard drive. On these types of media, you can place a file at the root of the volume called Autorun.inf, which describes for Windows some things you’d like to have happen when the volume is mounted. I added the following text to an Autorun.inf file on the root of my USB key:

[autorun]
action="Run KeePassX"
open=Win32\KeePassX.exe
icon=Win32\KeePassX.exe
shell\keepassx=&KeePassX
shell\keepassx\command=Win32\KeePassX.exe

The “action” keyword allows Windows to display an option called “Run KeePassX” in the list of stuff to do when a drive is mounted that contains mixed media. Unfortunately, the graphic files (icons, bitmaps, etc) on a QT application are stored separately from the binary, so Windows interprets them as picture files. Since there are both pictures AND executables on the USB key, Windows doesn’t know what you really want to do, so it asks you every time you insert the USB key.

On Vista, you have a few more options. You can add more entries under a “[Contents]” section that tells Vista exactly what to do in the case of a conflict. To me, it’s a no-brainer to have done this in XP, but that’s not the way things came out, so we have to put up with the confusion. Most often, CDROM’s that contain executables designed to be run when the disk is inserted are installation CD’s for software you purchase. These have all sorts of media, but they often come packaged up in CAB or ZIP files, so Windows is not confused. There are only executables, so there’s no ambiguity. Windows just runs the setup.exe or install.exe program, as specified in the “open” tag.

When specifying an “action”, the “open” option tells Windows what to do if you select the “Run KeePassX” option in the pop up menu when the key is inserted. The “icon” option is really neat because it not only tells Windows what icon to display next to the action in the pop up, but also what icon to display in file explorer when the drive is mounted. The “shell” option is used to add a context menu option to the menu that comes up when you right-click on the drive in file explorer.

Look here at msdn.microsoft.com to learn more about Autoplay on Windows platforms.

Now, I’ve got the best of both worlds, and access to my password database from either place. Could I be any happier about the state of my personal Internet security? I don’t think so.

[Edit: I lost my password database the other day – it was corrupted when I pulled the USB key out of my Linux machine while the program was open. I think the corruption occurred because I popped it into a Windows machine, opened the database, and then put the key BACK into the Linux USB socket, and saved the database. In any case, I HIGHLY recommend you backup your password database once in a while. Luckily, I had a recent copy saved off somewhere, and I was able to get back about 95 percent of my data. Now, I keep a backup of the database on the same USB key in a “Backup” directory, which I overwrite quite often. I also keep a backup on another disk that I backup once a week or so, if I’ve made changes during the interim.

One person I know stores his database in a subversion repository, and updates it on any of his machines. That’s nice to get the latest version on any of your own machines, but it doesn’t help you when you want to access your store on a machine that’s not yours. Still, it’s a good idea to keep it in a repository like this.]