Managing a Dynamic Java Trust Store

This is the latest (and probably last) in my series of client-side Java key and trust store management articles, and a good summary article for the topic, I hope.

It’s clear from the design of SSLContext in the JSSE that Java key and trust stores are meant to contain static data. Yet browsers regularly display the standard security warning dialog when connecting to sites whose certificates have expired or whose administrators haven’t bothered to purchase a CA-signed certificate. This dialog generally offers you three choices:

  • Get me out of here!
  • I understand the risks: add certificate for this session only
  • I understand the risks: add certificate permanently

In this article, I’d like to elaborate on what it means to “add certificate” – either temporarily or permanently.

Let’s start with a simple Java http(s) client:

public byte[] getContentBytes(URI uri, SSLContext ctx)
    throws Exception {
  URL url = uri.toURL();
  URLConnection conn = url.openConnection();
  if (conn instanceof HttpsURLConnection && ctx != null) {
    ((HttpsURLConnection)conn).setSSLSocketFactory(
        ctx.getSocketFactory());
  }
  InputStream is = conn.getInputStream();
  int bytesRead, bufsz = Math.max(is.available(), 4096);
  ByteArrayOutputStream os = new ByteArrayOutputStream(bufsz);
  byte[] buffer = new byte[bufsz];
  while ((bytesRead = is.read(buffer)) > 0)
    os.write(buffer, 0, bytesRead);
  byte[] content = os.toByteArray();
  os.close(); is.close();
  return content;
}

This client opens a URLConnection, reads the input stream into a byte buffer, and then closes the connection. If the connection is https – that is, an instance of HttpsURLConnection – it applies the SocketFactory from the supplied SSLContext.

NOTE: I’m purposely ignoring exception managment in this article to keep it short.

This code is simple and concise, but clearly there’s no way to affect what happens during application of the SSL certificates and keys at this level of the code. Certificate and key management is handled by the SSLContext so if we want to modify the behavior of the SocketFactory relative to key management, we’re going to have to do something with SSLContext before we pass it to the client. The simplest way to get an SSLContext is to call SSLContext.getDefault in this manner:

byte[] bytes = getContentBytes(
    URI.create("https://www.example.com/"), 
    SSLContext.getDefault());

The default SSLContext is fairly limited in functionality. It uses either default key and trust store files (and passwords!) or else ones specified in system properties – often via the java command line in this manner:

$ java -Djavax.net.ssl.keyStore=/path/to/keystore.jks \
 -Djavax.net.ssl.keyStorePassword=changeit \
 -Djavax.net.ssl.trustStorePath=/path/to/truststore.jks \
 -Djavax.net.ssl.trustStorePassword=changeit ...

In reality, there is no default keystore, which is fine for normal situations, as most websites don’t require X.509 client authentication (more commonly referred to as mutual auth). The default trust store is $JAVA_HOME/jre/lib/security/cacerts, and the default trust store password is changeit. The cacerts file contains several dozen certificate authority (CA) root certificates and will validate any server whose public key certificate is signed by one of these CAs.

More importantly, however, the default SSLContext simply fails to connect to a server in the event that a trust certificate is missing from the default trust store. But that’s not what web browsers do. Instead, they display the aforementioned dialog presenting the user with options to handle the situation in the manner that suits him or her best.

Assume the simple client above is a part of a larger application that adds certificates to the trust store during execution of other code paths and then expects to be able to use this updated trust store later during the same session. This dynamic reload functionality requires some SSLContext customization.

Let’s explore. SSLContext is a great example of a composite design. It’s built from several other classes, each of which may be specified by the user when initializing a context object. This practically eliminates the need to sub-class SSLContext in order to define custom behavior. The default context is eschewed in favor of a user-initialized instance of SSLContext like this:

public SSLContext getSSLContext(String tspath) 
    throws Exception {
  TrustManager[] trustManagers = new TrustManager[] { 
    new ReloadableX509TrustManager(tspath) 
  };
  SSLContext sslContext = SSLContext.getInstance("SSL");
  sslContext.init(null, trustManagers, null);
  return sslContext;
}

At the heart of this method is the instantiation of a new ReloadableX509TrustManager. The init method of SSLContext accepts a reference to an array of TrustManager objects. Passing null tells the context to use the default trust manager array which exihibits the default behavior mentioned above.

The init method also accepts two other parameters, to which I’ve passed null. The first parameter is a KeyManager array and the third is an implementation of SecureRandom. Passing null for any of these three parameters tells SSLContext to use the default. Here’s one implementation of ReloadableX509TrustManager:

class ReloadableX509TrustManager 
    implements X509TrustManager {
  private final String trustStorePath;
  private X509TrustManager trustManager;
  private List tempCertList 
      = new List();

  public ReloadableX509TrustManager(String tspath)
      throws Exception {
    this.trustStorePath = tspath;
    reloadTrustManager();
  }

  @Override
  public void checkClientTrusted(X509Certificate[] chain, 
      String authType) throws CertificateException {
    trustManager.checkClientTrusted(chain, authType);
  }

  @Override
  public void checkServerTrusted(X509Certificate[] chain, 
      String authType) throws CertificateException {
    try {
      trustManager.checkServerTrusted(chain, authType);
    } catch (CertificateException cx) {
      addServerCertAndReload(chain[0], true);
      trustManager.checkServerTrusted(chain, authType);
    }
  }

  @Override
  public X509Certificate[] getAcceptedIssuers() {
    X509Certificate[] issuers 
        = trustManager.getAcceptedIssuers();
    return issuers;
  }

  private void reloadTrustManager() throws Exception {

    // load keystore from specified cert store (or default)
    KeyStore ts = KeyStore.getInstance(
	    KeyStore.getDefaultType());
    InputStream in = new FileInputStream(trustStorePath);
    try { ts.load(in, null); }
    finally { in.close(); }

    // add all temporary certs to KeyStore (ts)
    for (Certificate cert : tempCertList) {
      ts.setCertificateEntry(UUID.randomUUID(), cert);
    }

    // initialize a new TMF with the ts we just loaded
    TrustManagerFactory tmf 
	    = TrustManagerFactory.getInstance(
            TrustManagerFactory.getDefaultAlgorithm());
    tmf.init(ts);

    // acquire X509 trust manager from factory
    TrustManager tms[] = tmf.getTrustManagers();
    for (int i = 0; i < tms.length; i++) {
      if (tms[i] instanceof X509TrustManager) {
        trustManager = (X509TrustManager)tms[i];
        return;
      }
    }

    throw new NoSuchAlgorithmException(
        "No X509TrustManager in TrustManagerFactory");
  }

  private void addServerCertAndReload(Certificate cert, 
      boolean permanent) {
    try {
      if (permanent) {
        // import the cert into file trust store
        // Google "java keytool source" or just ...
        Runtime.getRuntime().exec("keytool -importcert ...");
      } else {
        tempCertList.add(cert);
      }
      reloadTrustManager();
    } catch (Exception ex) { /* ... */ }
  }
}

NOTE: Trust stores often have passwords but for validation of credentials the password is not needed because public key certificates are publicly accessible in any key or trust store. If you supply a password, the KeyStore.load method will use it when loading the store but only to validate the integrity of non-public information during the load – never during actual use of public key certificates in the store. Thus, you may always pass null in the second argument to KeyStore.load. If you do so, only public information will be loaded from the store.

A full implementation of X509TrustManager is difficult and only sparsely documented but, thankfully, not necessary. What makes this implementation simple is that it delegates to the default trust manager. There are two key bits of functionality in this implementation: The first is that it loads a named trust store other than cacerts. If you want to use the default trust store, simply assign $JAVA_HOME/jre/lib/security/cacerts to trustStorePath.

The second bit of functionality is the call to addServerCertAndReload during the exception handler in the checkServerTrusted method. When a certificate presented by a server is not found in the trust manager’s in-memory database, ReloadableX509TrustManager assumes that the trust store has been updated on disk, reloads it, and then redelegates to the internal trust manager.

A more functional implementation might display a dialog box to the user before calling addServerCertAndReload. If the user selects Get me out of here!, the method would simply rethrow the exception instead of calling that routine. If the user selects It’s cool: add permanently, the method would add the certificate to the file-based trust store, reload from disk, and then reissue the delegated request. If the user selects I’ll bite: add temporarily, the certificate would be added to a list of temporary certificates in memory.

The way I’ve implemented the latter case is to add the certificate to a temporary list and then reload from disk. Strictly speaking, reloading from disk isn’t necessary in this case since no changes were made to the disk file but the KeyStore built from the disk image would have to be kept around for reloading into the trust manager (after the new cert was added to it), so some modifications would have to be made to avoid reloading from disk.

This same code might as well be used in a server-side setting but the checkClientTrusted method would have to be modified instead of the checkServerTrusted method as in this example.

Advertisements

Java Https Key Setup

In my last article, I showed how to remove all security from a secure web (https) transaction by installing dummy trust manager and host name verifier objects into an SSLSocketFactory. Today, I’m going to take it to the next level by demonstrating how to create a private key and self-signed certificate in a JKS keystore, exporting the public key certificate to a client-side trust store, and configuring our client to use the trust store to verify our server.

I’ll be using a Tomcat 6 server – mainly because it’s almost trivial to install and configure for SSL traffic. On my OpenSuSE 11.1 64-bit GNU/Linux machine, I’ve installed the tomcat6 package, and then I’ve gone into YaST’s service management panel and enabled the tomcat6 service.

Self-Signed Certificates

Let’s start by generating the proper keys. First, we’ll generate the server’s self-signed certificate, with embedded public/private key pair. For the common name (CN) field, I’ll make sure to enter the fully qualified domain name of my server (jmc-linux-64.provo.novell.com). This will ensure that my Java client code will properly compare the hostname used in my URL with the server’s certificate. Using any other value here would cause my client to fail with an invalid hostname exception. Here’s the Java keytool command line to create a self-signed certificate in a JKS key store called jmc-linux-64.keystore.jks:

$ keytool -genkey -alias jmc-linux-64 \
 -keyalg RSA -keystore jmc-linux-64.keystore.jks
Enter keystore password: password
Re-enter new password: password
What is your first and last name?
  [Unknown]:  jmc-linux-64.provo.novell.com
What is the name of your organizational unit?
  [Unknown]:  Engineering
What is the name of your organization?
  [Unknown]:  Novell, Inc.
What is the name of your City or Locality?
  [Unknown]:  Provo
What is the name of your State or Province?
  [Unknown]:  Utah
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=jmc-linux-64.provo.novell.com, OU=Engineering,
 O="Novell, Inc.", L=Provo, ST=Utah, C=US correct?
  [no]:  yes

Enter key password for 
         (RETURN if same as keystore password): <CR>
		
$

To view the new certificate and key pair, just use the -list option, along with the -v (verbose) option, like this:

$ keytool -list -v -keystore jmc-linux-64.keystore.jks
Enter keystore password: password

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: jmc-linux-64
Creation date: Jun 19, 2009
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Issuer: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Serial number: 4a3c006f
Valid from: Fri Jun 19 15:17:35 MDT 2009 until: Thu Sep 17 15:17:35 MDT 2009
Certificate fingerprints:
         MD5:  E5:37:9F:85:C9:76:60:FC:DC:01:81:AD:5F:FC:F4:9A
         SHA1: FD:E3:47:6C:AE:9B:75:3B:9C:6C:05:7B:C9:A4:B4:E6:07:F6:B5:FB
         Signature algorithm name: SHA1withRSA
         Version: 3


*******************************************
*******************************************

$

Server Configuration

Okay, now we have a server certificate with public and private key pair in a JKS keystore. The next step is to configure Tomcat to listen for https requests. The default configuration for Tomcat is to run a bare http server on port 8080. To enable the https server on port 8443, I edited the /usr/share/tomcat6/conf/server.xml file and uncommented the default entry for SSL that was already in place as a comment:

...
<!-- Define a SSL HTTP/1.1 Connector on port 8443
     This connector uses the JSSE configuration, when using APR, the
     connector should be using the OpenSSL style configuration
     described in the APR documentation -->

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
           maxThreads="150" scheme="https" secure="true"
           keystoreFile="/jmc-linux-64.keystore.jks" 
           keystorePass="password"
           clientAuth="false" sslProtocol="TLS" />
...

Make sure the sslProtocol is set to at least “SSLv3” – I just used “TLS” here. The important fields, however, are the keystoreFile and keystorePass fields, which I’ve set to the keystore we created in the previous step, and its password. You can put the keystore file anywhere on your file system accessible by the user running the tomcat service. On my system, the tomcat6 service is executed as root by default, so I just copied my keystore to the root of my file system.

After editing the file, I had to restart the tomcat6 service:

# rctomcat6 restart
Shutting down Tomcat (/usr/share/tomcat6)	... done
Starting Tomcat (/usr/share/tomcat6)		... done
#

Client-Side Trust Store

So much for server configuration. Now we have to configure the client’s trust store with the server’s self-signed certificate. This is done by exporting the certificate and public key from the server’s keystore, and then importing it into a client trust store. A trust store is just a JKS keystore that contains only trust certificates:

$ keytool -export -alias jmc-linux-64 \
 -keystore jmc-linux-64.keystore.jks -rfc \
 -file jmc-linux-64.cert
Enter keystore password: password
Certificate stored in file 
$
$ cat jmc-linux-64.cert
-----BEGIN CERTIFICATE-----
MIICezCCAeSgAwIBAgIESjwAbzANBgkqhkiG9w0BAQUFADCBgTELMAkGA1UEBhMCVVMxDTALBgNV
BAgTBFV0YWgxDjAMBgNVBAcTBVByb3ZvMRUwEwYDVQQKEwxOb3ZlbGwsIEluYy4xFDASBgNVBAsT
C0VuZ2luZWVyaW5nMSYwJAYDVQQDEx1qbWMtbGludXgtNjQucHJvdm8ubm92ZWxsLmNvbTAeFw0w
OTA2MTkyMTE3MzVaFw0wOTA5MTcyMTE3MzVaMIGBMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRh
aDEOMAwGA1UEBxMFUHJvdm8xFTATBgNVBAoTDE5vdmVsbCwgSW5jLjEUMBIGA1UECxMLRW5naW5l
ZXJpbmcxJjAkBgNVBAMTHWptYy1saW51eC02NC5wcm92by5ub3ZlbGwuY29tMIGfMA0GCSqGSIb3
DQEBAQUAA4GNADCBiQKBgQCOwb5migz+c1mmZS5eEhBQ5wsYFuSmp6bAL7LlHARQxhZg62FEVBFL
Y2klPoCGfUoXUFegnhCV5I37M0dAQtNLSHiEPj0NjAvWuzagevE6Tq+0zXEBw9fKoVV/ypEsAxEX
6JQ+a1WU2W/vdL+x0lEbRpRCk9t6yhxLw16M/VD/GwIDAQABMA0GCSqGSIb3DQEBBQUAA4GBAC5E
kniYYFxwZUqg9ToFlF0LKjGZfttkXJoTMfOFwA6OXrO6cKdzS04srxhoDzkD8V4RskPxttt0pbKr
iAoGKT/9P4hpDb0Ej4urek9TxlrnoC8g0rOYaDfE57SMStDrCg2ha4IuJFtJOh1aMcl4pm/sk+JW
7U/cWyW9B7InJinZ
-----END CERTIFICATE-----

$
$ keytool -import -alias jmc-linux-64 \
 -file jmc-linux-64.cert \
 -keystore jmc-linux-64.truststore.jks
Enter keystore password: trustpass
Re-enter new password: trustpass
Owner: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Issuer: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Serial number: 4a3c006f
Valid from: Fri Jun 19 15:17:35 MDT 2009 until: Thu Sep 17 15:17:35 MDT 2009
Certificate fingerprints:
         MD5:  E5:37:9F:85:C9:76:60:FC:DC:01:81:AD:5F:FC:F4:9A
         SHA1: FD:E3:47:6C:AE:9B:75:3B:9C:6C:05:7B:C9:A4:B4:E6:07:F6:B5:FB
         Signature algorithm name: SHA1withRSA
         Version: 3
Trust this certificate? [no]:  yes
Certificate was added to keystore

$

We now have a file called jmc-linux-64.truststore.jks, which contains only the server’s public key and certificate. You can show the contents of the truststore JKS file with the -list option, like this:

$ keytool -list -v -keystore jmc-linux-64.truststore.jks
Enter keystore password: trustpass

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: jmc-linux-64
Creation date: Jun 19, 2009
Entry type: trustedCertEntry

Owner: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Issuer: CN=jmc-linux-64.provo.novell.com, OU=Engineering, O="Novell, Inc.", L=Provo, ST=Utah, C=US
Serial number: 4a3c006f
Valid from: Fri Jun 19 15:17:35 MDT 2009 until: Thu Sep 17 15:17:35 MDT 2009
Certificate fingerprints:
         MD5:  E5:37:9F:85:C9:76:60:FC:DC:01:81:AD:5F:FC:F4:9A
         SHA1: FD:E3:47:6C:AE:9B:75:3B:9C:6C:05:7B:C9:A4:B4:E6:07:F6:B5:FB
         Signature algorithm name: SHA1withRSA
         Version: 3


*******************************************
*******************************************

$

A Simple Https Client

We have several options for how to consume this trust store in client code. I’ll take the easy route today, but watch for another article that describes more complex mechanisms that provide more flexibility. Today, I’ll just show you how to set system properties on our client application. This client is very simple. All it does is connect to the server and display the contents of the web page in raw html to the console:

import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

public class HttpsClient
{
  private final String serverUrl;

  public HttpsClient(String serverUrl) 
  {
    this.serverUrl = serverUrl;
  }

  public void connect() 
  {
    try
    {
      HttpURLConnection conn = null;
      URL url = new URL(serverUrl);

      try
      {
        conn = (HttpURLConnection)url.openConnection();
        conn.setRequestMethod("GET");
        conn.setDoOutput(false);
        conn.setDoInput(true);
        conn.connect();
        InputStream is = conn.getInputStream();

        Integer bytes;
        byte [] buffer = new byte[512];
        while ((bytes = is.read(buffer, 0, 512)) > 0)
          System.out.write(buffer, 0, bytes);
      }
      catch (IOException e) { e.printStackTrace(); }
    }
    catch(MalformedURLException e) { e.printStackTrace(); }
  }

  public static void main(String[] args) 
  {
    HttpsClient client = new HttpsClient(
        "https://jmc-linux-64.provo.novell.com:8443");
    client.connect();
  }
}

Executing this client as is, without an assigned trust store will cause it to use the default trust store ($JAVA_HOME/lib/security/cacerts), which doesn’t contain our server’s public certificate, so it will fail with an exception:

javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: 
PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
... stack trace ...
Caused by: 
sun.security.validator.ValidatorException: 
PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
... stack trace ...
Caused by: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
... stack trace ...  

Configuring the Client Trust Store

The quick way to get this client to work properly is to assign our client’s trust store (containing the server’s public key and self-signed certificate) to JSSE system properties in this manner:

$ java -Djavax.net.ssl.trustStore=jmc-linux-64.truststore.jks \
  -Djavax.net.ssl.trustStorePassword=trustword

If you get the path to the trust store file wrong, you’ll get a different cryptic exception:

javax.net.ssl.SSLException: 
java.lang.RuntimeException: Unexpected error: 
java.security.InvalidAlgorithmParameterException: 
the trustAnchors parameter must be non-empty
... stack trace ...
Caused by: java.lang.RuntimeException: Unexpected error: 
java.security.InvalidAlgorithmParameterException: 
the trustAnchors parameter must be non-empty
... stack trace ...
Caused by: 
java.security.InvalidAlgorithmParameterException: 
the trustAnchors parameter must be non-empty
... stack trace ...

And if you get the password wrong, you’ll get yet another (somewhat less) cryptic exception:

java.net.SocketException: 
java.security.NoSuchAlgorithmException: 
Error constructing implementation 
(algorithm: Default, provider: SunJSSE, 
class: com.sun.net.ssl.internal.ssl.DefaultSSLContextImpl)
... stack trace ...
Caused by: java.security.NoSuchAlgorithmException: 
Error constructing implementation 
(algorithm: Default, provider: SunJSSE, 
class: com.sun.net.ssl.internal.ssl.DefaultSSLContextImpl)
... stack trace ...
Caused by: java.io.IOException: 
Keystore was tampered with, or password was incorrect
... stack trace ...
Caused by: java.security.UnrecoverableKeyException: 
Password verification failed
... stack trace ...

In these examples, my client is using my server’s fully qualified domain name in the URL, which is the common name we used when we created the self-signed certificate:

  ...
  public static void main(String[] args) 
  {
    HttpsClient client = new HttpsClient(
        "https://jmc-linux-64.provo.novell.com:8443");
    client.connect();
  }
}

This is the only name that will work with this trust store. In my next article I’ll show you how to generate certificates that work with aliases like the IP address. I’ll also show you how to add a hostname verifier to allow our client code to be a bit more intelligent about which aliases it rejects out of hand.

Java Secure HTTP Client Key Management

My current project at Novell involves the development of a ReSTful web service for submission of audit records from security applications. The server is a Jersey servlet within an embedded Tomcat 6 container.

One of the primary reasons for using a ReSTful web service for this purpose is to alleviate the need to design and build a heavy-weight audit record submission client library. Such client libraries need to be orthogonally portable across both hardware platforms and languages in order to be useful to Novell’s customers. Just maintaining the portability of this client library in one language is difficult enough, without adding multiple languages to the matrix.

Regardless of our motivation, we still felt the need to provide a quality reference implementation of a typical audit client library to our customers. They may incorporate as much or as little of this code as they wish, but a good reference implementation is worth a thousand pages of documentation. (Don’t get me wrong, however–this is no excuse for not writing good documentation! The combination of quality concise documentation and a good reference implementation is really the best solution.)

The idea here is simple: Our customers won’t have to deal with difficulties that we stumble upon and then subsequently provide solutions for. Additionally, it’s just plain foolish to provide a server component for which you’ve never written a client. It’s like publishing a library API that you’ve never written to. You don’t know if the API will even work the way you originally intended until you’ve at least tried it out.

Since we’re already using Java in the server, we’ve decided that our initial client reference implementation should also be written in Java. Yesterday found my code throwing one exception after another while simply trying to establish the TLS connection to the server from the client. All of these problems ultimately came down to my lack of understanding of the Java key store and trust store concepts.

You see, the establishment of a TLS connection from within a Java client application depends heavily on the proper configuration of a client-side trust store. If you’re using mutual authentication, as we are, then you also need to properly configure a client-side key store for the client’s private key. The level at which we are consuming Java network interfaces also demands that we specify these stores in system properties. More on this later…

Using Curl as an Https Client

We based our initial assumptions about how the Java client needed to be configured on our use of the curl command line utility in order to test the web service. The curl command line looks something like this:

  curl -k --cert client.cer --cert-type DER --key client-key.pem
    --key-type PEM --header "Content-Type: application/audit+json"
    -X POST --data @test-event.json https://10.0.0.1:9015/audit/log/test

The important aspects of this command-line include the use of the –cert, –cert-type, –key and –key-type parameters, as well as the fact that we specified a protocol scheme of “https” in the URL.

With one exception, the remaining options are related to which http method to use (-X), what data to send (–data), and which message properties to send (–header). The exception is the -k option, and therein lay most of our problems with this Java client.

The curl man-page indicates that the -k/–insecure option allows the TLS handshake to succeed without verifying the server certificate in the client’s CA (Certificate Authority) trust store. The reason this option was added was because several releases of the curl package shipped with a terribly out-dated trust store, and people were getting tired of having to manually add certificates to their trust stores everytime they hit a newer site.

Doing it in Java

But this really isn’t the safe way to access any secure public web service. Without server certificate verification, your client can’t really know that it’s not communicating with a server that just says it’s the right server. (“Trust me!”)

During the TLS handshake, the server’s certificate is passed to the client. The client should then verify the subject name of the certificate. But verify it against what? Well, let’s consider–what information does the client have access to, outside of the certificate itself? It has the fully qualified URL that it used to contact the server, which usually contains the DNS host name. And indeed, a client is supposed to compare the CN (Common Name) portion of the subject DN (Distinguished Name) in the server certificate to the DNS host name in the URL, according to section 3.1 “Server Identity” of RFC 2818 “HTTP over TLS”.

Java’s HttpsURLConnection class strictly enforces the advice given in RFC 2818 regarding peer verification. You can override these constraints, but you have to basically write your own version of HttpsURLConnection, or sub-class it and override the methods that verify peer identity.

Creating Java Key and Trust Stores

Before even attempting a client connection to our server, we had to create three key stores:

  1. A server key store.
  2. A client key store.
  3. A client trust store.

The server key store contains the server’s self-signed certificate and private key. This store is used by the server to sign messages and to return credentials to the client.

The client key store contains the client’s self-signed certificate and private key. This store is used by the client for the same purpose–to send client credentials to the server during the TLS mutual authentication handshake. It’s also used to sign client-side messages for the server during the TLS handshake. (Note that once authentication is established, encryption happens using a secret or symetric key encryption algorithm, rather than public/private or asymetric key encryption. Symetric key encryption is a LOT faster.)

The client trust store contains the server’s self-signed certificate. Client-side trust stores normally contain a set of CA root certificates. These root certificates come from various widely-known certificate vendors, such as Entrust and Verisign. Presumably, almost all publicly visible servers have a purchased certificate from one of these CA’s. Thus, when your web browser connects to such a public server over a secure HTTP connection, the server’s certificate can be verified as having come from one of these well-known certificate vendors.

I first generated my server key store, but this keystore contains the server’s private key also. I didn’t want the private key in my client’s trust store, so I extracted the certificate into a stand-alone certificate file. Then I imported that server certificate into a trust store. Finally, I generated the client key store:

  $ keytool -genkey -alias server -keyalg RSA \
  > -storepass changeit -keystore server-keystore.jks
  What is your first and last name?
    [Unknown]:  audit-server
  What is the name of your organizational unit?
    [Unknown]:  Eng
  What is the name of your organization?
    [Unknown]:  Novell
  What is the name of your City or Locality?
    [Unknown]:  Provo
  What is the name of your State or Province?
    [Unknown]:  Utah
  What is the two-letter country code for this unit?
    [Unknown]:  US
  Is CN=audit-server, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US correct?
    [no]:  yes

  Enter key password for <server>
          (RETURN if same as keystore password):  
  $
  $ keytool -exportcert -keystore server-keystore.jks \
  > -file server.der -alias server -storepass changeit
  Certificate stored in file <server.der>
  $
  $ keytool -importcert -trustcacerts -alias server \
  > -keystore server-truststore.jks -storepass changeit \
  > -file server.der
  Owner: CN=audit-server, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Issuer: CN=audit-server, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Serial number: 491cad67
  Valid from: Thu Nov 13 15:42:47 MST 2008 until: Wed Feb 11 15:42:47 MST 2009
  Certificate fingerprints:
           MD5:  EE:FA:EE:78:A8:42:2B:F2:3A:04:50:37:D3:94:B3:C0
           SHA1: 4E:BA:9B:2F:FC:84:10:5A:2E:62:D2:5B:B3:70:70:B5:2F:03:E1:CD
	   Signature algorithm name: SHA1withRSA
           Version: 3
  Trust this certificate? [no]:  yes
  Certificate was added to keystore
  $
  $ keytool -genkey -alias client -keyalg RSA -storepass changeit \
  > -keystore client-keystore.jks
  What is your first and last name?
    [Unknown]:  audit-client
  What is the name of your organizational unit?
    [Unknown]:  Eng
  What is the name of your organization?
    [Unknown]:  Novell
  What is the name of your City or Locality?
    [Unknown]:  Provo
  What is the name of your State or Province?
    [Unknown]:  Utah
  What is the two-letter country code for this unit?
    [Unknown]:  US
  Is CN=audit-client, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US correct?
    [no]:  yes

  Enter key password for <client>
          (RETURN if same as keystore password):  
  $
  $ ls -1
  client-keystore.jks
  server.der
  server-keystore.jks
  server-truststore.jks
  $

Telling the Client About Keys

There are various ways of telling the client about its key and trust stores. One method involves setting system properties on the command line. This is commonly used because it avoids the need to enter absolute paths directly into the source code, or to manage separate configuration files.

  $ java -Djavax.net.ssl.keyStore=/tmp/keystore.jks ...

Another method is to set the same system properties inside the code itself, like this:

  public class AuditRestClient
  {
    public AuditRestClient() 
    {
      System.setProperty("javax.net.ssl.keyStore", 
          "/tmp/keystore.jks");
      System.setProperty("javax.net.ssl.keyStorePassword", 
          "changeit");
      System.setProperty("javax.net.ssl.trustStore", 
          "/tmp/truststore.jks");
      System.setProperty("javax.net.ssl.trustStorePassword", 
          "changeit");
    }
    ...

I chose the latter, as I’ll eventually extract the strings into property files loaded as needed by the client code. I don’t really care for the fact that Java makes me specify these stores in system properties. This is especially a problem for our embedded client code, because our customers may have other uses for these system properties in the applications in which they will embed our code. Here’s the rest of the simple client code:

    ...
    public void send(JSONObject event) 
    {
      byte[] bytes = event.toString().getBytes();
      HttpURLConnection conn = null; 
		
      try
      {
        // establish connection parameters
        URL url = new URL("https://10.0.0.1:9015/audit/log/test");
        conn = (HttpURLConnection)url.openConnection();
        conn.setRequestMethod("POST");
        conn.addRequestProperty("Content-Length", "" + bytes.length);
        conn.addRequestProperty("Content-Type", "application/audit1+json");
        conn.setDoOutput(true);
        conn.setDoInput(true);
        conn.connect();

        // send POST data
        OutputStream out = (OutputStream)conn.getOutputStream();
        out.write(bytes);
        out.flush();
        out.close(); 		

        // get response code and data
        System.out.println(conn.getResponseCode());
        BufferedReader read = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        String query = null;
        while((query = read.readLine()) != null)
          System.out.println(query);
      }
      catch(MalformedURLException e) { e.printStackTrace(); }
      catch(ProtocolException e) { e.printStackTrace(); }
      catch(IOException e) { e.printStackTrace(); }
      finally { conn.disconnect(); }
    }
  }

Getting it Wrong…

I also have a static test “main” function so I can send some content. But when I tried to execute this test, I got an exception indicating that the server certificate didn’t match the host name. I was using a hard-coded IP address (10.0.0.1), but my certificate contained the name “audit-server”.

It turns out that the HttpsURLConnection class uses an algorithm to determine if the server that sent the certificate really belongs to the server on the other end of the connection. If the URL contains an IP address, then it attempts to locate a matching IP address in the “Alternate Names” portion of the server certificate.

Did you notice a keytool prompt to enter alternate names when you generated your server certificate? I didn’t–and it turns out there isn’t one. The Java keytool utility doesn’t provide a way to enter alternate names–a standardized extension of the X509 certificate format. To enter an alternate name containing the requisite IP address, you’d have to generate your certificate using the openssl utility, or some other more functional certificate generation tool, and then find a way to import these foreign certificates into a Java key store.

…And then Doing it Right

On the other hand, if the URL contains a DNS name, then HttpsURLConnection attempts to match the CN portion of the Subject DN with the DNS name. This means that your server certificates have to contain the DNS name of the server as the CN portion of the subject. Returning to keytool, I regenerated my server certificate and stores using the following commands:

  $ keytool -genkey -alias server -keyalg RSA \
  > -storepass changeit -keystore server-keystore.jks
  What is your first and last name?
    [Unknown]:  jmc-test.provo.novell.com

  ... (the rest is the same) ...
  
  $ keytool -exportcert -keystore server-keystore.jks \
  > -file server.der -alias server -storepass changeit
  Certificate stored in file <server.der>
  $
  $ keytool -importcert -trustcacerts -alias server \
  > -keystore server-truststore.jks -storepass changeit \
  > -file server.der
  Owner: CN=jmc-test.provo.novell.com, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Issuer: CN=jmc-test.provo.novell.com, OU=Eng, O=Novell, L=Provo, ST=Utah, C=US
  Serial number: 491cad67
  Valid from: Thu Nov 13 15:42:47 MST 2008 until: Wed Feb 11 15:42:47 MST 2009
  Certificate fingerprints:
           MD5:  EE:FA:EE:78:A8:42:2B:F2:3A:04:50:37:D3:94:B3:C0
           SHA1: 4E:BA:9B:2F:FC:84:10:5A:2E:62:D2:5B:B3:70:70:B5:2F:03:E1:CD
	   Signature algorithm name: SHA1withRSA
           Version: 3
  Trust this certificate? [no]:  yes
  Certificate was added to keystore
  $

Of course, I also had to change the way I was specifying my URL in the client code:

  ...
  URL url = new URL("https://jmc-test.provo.novell.com:9015/audit/log/test");
  conn = (HttpURLConnection)url.openConnection();
  ...

At this point, I was finally able to connect to my server and send the message. Is this reasonable? Probably not for my case. Both my client and server are within a corporate firewall, and controlled by the same IT staff, so to force this sort of gyration is really unreasonable. Can we do anything about it? Well, one thing that you can do is to provide a custom host name verifier like this:

  ...
  URL url = new URL("https://jmc-sentinel.dnsdhcp.provo.novell.com:9015/audit/AuditLog/test");
  conn = (HttpsURLConnection)url.openConnection();
  conn.setHostnameVerifier(new HostnameVerifier()
  { 
    public boolean verify(String hostname, SSLSession session) 
        { return true; }
  });
  conn.setRequestMethod("POST");
  ...

When you do this, however, you should be aware that you give up the right to treat the connection as anything but an https connection. Note that we had to change the type of “conn” to HttpsURLConnection from its original type of HttpURLConnection. This means, sadly, that this code will now only work with secure http connections. I chose to use the DNS name in my URL, although a perfectly viable option would also have been the creation of a certificate containing the IP address as an “Alternate Name”.

Is This Okay?!

Ultimately, our client code will probably be embedded in some fairly robust and feature-rich security applications. Given this fact, we can’t really expect our customers to be okay with our sample code taking over the system properties for key and trust store management. No, we’ll have to rework this code to do the same sort of thing that Tomcat does–manage our own lower-level SSL connections, and thereby import certificates and CA chains ourselves. In a future article, I’ll show you how to do this.