Sunday, March 22, 2009

Installing SSL (HTTPS) Certificate Into Java Keystore Without The Certificate File

I stumbled into a problem with Java and SSL. Our Archiva server serves Maven2 repository through SSL. I could browse it without issues using Firefox, after accepting the SSL security certificate. But Maven2 failed to download artifacts (and it didn't even raise a warning). Turned out that Java did not have the SSL certificate installed. I did not have the SSL certificate file, so how do I install it?

Turns out it's very simple. Here's how. Open the site in Firefox using an https link. You might be prompted to resolve the security certificate issue. If so, do that first. After you successfully load the page, click on the left tip of the address bar. You should see something like this:



Click "More Information...". Then click "View Certificate", open "Details" tab and click "Export...". Select "X.509 Certificate with chain (PEM)" and save the file. This file will be compatible with Java's keytool command. You can use a variant of the following command to install the certificate:

keytool -keystore "%JAVA_HOME%\jre\lib\security\cacerts" -import -file [PATH_TO_FILE]

Friday, March 13, 2009

Java Checked Exceptions Revisited

As promised this is a follow up to the discussion of Java's checked exceptions. I consider this topic important enough to visit it again. I will start by summarizing different aspects of checked exceptions, including my own ideas as well as ideas from external sources. Then I will suggest a couple of solutions to the problem.

Facts about checked exceptions:

  • A method that contains a statement that may throw a checked exception must either handle that exception in a try/catch block or declare the exception in the method's signature using the throws clause

  • Checked exceptions are only checked at compile time (see JLS). This fact is actually not widely known. If for example, you compile your application against a library which uses runtime exceptions, your application will still run if, after compilation, you replace the library with another version of the same library in which runtime exceptions were refactored into checked exceptions. This is because JVM does not perform the check when loading and running classes.


Pros

  • Checked exceptions automatically remind of themselves and have smaller chance to go unhandled. Consequently, if developers follow the best practice to make recoverable errors checked and unrecoverable errors unchecked, then checked exceptions will encourage recovery rather than application failure.

  • As James Gosling explains here, "the knowledge of the situation is always fairly localized". I would replace the word "always" with "sometimes", but the point stands. If the knowledge of a situation is localized then handling the exception closer to the point of occurence is likely the best approach. And again, with a checked exception the compiler will remind you of that.


Cons

  • Bruce Eckel gives an excellent overview of psychological phenomena observed in many Java developers when it comes to checked exceptions. Checked exceptions get swallowed, making it hard to track down bugs. Declaration of "throws Exception" or even worse, "throws Throwable", is not uncommon, and it defeats the purpose of checked exceptions altogether.

  • Checked exceptions reduce encapsulation (a more detailed example will follow). Implementations of an interface are not allowed to throw exceptions other than those declared by the interface. This is good. The bad part is that checked exceptions that your code cannot recover from locally is forced to handle the exception, at least you have to wrap it into a runtime exception. Wrapping exceptions means you can no longer use standard exception hierarchy.

  • Apparently Sun found a situation when checked exceptions should be bypassed (they tricked the compiler). Why should we not have the same option?

  • Versionability and scalability - checked exceptions effectively add themselves into the method's signature with API-breaking consequences to the client code. Additionally, as the project grows and the number of used libraries grows the number of possible exceptions grows with it. With checked exceptions this means that methods up the stack have two choices: 1) declare the grand-parent of all thrown exceptions in the throws clause and 2) list them all. Declaring the grand-parent exception is not desirable or not possible as it usually turns out to be either java.land.Exception or java.lang.Throwable. Listing all possible exceptions has a scalability issue in that a) it increases code bloat with long throws clauses proliferating throughout the code base and b) every time you add a new checked exception into the mix, you have to update all methods up the stack

  • Code Testability - quite often there are cases when a checked exception will never happen. This makes the catch block untestable. See details in the post.


Locality and Encapsulation
When James Gosling talks about localized knowledge of exceptions I believe what he means is that exceptions are local in the sense that they do not (or should not) travel very high up the call stack. For example, if method1 calls method2, method2 calls method3, ..., method9 calls method10 and method10 throws an exception, then James would rather handle that exception in method10 or method9, but wouldn't let it go too far up the stack, say, up to method2. Now, exceptions were specifically created to fly up the stack until someone cares to handle them. Where exactly that happens depends on the design of a given program. There is a perfectly solid program design according to which exception handling happens very high up the stack. This design stems from the practice of using exceptions as special, exceptional, out-of-the-ordinary conditions.

Let's go straight to Wikipedia and read the definition:

Exception handling is a programming language construct or computer hardware mechanism designed to handle the occurrence of exceptions - special conditions that change the normal flow of execution.


Now, what is the value of classifying a particular condition as "special" or being ouside of "the normal flow of execution" if it is part of the method signature like all the "normal" conditions and has the same consequences to the API and the client code? I believe the idea of exceptions is to give the programmer the ability to design their logic as if no exceptions ever occur in the flow, but also to create a "special" mechanism (as special as the exceptions themselves) to deal with exceptional situations outside the normal flow. Forcing the client code to handle an exception is forcing changes to the normal flow of execution. Does not this immediately destroy the purpose exceptions? The exception becomes more like a return status code rather than something that represents an exceptional situation.

In fact locality is not something that the designer of the exception class can decide. Java interfaces, dependency injection and AOP allow us to design programs with very high level of separation of concerns. For example, it is an accepted practice to start a database transaction as soon as a client request (think RPC or HTTP) is accepted, then run all the business logic and finally commit or rollback the transaction based on the outcome. This translates to the following call stack sequence:



See how your database code consists of two layers. One - the Transaction Manager - is up the stack, responsible for opening and committing/rolling back transactions. The second layer is down the stack, responsible for data access - Data Access Object (DAO). All the Business Logic - the majority of your code - is in between. Note, that your Business Logic code calls DAO and it is abstracted from the database driver. It does not "know" if your data access is implemented using pure JDBC or Hibernate, whether it is a relational database or document-oriented, like CouchDB. Here we observe the power of separation of concerns. At any moment we may decide to migrate from one type of database to another and we do not have to change a single line in our business logic. Unless! Unless we use checked exceptions. In order to let a checked exception fly through the business logic layer, every method in the business logic has to declare the exception. For example, if the database is accessed using JDBC, then the business layer will declare throws SQLException. Now, even without going any further we see the problem with this. Namely, the separation of concerns is no more, because the business logic is "aware" of the SQL nature of the data access. You now have an extra week of work to migrate from SQL to something else. The separation of concerns would not break if SQLException were runtime.


Now let's see how the designers of Java could deal with the situation.

Solution I - Remove Checked Exceptions from the Language

As I mentioned above checked exceptions are only checked at compile time. This means that when you run your Java program all exceptions are runtime. At the compiler level, removing the check is backwards-compatible change too, as exception checking is a restriction on the Java source code. All previously written Java code will still compile without any checking.

But is there a way to implement an exception mechanism in Java that has the benefits of checked exceptions but without their drawbacks?

Solution II - Annotations

In addition to Solution I introduce a set of standard annotations applicable to exception classes.

@Recoverable - indicates that the program may be able to recover from an exception of this type.

@Documented - indicates that this exception should be documented in the JavaDoc comment.

With these annotations in place the developer could configure their tools to highlight instances when a @Recoverable or @Documented exceptions are not handled or documented. Then James Gosling would configure his tools to fail compilation in this case. I wouldn't.

Solution III - Compiler Option

Just as we can enable/disable assertions at runtime, we could have an option to enable/disable exception checking at compile time. Simple, easy to implement.

Wednesday, March 11, 2009

Java Checked Exceptions and Code Testability

There has been a lot of blogs, articles and discussions about the dangers of Java's checked exceptions (Does Java need Checked Exceptions?, Java's checked exceptions were a mistake). Some of the prominent Java technologies, such as Hibernate and Springframework, explicitly reject checked exceptions and not only use runtime exceptions in their API, but also wrap Java standard checked exceptions to make them runtime (SQLException vs HibernateException). I am not going to restate any of the points outlined in those sources, but I will admit in advance that I am an opponent of checked exceptions. In this post I am going to show another aspect of the problem, one that I haven't seen mentioned on the web yet. Namely, I am going to show how checked exceptions reduce the testability of the code.

Let's start with an example. As you may know, Java I/O system uses checked exceptions, with java.io.IOException being the great grandfather of all. Any I/O system is error prone, so you can get all kinds of conditions. However, there are cases when I/O exceptions are never expected to happen. One of these cases concerns static classpath resources. Static classpath resources could be bundled icons, properties, translations, etc. The important thing here is that a static resource is an inseparable part of the application. It is loaded from the same location from which your application classes are loaded. If a classpath resource is missing or is corrupted, then the application is corrupted and may not run correctly. The effect of a missing resource is the same as having one of your jar or class files missing or corrupted. Now, we know that an attempt to initialize a class that is not in the classpath will result in a NoClassDefFoundError. Errors in Java are unchecked exceptions. This was a good decision. Why? Because if an application class is missing in the classpath then the application has not been properly built or initialized and there is nothing the application can do to reliably recover from such a condition. Consequently you never write unit or integration tests that test for those kind of conditions. The same applies to bundled static classpath resources. However, when loading static resources you have to do I/O manually and handle all the checked exceptions in your application code. And this is where your code testability falls. A code sample is worth a thousand words, so let's dive into the code and demonstrate.

Suppose you have a classpath resource example/AppProperties.properties with the following contents:

greeting=Hello, World!

A class example/AppProperties.java that loads the resource from the class path:

package example;

import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;

public class AppProperties {

public String getGreeting() {
Properties props = new Properties();
try {
InputStream inStream =
AppProperties.class.getResourceAsStream("AppProperties.properties");
props.load(inStream);
inStream.close();
} catch (IOException e) {
// What to do here?
throw new RuntimeException("This should never have happened!", e);
}
return props.getProperty("greeting");
}

}


And a test case for this class example/AppTest.java:

package example;

import junit.framework.Assert;

import org.junit.Test;

public class AppTest {

@Test
public void testAppProperties() {
Assert.assertEquals("Hello, World!",
new AppProperties().getGreeting());
}

}


In my opinion a simple class like AppProperties should have 100% code line coverage by tests. Let's see what we get when running with a code coverage tool (I used Eclemma):



[+] Click to enlarge


What we see here is that the exception did not happen, so we never caught it and the catch block was never executed, causing incomplete line coverage. More over, there is no easy way to mock out a condition that will make it happen, because the resource is always there and the build system will ensure it is there with all the rest of the classes.

And no, I do not want to propagate an IOException by declaring it in the throws clause of the method, because I do not want to expose the clients to the implementation details (could become SQL in the future). Also even if I do, the client code will have to handle the exception and then the client code will not be covered. So I wrapped the exception into a RuntimeException, which is unchecked. I could use a different unchecked exception, but chose not to in order to keep it simple.

Back to the general idea. Sometimes we face a situation when we are forced to handle exceptions that will "never" happen. That is, "never" in the sense that they can only happen due to improper application assembly. They are not expected. Consequently there is no point in writing any recovery code or test cases for those conditions. However, checked exceptions force us to write those try/catch blocks, and those blocks are not testable.

This is not about trying to reach 100% coverage. That question deserves its own discussion. As noted in a very insightful article you should only use code coverage as a "clue" to places in the code that may contain bugs. Some of these clues may be false alarms, i.e. a piece of code is not tested but it does not contain bugs. The problem that comes with checked exceptions is that they produce many more of these false alarms. Suppose you run code coverage tool once a week and then go through results in search for bug-prone areas. Because try/catch blocks similar to the one above will never be covered you will be forced to return to that piece of code once a week, review it and say "Oh, it's just that condition that will never happen" and move on. Of course, it will only take you several seconds to verify it depending on code complexity, but if your code contains hundreds of places like this you may waste hours of your time, every week.

I am thinking about writing a follow-up with a conclusion in which I want to use points made for and against checked exceptions by others and put them on some sort of imaginary scale. I also have a middle-ground solution to the problem. But this is next time.

Thank you for reading. Your comments are welcome.