Saturday, July 30, 2011

Maven's assumptions

Maven does both - 'makes assumptions' as well as 'enforces' a lot of standards and good practices for project programming and management at an enterprise-level. It also provides some good features such as resource filtering, though not enabled by default, suggest good programming practice. I wish to list down all of them as they come to my mind from the very obvious to the subtle ones below:

  1. Projects would have a standard directory layout relative to project's home directory to place source code, target binaries, test source code, test binaries, resources etc as described in the Introduction to Standard Directory Layout.
  2. Projects would adopt unit-testing as a standard practice by including the test-compile and test phases as part of the build lifecycle. Maven even goes a step further by failing a build if the test source does not compile or the unit tests fail in its default build lifecycle.
  3. Projects provide a clean directory/folder separation between application source code and the resources required (such as Spring applicationContext files, Hibernate configuration files, log4j properties file etc) and not put everything together in project home folder.
  4. Projects need not hardcode application properties inside the configuration resources or source code and instead make use of resource filtering - i.e. a property like jdbc.driverName could be externalized in the /src/main/filters and its value will be substituted wherever it is referenced. Thus any changes to it are localized to one file under /src/main/filters although it can be referenced in several places in configuration and code.
  5. Modern-day applications (represented by an aggregate project model / POM in Maven) will be divided into separate modules - app-domain-model, app-DAO, app-business-logic, app-web-modules, app-webservices-modules, app-utils etc - instead of old-school monolithic applications. Maven multi-module projects and the Reactor plugin help achieve this.
  6. Projects would need to be built for various platforms (Windows, Unix, Linux etc) and deployment environments (dev, staging, test, production) and builds do differ based on the platform and enviroment needed. Eg. different database servers required for each enviroment thereby a different jdbc.url, jdbc.username, jdbc.password for each environment. Maven Profiles are a great feature that if leveraged well can make the build process seamless across environments.

Maven references

Recently I have been learning Maven as it is widely deployed and used at an organizational level in a very clever and effective manner. However, the Maven documentation, although quite extensive for an open-source project does not flow logically to educate a newbie. After scouring through most of the available online information in a not so logical progression and then trying to connect the pieces together, I came to a conclusion that the following would be the ideal order to comprehend Maven.

Online reference books made available by Sonatype.

All about Maven settings.

Core of Maven - Project Object Model and all its intricacies - multi-module builds, profiles, reactors etc

Tuesday, July 12, 2011

CVS and (no) Atomic Commits

Every project that I have worked so far has used a different version control system. My history of using version control systems has been thus - Rational ClearCase in my very first job/project, then Visual Sourcesafe, then Subversion for a very brief period of time (about 3 months) and then Perforce for the longest time as yet(for a little over 4 years) and now I am currently using CVS.

Using Perforce was a pleasant experience and its atomic commits combined with its changelist/changeset feature is something that I am missing while using CVS currently. In CVS, each file committed to the repository has its own, independent version number and history which sure is a limitation. I do not remember enough about Rational ClearCase except that its config-specs pretty much allowed a much efficient branching and merging in a manner that could emulate atomic commits. So this blog entry is about atomic commits and changesets and how they reduce the frequency of build breaks which seem to happen every once in a while in a high-traffic team workload.

A typical reason why a build could break is that developers fail to commit all of the files that go in as part of the task or a bug fix. As a consequence when one or more of the checked-in files try to access constructs(classes, methods, constants etc) newly introduced in the file that was forgotten to be checked-in. And thats the build breaking right in your face!

In my opinion, the kind of SCM being used as the source code repository(read CVS, Visual SourceSafe) can also be a contributing factor towards this. Ideally, when a developer commits files to the repository, it is great if the files are grouped together as a single atomic change towards the bug fix, new feature development or new task. So even mentally developers start to group all files together, thereby reducing the probability of a build break. Also in the event of a networking failure, atomic commits save the build by ensuring all or none of the files are submitted to the repository.

Perforce takes one more step beyond these atomic commits - if a developer modifies any of the local files that are mapped to the repository, it automatically puts them in a pool called 'changelist'. That way there is no chance that a developer could have forgotten to check-in any file.
Changelists serve two purposes:
- to organize your work into logical units by grouping related changes to files together
- to guarantee the integrity of your work by ensuring that related changes to files are checked in together.

Now different version control systems record this atomic commit differently in their history. For Subversion, a changeset/changelist is just a collection of changes with a unique name. The commit will create a new revision number which can forever be used as a "name" for the change. As per Subversion documentation:
    "In Subversion, a global revision number N names a tree in the repository: it's the way the repository looked after the Nth commit. It's also the name of an implicit changeset: if you compare tree N with tree N-1, you can derive the exact patch that was committed. For this reason, it's easy to think of “revision N” as not just a tree, but a changeset as well. If you use an issue tracker to manage bugs, you can use the revision numbers to refer to particular patches that fix bugs—for example, “this issue was fixed by revision 9238”. Somebody can then run svn log -r9238 to read about the exact changeset which fixed the bug, and run svn diff -r9237:9238 to see the patch itself."

Perforce keeps track of each file's independent revision history as well as changelist numbers. Again you can associate changelist numbers with a bug/task tracking database and we can go back and forth between the SCM and the bug/task tracking database.

As I searched through the web, I did find some solutions and forums that discussed ways to get around this limitation which will be the next task on my agenda.

Wednesday, June 29, 2011

Java 5 Enums

Enums or enumerated types basically means a type that can be defined to have a certain set of fixed values as per the problem domain. Historically, enums (via enum keyword and any associated semantics) were missing from the featureset provided by versions upto Java 1.4.
However developers tried to achieve the "same enum effect" via something like this:
Example 1:
public class Currency {
  public static final int USD = 1;
  public static final int EUR = 2;
  public static final int GBP = 3;
  public static final int YEN = 4;
}

public class CurrencyConverter {
  public void convertCurrency(int fromCurrency, int toCurrency) { ... }

  public static void main(String[] args) {
    CurrencyConverter cc = new CurrencyConverter();
    cc.convertCurrency(Currency.USD, Currency.YEN);
  }
}
Additional currencies could be added to the Currency class by defining new constants. However, the convertCurrency (int, int) method lacks typesafety since the method signature indicates it can accept any int. However, the only acceptabe range of ints is 1 through 4. If we call the method outside of the range of agreed upon constants , e.g. convertCurrency(8, 10), the program fails.

The above can be avoided if we accept that Enumerations should be treated as a separate type. Implementing them as a sequence of integers is not helpful. To define enumerations as their own type, you do the following:

1. Replace the primitive ints above with 'static final' object references to the same class defining the enumerated constants.
2. Disallow any object creation of the class via a private constructor.

Example 2:
public final class Currency {
  public static final Currency USD = new Currency(1);
  public static final Currency EUR = new Currency(2);
  public static final Currency GBP = new Currency(3);
  public static final Currency YEN = new Currency(4);

  int value;

  private Currency(int value){
    this.value = value;
  }
}

public class CurrencyConverter {
   public void convertCurrency(Currency fromCurrency, Currency toCurrency) { ... }

   public static void main(String[] args) {
     CurrencyConverter cc = new CurrencyConverter();
     cc.convertCurrency(Currency.USD, Currency.YEN);
   }
}
The convertCurrency(Currency, Currency) now takes the Currency type instead of an int.

Secondly the acceptable values of Currency can only be defined inside the class due to the private constructor.This along with the fact that Currency is a final class ensures that Currency.USD, Currency.EUR, Currency.GBP and Currency.YEN are the only instances of the Currency class.

It also means that we can use the identity comparison (==) operator instead of the equals() method when comparing enum values. Identity comparison (==) is always faster than equals since we are only comparing object references in the former as opposed to object values in the latter.

However the above typesafety approach comes with the following disadvantages:
1. The above implementation is not Serializable and Comparable by default. It means we can have issues using them in the context of RMI and EJBs. In case, if we make them Serializable, constructing the object again creates a new instance of the same Currency by ignoring its private constructor completely and does not retrieve the same instance that was serialized. This means == comparison fails to identify the equality of a serialized and a non-serialized Currency. Also it means we no longer have a unique single instance of the currency type. We have to implement more boiler-plate code like implementing the readResolve method as suggested in http://www.javaworld.com/javaworld/javatips/jw-javatip122.html?page=2.

2. We cannot switch over the above enum values (remember it is easier to switch over ints) to get any business logic done. If we need to switch, it can be facilitated by providing a getValue() method that returns the int value.

Example 3:
Inside Currency class,
public class Currency {
      ....

      public int getValue() {
        return value;
      }
   }

   public class CurrencyConverter {
     public void convertCurrency(Currency fromCurrency, Currency toCurrency) { ... }
   }
Java 5 enums are a typesafe feature and overcome all the above problems listed with the Enumerated pattern. In addition to facilitating a way to list a set of constant values, they also provide features such as :

1. All defined enums implicitly extend from java.lang.Enum just as all objects implicitly extend from java.lang.Object.

2. The above feature taks care of default implementation for toString(), equals(), hashCode() methods.

3. They are Serializable and Comparable by default without generating duplicate instances during deserialization

4. They can be used in switch-case statements.

5. They can have behavior ( via member variables , methods , constructors, interface implementations etc) in addition to just specifying the constants.

Simplest example of Java 5 enum class with no additional behavior.

Example 4:

public enum Currency { USD,GBP,EUR,YEN }

public class CurrencyConverter
{
   public void printCurrencies() {
     for (Currency currency : Currency.values()) {
        System.out.println(currency);
        System.out.println(currency.ordinal());
   }
} 

}
Currency is an enum type, and all the above enum values, viz USD, GBP, EUR, YEN are of type Currency.

We can iterate through all the instances of Currency via the static values() method and take advantage of the toString() method in the print statement.

Example 5:
We can also add behavior via member fields and methods.

public enum Currency 
  {
    USD("United States"),
    GBP("United Kingdom"),
    EUR("Europe"),
    YEN("Japan")

    String country;

   public Currency(String country){
     this.country = country;
   }

   public Currency getCurrencyForCountry(String country) {
     return Currency.valueOf(country);
   }
 }

When you need to provide custom behavior based on the enum values, there are 2 ways of doing it. Either you switch case based on the enum values in the application code or a yet better way is to move the custom logic inside the enum class as follows:

Example 6:
public class Client 
   {
     enum HttpStatusCode
     {
        HTTP200("HTTP 200") {
         @Override
         void printMessage() {
            System.out.println("Successful Transaction ");
         }
     },
     HTTP401("HTTP 401 Error") {
        @Override
        void printMessage() {
          System.out.println("Authentication Failure");
        }
     },
     HTTP404("HTTP 404 Error") {
          @Override
          void printMessage() {
             System.out.println("Requested resource not found at specified location");
           }
     },
     HTTP500("HTTP 500 Error") {
       @Override
       void printMessage() {
          System.out.println("An error occured on server-side, please have   patience.");
      }
    };

    String statusString;

    HttpStatusCode(String statusString) {
       this.statusString = statusString;
    }

    abstract void printMessage();
 }

   public static void main(String[] args)
   {
     HttpStatusCode status = connectToServer();
     status.printMessage();
   }
 }

Switching over case statements could be used if you have no option of modifying the enum class code. This can happen in cases where the enum class is generated - e.g. using JAXB - from an XSD. More on this in a later blog.

So this covers the basics of Java 5 Enums. Java 5 also provides 2 data structures - java.util.EnumSet and java.util.EnumMap. More on this again will be in yet another blog.


Monday, January 5, 2009

Tomcat 6 and class loading

In continuation with the previous blog entry, I would also like to write about Tomcat 6 (in particular) and class loading pattern that it adopts. Java allows the creation of custom class loaders by implementing the java.lang.ClassLoader. Now Tomcat 6 creates the following class loaders on startup. They share a parent-child relationship too, but NOTE that the delegation pattern is a bit different as will be explained

Although invisible in default installation of Tomcat 6, there are additional shared and server class loaders also available and they fall below the Common class loader in the hierarchy. Each of the class loaders has a responsibility to load classes from certain specific areas, noted below:  

1. Bootstrap + Extension class loader - It loads the Java run-time classes in the JDK as well as any classes from the jars in the Extensions folder.  

2. System class loader - As noted in the previous blog, the System class loader is responsible for loading the classes and the Jar classes present in the CLASSPATH. But an important NOTE here: Tomcat clears the user-set CLASSPATH entry in its startup.bat or startup.sh file. Instead it sets the CLASSPATH to be following: $CATALINA_HOME/bin/bootstrap.jar $CATALINA_HOME/bin/tomcat-juli.jar  

3. Common class loader - This is a Tomcat 6 provided class loader. It loads the classes present in the following folder - $CATALINA_HOME/lib. These classes are available to Tomcat as well as all the web applications that will be hosted on this instance of Tomcat. Although developers can reference the APIs from the jars inside the $CATALINA_HOME/lib directory, they shouldn't be placing their own custom classes and/or jars in there. If developers need certain custom classes and/or jars to be shared by all web applications, then they should be placed where the shared class loader can see them. Note that Tomcat 6.0.14 the $CATALINA_HOME/shared/lib directory does not exist. So this can be done in Tomcat 6 as foll:
  • Create your own $CATALINA_HOME/shared/lib directory.
  • Modify $CATALINA_HOME/conf/catalina.properties by changing the line: shared.loader = ${catalina.home}/shared/lib
However the above does not apply to certain 3rd party libraries such as database drivers etc where Tomcat itself would need to set up data sources. Such jars have to be placed in the $CATALINA_HOME/lib folder for the common class loader to see. One can also add more jars for the common class loader without placing them under the $CATALINA_HOME/lib folder. This can be done by modifying $CATALINA_HOME/conf/catalina.properties by changing the property common.loader as above.  

4. WebappX class loaders - Tomcat creates a class loader for every webapp that is deployed in its instance. This class loader loads classes under WEB-INF/classes and WEB-INF/lib folder. It is for these class loaders where the delegation model deviates, thanks to the Servlet Specification which states as follows: "It is recommended also that the [web] application class loader be implemented so that classes and resources packaged within the WAR are loaded in preference to classes and resources residing in container-wide library JARs."
However the above specification cannot override the Java standard delegation model of delegating to Bootstrap and System class loaders. It only is used to override the parent-child relationships that are introduced by Tomcat - ie. Common, Shared and WebappX class loaders. So when an application requests a class, the class loading hierarchy is as follows:
  1. The bootstrap class loader looks in the core Java classes folders.
  2. The system class loader looks in the $CATALINA_HOME/bin/bootstrap.jar and
  3. $CATALINA_HOME/bin/tomcat-juli.jar
  4. The WebAppX class loader looks in WEB-INF/classes and then WEB-INF/lib
  5. The common class loader looks in $CATALINA_HOME/lib folder.
  6. The shared class loader looks in $CATALINA_HOME/shared/classes and $CATALINA_HOME/shared/lib if the shared.loader property is set in conf/catalina.properties file.

Sunday, January 4, 2009

Java and Class loading delegation model

The role of a class loader in Java is to hide the details of loading classes - like searching the file system (local as well as network) for the class file, loading the class file, returning it to JVM as a Class class so that JVM can use the Class class to instantiate the requested object in the application. Since J2SE, the JVM is provided 3 distinct primary classloaders -
  1. Bootstrap class loader - This class loader is written in native language and comes as part of the JVM implementation. It loads all the core Java classes. (e.g java.lang.* etc). The location of these jars depends on the implementation of JVM. Sun's JVM looks in the jdk/jre/lib directory. 
  2. Extension class loader - Usually, developers make use of the CLASSPATH environment variable to load application classes, 3rd party jars etc. However, the CLASSPATH can become too unwieldy to handle and prone to errors using this approach. So since Java 1.2 , we can drop the 3rd party jars into a standard extension directory - jdk/jre/lib/ext - and JVM will find them. The extension class loader loads all the classes found in one or more of these extension directories.
  3. System class loader - This class loader locates and loads the classes in the directories and jar files specified on the CLASSPATH variable. It also loads the application's main class (the one containing the main() method).

 The above 3 exist in a parent-child relationship as follows:

A delegation model is utilized by the JVM in order to determine which class loader amongst the above 3 to use to load a particular class. The model works in a "Delegate to Parent-Before-Looking" as follows: When an application requests a Class (e.g String str = "Hello World"; or MyClass myobj = new MyClass()) :
  1. Each class loader delegates the request to its parent.
  2. Once the topmost class loader is reached, the bootstrap class loader, it tries to load the class. If it is unable, its child will try.
  3. If one of the class loaders finds the class, it is returned as a Class object. Following the delgation pattern , if the lowermost class loader in the hierarchy (System class loader) does not find the class, a ClassNotFoundException is thrown.

Sunday, December 7, 2008

Java 5 Concurrency: The Executor framework

The Java platform has always provided support for multi-threaded/concurrent programming. However, prior to Java 5, the support was in the form of primitive constructs in the programming language itself. Java 5 steps up and provides concurrency utility frameworks and data structures in the java.util.concurrent package. One of the utilities provided is the task scheduling framework better known as the Executor framework. The JVM runs as a process and our application is one of the threads in the JVM. There are various other "system" threads running to do tasks like garbage collection, memory management etc. But from the application's perspective, there is the single "main" thread to begin with. The application, in itself, can spawn a number of threads to perform various helper tasks for various reasons like performance etc. Prior to Java 5 , spawning a new thread to perform a task was most commonly done as follows, although you could also do by extending the Thread class, but it is not highly recommended:

public void mainMethod() {
 HelperTask task = new HelperTask(); // Step 1: Create an object representing the task
 Thread t = new HelperThread(task);  // Step 2: Create a new thread for executing the task
 t.start();                        //  Step 3: Start the new thread
}

public class HelperTask implements Runnable {
public void run() {
      doHelperTask();
}
}
We have the following issues:
  1. Most of the code related to thread creation and task delegation to the thread is a part of the application itself. We need a way to abstract the above steps away from the application.
  2. Also what if we have multiple helper tasks or a scenario where every single user action requires a new Thread to be spawned to process? Creating a lot many threads with no bounds to the maximum threshold can cause out application to run out of memory.
  3. Secondly although threads are light-weight (but only as compared to the process) , creating them utilizes a lot of resources. In such a situation, having a ThreadPool is a better solution so that only fixed number of Threads are created and re-used later.
  4. Another short-coming of the Runnable interface's void run() method is that the task executed within run() has no way of returning any result back to the main thread. So work-arounds designed around that would be that the asynchronous task either updates certain database table(s) or some file(s) or some such external data structure(s) to communicate the result to the main thread.
The Executor framework addresses all the above issues and in addition also provides additional life-cycle management features for the threads. The Executor framework consists of the following important interfaces:

  1.  Callable: This interface is similar in concept to Runnable interface, ie it represents the asynchronous task to be executed. The only difference is that its call() method returns a value, ie the asynchronous task will be able to return a value once it is done executing. 
  2. Executor, ExecutorService and ScheduledExecutorService - Each of these interfaces adds more functionality to the previous one in thread and their life-cycle management. The Executor abstracts the Thread creation (as seen in Step 1 above) and executes all Runnable tasks. The ExecutorService extends the Executor and is able to execute Callable tasks in addition to Runnable tasks. It also contains life cycle management methods. The ScheduledExecutorService allows us to schedule the asynchronous tasks thereby adding support for delayed and periodic task execution. 
  3. Future: This interface represents the result of the asynchronous task which itself could be represented as Callable. The ExecutorService which can execute Callable tasks returns a Future object to return the result of the Callable task.