Showing posts with label Hibernate. Show all posts
Showing posts with label Hibernate. Show all posts

Wednesday, August 17, 2011

Key Concepts to Understand Spring's Consistent Transaction Handling

Class FooBarService shows simplified pseudo code for different transaction handling paradigms using JDBC,Hibernate and JTA.
Class FooBarServiceSpring shows
simplified pseudo code for consistent transaction handling using Spring.
Obviously Spring's transaction handling is quite different from JDBC and Hibernate; but is closer to JTA. However, by no mean does Spring only support JTA. Instead, it is only a matter of configuring different PlatformTransactionManager (PTM hereafter) in your application context in order to support different transaction handling paradigms.  For example, Spring's DataSourceTransactionManager, HibernateTransactionManager and JTATransactionManager are for JDBC, Hibernate and JTA, respectively.

This discussion can help readers understand some internals of Spring's  PTM implementations on heterogeneous native transaction platforms.


//different transaction paradigms using JDBC,Hibernate and JTA
class FooBarService {
  //local transaction paradigm using JDBC
  public void foo() {
    //creates a session with the backend database
    Connection conn = dataSource.getConnection();
    //starts a new local transaction.
    conn.setAutoCommit(false);

    updates-data-through-this-conn

    //commits the local transaction
    conn.commit();
    //closes the session
    conn.close();
  }

  //local transaction paradigm using Hibernate
  public void bar() {
    //creates a session with the backend database
    Session session = sessionFactory.openSession();
    //starts a new local transaction.
    Transaction tx = session.beginTransaction();

    updates-data-through-this-session

    //commits the local transaction
    tx.commit();
    //closes the session
    session.close();
  }

  //Global transaction paradigm using JTA
  public void fooBar() {
    userTransaction.begin();

    //creates a session with the backend database1
    Connection conn1 = dataSource1.getConnection();
    updates-data-through-this-conn1

    //creates another session with the backend database2
    Connection conn2 = dataSource2.getConnection();
    updates-data-through-this-conn1

    //closes the two sessions
    conn1.close();
    conn2.close();

    userTransaction.commit();
  }
}


//consistent transaction handling paradigms using Spring
class FooBarServiceSpring {
  @Transactional
  public void foo() {
    //updates data through some data source connection
  }
 
  @Transactional(propagation=Propagation.REQUIRES_NEW)
  public void bar() {
    //updates data through some data source connection
  }
  @Transactional
  public void fooBar() {
    foo();
    bar();
  }
}


1.
Key Transaction Abstraction
The following 3 interfaces are key to understand Spring's transaction handling:
  • TransactionDefinition
    It allows you to define transaction requirements (isolation level, propagation behavior, timeout and read-only status) before your transaction is started.
    In FooBarServiceSpring, you specify your TransactionDefinition in @Transactional.
  • TransactionStatus
    Once your transaction is started, this interface allows you to query its current status including rollback flag and new vs existing flag, and to mark it to rollback only (for example when you encounter an exception).
    Spring's default TransactionStatus implementation also includes the underlying native transaction such as a connection for JDBC, a session for Hibernate and a user transaction for JTA.
    In FooBarServiceSpring, @Transactional allows you to specify your rollback rules. For other status information, you either implicitly knows(such as IsNewTransactio and IsComplete) or can retrieve from the underlying native transaction bound to the thread (Spring binds JDBC connection and Hibernate Session to the thread; JTA's UserTransaction is also bound to a standard JNDI name). Please note that Spring doesn't bind TransactionStatus to the thread because PTM can pass it around in your AOP proxied methods.
  • PlatformTransactionManager
    It wires together the above 2 interface and works as a coordinate overall.
    Specifically its getTransaction() method takes your TransactionDefinition and creates a TransactionStatus representing either a new or existing transaction.  Its commit() and rollback() methods take the returned TransactionStatus and commit and rollback the target transaction, respectively.
    It allows you to demarcate transactions as a singleton because it can get different transactions bound on different threads.
    In FooBarServiceSpring, @Transactional uses whatever transaction manager you configured in your application context such as DataSourceTransactionManager for JDBC.
When we say Spring transaction, we really mean TransactionDefinition and TransactionStatus. The PTM is more like a JTA UserTransaction / TransactionManager. But once again, it handles all types of native transaction API's behind the scene.
    2. Physical Transaction and Logical Transaction
    FooBarServiceSpring's foobar() calls foo() and bar(). Suppose there is no transaction before you call foobar(). So foobar() creates a new physical transaction while foo() starts a nested logical transaction due to its default propagation being REQUIRED; the logic transaction ends when foo() returns. 
    But bar() suspends the current transaction and creates a new physical transaction due to its propagation being REQUIRED_NEW. The new transaction ends and the suspended transaction is resumed when bar() exits.
    Because each (physical or logic) transaction always has a begin and end, we sometimes use transaction scope.

    For data persistence, only the outer physical transaction can commit or rollback. But
    on each individual inner logic transaction (method) level, you always specify your TransactionStatus and can mark transaction to rollback . However the embed native transaction in TransactionStatus is either new (if it is a physical one) or existing (if it participates in an existing one).
    You can also turn on the "validateExistingTransaction" flag in PTM
    so that the inner logic transaction will reject participation if its TransactionStatus is not compatible to the outer TransactionStatus.

    3. Resource, Connection, Session, Transaction and Resource Manager (RM)
    Resource can mean any valuable data; but in PlatformTransactionManager it is either a connection (for JDBC and JMS etc) or session (for Hibernate and JMS etc).

    Both connection and session represent a communication link with the some backend resource manager (RM for short). In the database world, both termscan be used interchangeably. Although Hibernate uses session, it uses a connection behind the scene.
     
    Transaction represents a unit of work. We mentioned in the above Section 1 that Spring's abstract transaction covers an underlying native transaction.
    For JDBC and Hibernate, you create local transactions using connections / sessions and because they have one-to-one relationship, you can sometimes interchange the 2 terms. In other words, if the inner logic transaction participants in the outer physical transaction, it must reuse the the same connection / session.
    For JTA, you creates global transactions using transaction managers; multiple connections can be enlisted in global transactions as transaction branches. One thing that is very different from JDBC and Hibernate is that you can conduct DML changes using different connections to the same backend RM instance in the same global transaction (This is because when a connection is enlisted, XID is created to represent this branch both in the JTA and the RM. Even you use a different connection, the RM can still identify the same transaction using the passed-in XID).

    RM managers resources (transaction) on behalf of you. It is commonly used in JTA/XA world. For most of us, databases are common RM.

    4. Transaction and Session -- Which Comes First?
    In the foo() and bar() of FooBarService, you first create a connection / session, then starts a transaction. Before you close the connection / session, you first commit / rollback the transaction.
    But in the foobar() of
    FooBarService, you first starts a transaction, then create a connection / session. Before you commit / rollback the transaction, you first close the connection / session.
    In FooBarServiceSpring, all you have is just @transactional and you are not allowed to open or close any connection / session explicitly. So how can Spring hide the difference between JDBC, Hibernate and JTA and whois taking care of connections / session anyway?
    Data access experience tells us that we always need a connection / session and its lifecycle methods (open, close etc) are just boilerplate and resource will leak if you forget to call the close method (actually this happens very often). But we must define transaction requirements by ourselves. For example, only you know that the bar() in
    FooBarServiceSpring requires REQUIRE_NEW propagation based on your business rules.

    So Spring is taking care of connections / sessions behind the scene. You just need to specify you transaction requirement using @transactional. The next section tell you more details. 
    5. Transaction and Session -- Resource Synchronization with Transaction
    Because you always explicitly specify your transaction requirements, Spring synchronizes the needed resource (connection or session) with the transaction at both transaction start time and ending time or at the transaction scope boundaries.
     

    For JDBC and Hibernate, Spring binds the connection / session (hence also local transaction) to the thread so that it is always available on the execution call stack. 
    Spring PTM automatically either retrieves an existing connection / session from the thread (if an outer transaction is already there) or creates a new connection / session and binds it to the thread before starting a transaction.
    When transaction is right before being committed or rolled back, Spring PTM also automatically closes the connection / session and unbinds it from the thread.


    For JTA, Spring gets the UserTransaction from the standard JNDI name, so the global transaction is also available on the execution call stack.
    When you retrieves a connection / session, it is automatically enlisted in the global transaction (Spring doesn't help here; the underlying XADataSource usually does the magic).

    When the JTA transaction is right before being committed or rolled back, Spring PTM once again automatically closes the connection / session .

    6. Nested Transaction and Transaction Suspend / Resume
    • For nested transaction, both  the outer and inner transaction are live in your thread while when a transaction is suspended, your thread work on another new transaction;
    • Nested transactions are dependent i.e. only the outer transaction is physical and can commit the change; but you can rollback the inner logic transaction.
      The suspended transaction and the new transaction are two physical transactions and are independent i.e. they can both commit or rollback independently;
    • Suspending tranasction is deadlock-prone because the backend RM still holds the resource locks on the suspended transaction even your JTA has switched to another new transaction. You should be very careful even your application also locks some resources in the suspended transaction that are needed by the new transaction;
    • Nested transactions are only supported by JDBC with the savepoint feature. Because XA doesn't support nested transactions, neither JTA can (This seems to be ironic because most databases support savepoint);
    • Suspending a JDBC transaction in Spring means first unbinds it from the thread, then creates a new connection and binds to the thread. For JTA, Spring just delegates the suspend/resume to the underlying UserTransaction in addition to its own bookkeeping for resource synchronizations.
    7. Propagation.SUPPORTS and No Transaction
    No transaction means your data changes are committed on each individual SQL statement level (it is autocomit for JDBC) instead of grouping several SQL statements in a transaction.

    Propagation.SUPPORTS also commits data changes on each individual SQL statement level if there is no existing transaction. But if you set your TransactionSynchronization in PTM to ALWAYS, it also supports resource synchronization on an empty transaction scope (empty transaction scope means no underlying native transaction).
    To make it concrete, the same JDBC connection or Hibernate Session will bind to the thread inside the empty transaction scope for reuse in the execution call stack until the scope ends.  Be aware that the connection is of course in autocommit mode.

    Please remember that if you empty transaction scope has active resource synchronization, DON'T nested Propagation.REQUIRED or Propagation.REQUIRED_NEW in it because resources synchronizations for the 2 transaction scopes will conflict (unfortunately you have to read the PMT source code in order  to get to the bottom).



    Monday, October 25, 2010

    Is the "ORDER BY" in HQL and EJB-QL based on lexicography or dictionary?

    This topic is inspired by Dominik Dunz's comment on my Hibernate tuning article "Revving up Your Hibernate Engine" on InfoQ.

    In Java, you sort strings lexicographically (based on the underlying character's encoding values) using class String's compareTo() method.
    You can also sort strings based on a locale's dictionary using class Collator's compare() method.

    So now you should ask which sorting the "order by" I wrote in HQL or EJB-QL supports?
    Currently there is no any QL syntax for you specify either a lexicographic or dictionary order. So both HQL and EJB-QL just literally pass the "order by" clause to the back-end database. It is your database session that decides the sorting.

    In case of Oracle, it also supports sorting lexicographically (binary in Oracle's term) or based on dictionary(linguistic in Oracle's term).
    Your Oracle session decides the sorting. Specifically if the session's NLS_COMP is "binary" it will sort lexicographically(based on the string's underlying encoding values).
    If the session's NLS_COMP is "linguistic", it will sort based on the dictionary of the locale that you specified in NLS_SORT.

    If you use Oracle's JDBC thin driver in an application server, the application server's JVM decides the values of NLS_COMP and NLS_SORT.

    You can always do sorting in your application tier based on your business logic instead of relying on your database. But your application tier sort probably will be slower than your DB sorting.
    However there are many complications if you want to use your back-end database sorting to implement your business logic sorting.
    1. Your database may only support lexicographical sorting;
    2. Even lexicographical sorting is much simpler than dictionary sorting, your database session's character encoding may not be Java string's UNICODE. However it may not be a big deal to change your DB's charactor encoding to Java string's UNICODE or be a subset of Java string's UNICODE.
      You also need to make sure that your DB's lexicographical sorting is the same as your Java's. In case of Oracle, it basically has the same lexicographical sorting as Java String's compareTo() method.
    3. Java's linguistic sorting may not be the same as your DB's. You need to carefully exam documents from both Java and your DB.
      You can find Oracle's linguistic sorting logic from this link.
    4. It is becoming more complicated if you have a there-tier architecture where the front-end UI (either Swing or a browser) decides the sorting logic because the same database session in the back-end can be shared by different front-end user sessions.
      You have to change your DB session's sorting whenever your fron-end UI has changed.

    Thursday, October 14, 2010

    How to use 2 or more data sources in Hibernate along with Spring's Declarative Transaction Management?

    You may quickly response "just use Spirng's JtaTransactionManager".
    But wait. Before deciding to use JTA, you should make sure that local transaction really doesn't meet your requirement because JTA requires many more resources and is much slower than local transactions.
    Even you have 2 or more data sources, you don't need to use use JTA in the following cases:
    • No business method has to access more than 1 data source;
    • Event your business method has to access more than 1 data source, you can still use a technique similar to “Last Resource Commit Optimization" with local transactions if you can tolerate occasional data inconsistency. 
    Here is the example in my "Revving up Your Hibernate Engine": 

    Our application has several service layer methods which only deal with database “A” in most instances; however occasionally they also retrieve read-only data from database “B”. Because database “B” only provides read-only data, we still use local transactions on both databases for those methods.
    The service layer does have one method involving data changes on both databases. Here is the pseudo-code:
    //Make sure a local transaction on database A exists
    @Transactional (readOnly=false, propagation=Propagation.REQUIRED)
    public void saveIsoBids() {
      //it participates in the above annotated local transaction
      insertBidsInDatabaseA();
      //it runs in its own local transaction on database B
      insertBidRequestsInDatabaseB(); //must be the last operation

    Because insertBidRequestsInDatabaseB() is the last operation in saveIsoBids (), only the following scenario can cause data inconsistency:
    The local transaction on database “A” fails to commit when the execution returns from saveIsoBids ().
    However even if you use JTA for saveIsoBids (), you still get data inconsistency when the second commit phase fails in the two phase commit (2PC) process. So if you can deal with the above data inconsistency and really don’t want JTA complexities for just one or a few methods, you should use local transactions. 

    Now suppose you will use local transaction, i.e.Spring's HibernateTransactionManager. In your context XML, you define the needed transaction manager bean:
      <tx:annotation-driven transaction-manager="txManager1"/>
      <bean id="txManager1"
     class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory" ref="sessionFactory1"/>
      </bean>

      <bean id="sessionFactory1"
     class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
        <property name="dataSource" ref="dataSource1"/>
      </bean>

    The above XML just configured one data source and its related Hibernate session factory and transaction manager.
    The above annotation-driven element only allows you to specify one transaction manager (it should do this way otherwise which transaction manager will the annotated service method use?) and it is global.
    So what happens if you specify another transaction manager in a different context XML:
      <tx:annotation-driven transaction-manager="txManager2"/>
      <bean id="txManager1"

    Based on my testing, the result is unexpected.

    The solution is you can only use the transaction manager along with Spring's declarative transaction; other transaction managers must use Spring's XML configuration:

    <tx:advice id="txAdvice" transaction-manager="txManager2">
      <tx:attributes>
        <!-- all other methods starting with 'get' are read-only -->
        <tx:method name="get*" read-only="true"/>

        <!-- other methods use the default transaction settings (see below) -->
        <tx:method name="*"/>
      </tx:attributes>
    </tx:advice>

    <bean id="txManager2"
     class="org.springframework.orm.hibernate3.HibernateTransactionManager">
      <property name="sessionFactory" ref="sessionFactory2"/>
    </bean>

    <aop:config>
      <aop:pointcut id="serviceOperation" expression="...ignored"/>
      <aop:advisor advice-ref="txAdvice" pointcut-ref="serviceOperation"/>
    </aop:config>

    Asynchronous (non-blocking) Execution in JDBC, Hibernate or Spring?

    There is no so called asynchronous execution support in JDBC mainly because you want to wait for the result of your DML or DDL most of the time or because there is too much complexity involved between the back-end database and the front end JDBC driver. 
    Some database vendors do provide such support in their native drives. For example Oracle supports non-blocking calls in its native OCI driver. Unfortunately it is based on polling instead of callback or interrupt.
    Neither Hibernate or Spring supports this feature.

    But sometimes you do need such a feature. For example some business logic is still implemented using legacy Oracle PL/SQL stored procedures and they run pretty long. The front-end UI doesn't want to wait for its finish and it just needs to check the running result later in a database logging table into which the store procedure will write the execution status.
    In other cases your front-end application really cares about low latency and doesn't care too much about how individual DML is executed. So you just fire a DML into the database and forget the running status.

    Nothing can stop you from making asynchronous DB calls using multi-threading in your application. (Actually even Oracle recommends to use multi-thread instead of polling OCI for efficiency).
    However you must think about how to handle transaction and connection (or Hibernate Session) in threads.
    Before continuing, let's assume we are only handling local transaction instead of JTA.

    1. JDBC
    It is straightforward. You just create another thread (DB thread hereafter) from the calling thread to make the actual JDBC call.
    If such a call is frequent, you call use ThreadPoolExecutor to reduce thread's creation and destroy overhead.

    2. Hibernate
    You usually use session context policy "thread" for Hibernate to automatically handle your session and transaction.
    With this policy, you get one session and transaction per thread. When you commit the transaction, Hibernate automatically closes the session.
    Again you need to create a DB thread for the actual stored procedure call.

    Some developer may be wondering whether the new DB thread inherits its parent calling thread's session and transaction.
    This is an important question. First of all, you usually don't want to share the same transaction between the calling thread and its spawned DB thread because you want to return immediately from the calling thread and if both threads share the same session and transaction, the calling thread can't commit the transaction and long running transaction should be avoided.
    Secondly Hibernate's "thread" policy doesn't support such inheritance because if you look at Hibernate's corresponding ThreadLocalSessionContext, it is using ThreadLocal class instead of InheritableThreadLocal.

    Here is a sample code in the DB thread:
    // Non-managed environment and "thread" policy is in place
    // gets a session first
    Session sess = factory.getCurrentSession();
    Transaction tx = null;
    try {
      tx = sess.beginTransaction();

      // call the long running DB stored procedure

      //Hibernate automatically closes the session 
      tx.commit();
    }
    catch (RuntimeException e) {
      if (tx != null) tx.rollback();
      throw e;
    }

    3.Spring's Declarative Transaction

    Let's suppose your stored procedure call is included in method:
      @Transactional(readOnly=false)
      public void callDBStoredProcedure();

    The calling thread has the following method to call the above method asynchronously using Spring's TaskExecutor:
      @Transactional(readOnly=false)
      public void asynchCallDBStoredProcedure() {
            //creates a DB thread pool
            this.taskExecutor.execute(new Runnable() {
                @Override
                public void run() {
                    //call callDBStoredProcedure()
                }
            });
      }

    You usually configure Spring's HibernateTransactionManager and the default proxy mode (aspectj is another mode) for declarative transactions. This class binds a transaction and a Hibernate session to each thread and doesn't Inheritance either just like Hibernate's "thread" policy.

    Where you put the above method callDBStoredProcedure() makes a huge difference.
    If you put the method in the same class as the calling thread, the declared transaction for callDBStoredProcedure() doesn't take place because in the proxy mode only external or remote method calls coming in through the AOP proxy (an object created by the AOP framework in order to implement the transaction aspect. This object supports your calling thread's class by composing an instance of your calling thread class) will be intercepted. This meas that "self-invocation", i.e. a method within the target object (the composed instance of your calling thread class in the AOP proxy) calling some other method of the target object, won't lead to an actual transaction at runtime even if the invoked method is marked with @Transactional!

    So you must put callDBStoredProcedure() in a different class as a Spring's bean so that the DB thread in method asynchCallDBStoredProcedure() can load that bean's AOP proxy and call callDBStoredProcedure() through that proxy. 

    Wednesday, October 13, 2010

    Protect Hibernate Collection

    Suppose you have a unidirectional one-to-many association between department and employee.The department meta configuration uses eager loading for its employees; and the cascade is "all, delete-orphan".

    A developer usually makes the following set association in the department pojo:

    public class Department {
      private Set employees;

      public SetgetEmployees() {
          if (this.employees == null) {
            this.employees = new HashSet();
          }
          return this.employees;
      }

      public void setEmployees(Set employees) {
        this.employees = employees;
      }

    } //end of class Department

    The problem is with the method setEmployees(). Suppose you loaded a department object along with its 10 employees to your front-end UI, then you removed 2 employees - employee1 and employee2, form the set and added a new one employee11.
    If you pass your new set of employees in a new Java Set instance and call setEmployees(), you will not get what you want.
    This is the result: employee1 and employee2 are not deleted from your back-end DB as you expect. But the new employee11 was indeed inserted into the back-end DB.

    This is why. When Hibernate initializes the employees set, it replaces it with its own version of set "PersistentSet" for bookkeeping among other reasons.
    So if you remove the 2 employees from Hibernate's set, it will remember your deleting action. Otherwise Hibernate simply doesn't know your want to delete anything.

    This will further cause unique constraint problem if you later changed your mind and don't want to delete a previously removed employee from the set.
    For example, you first removed employee1 from the set, then you rolled it back by creating a new employee instance that has the same identify value(suppose the identify property is SSN and your DB has a unique constraint on SSN).
    You are allowed to put the new employee into the set because you have removed the original employee1 from the set. But when you try to save the set into the database, you will get a unique constraint problem because the original employee1 is not deleted from the DB.

    So you should defensively change method setEmployees() to protected and add some helper method. So the Department class looks like:

    public class Department {
      private Set employees;

      public SetgetEmployees() {
          if (this.employees == null) {
            this.employees = new HashSet();
          }
          return this.employees;
      }

      //leaves it to Hibernate
      protected void setEmployees(Set employees) {
        this.employees = employees;
      }
      public boolean addEmployee(Employee employee) {
        return getEmployees().add(employee);
      }
      public boolean removeEmployee(Employee employee) {
        return getEmployees().remove(employee);
      }

    } //end of class Department



    Lastly why could you still save the new employ11 even using a new Java Set instance. This is because the new employee11's ID value is either null or 0 that is different from the id's "unsaved-value".

    Monday, August 16, 2010

    Daylight Saving Time (DST) and Timezone Handling in Java, JDBC and Oracle

    The following problem has been puzzling me for 2 days.
    Because MISO (Midwest ISO. A power market for several Mid West regions) doesn't support DST, it sent us data at hour 2 on Mar 14,2010 which is the DST beginning date (the hour 2 is supposed to be skipped if the ISO supports DST).

    The date values in question are stored in an Oracle column called endDate which is of Oracle's date type without time zone information. In other words, you interpret such a date type's components (year,month,day,hour,minute,second and millisecond) in your local timezone.
    If your data are across timezone, you should use Oracle's "timestamp with timezone" or "timestamp with local timezone". Such a date type has an additional timezone component based on which you interpret other components.

    This additional timezone component is the key to understand the difference between Oracle's date type and Java's date type.
    Because Java's date type represents the specified number of milliseconds since the standard base time known as "the epoch", namely January 1, 1970, 00:00:00 GMT. (Don't miss this GMT timezone!)
    When you try to load an Oracle date value without a timezone into a Java date object, you will face difficulty without using any timezone in Java.

    We used Hibernate's TimestampType to map this DB column to a Java Date type.
    I used the following code to extract the hour in the endDate:
        Calendar cal = Calendar.getInstance();    //(1)
      int he;                                   //(2)
           
      cal.setTime(getEndDate());                //(3)
      he = cal.get(Calendar.HOUR_OF_DAY);       //(4)
    The code is running in east coast which is currently in DST (it is August 2010). The hour in line (4) returns 1 for the hour 2 in question and the getEndDate()'s toString() also shows hour 1 instead of hour 2 or hour 3.

    Getting to know why this happened is quite involving and confusing mainly due to the DST switch. We explain it in three steps.

    First we need to know how Hibernate and JDBC driver retrieves you endDate value to a Java date object.
    Hibernate's TimestampType just calls the following ResultSet's method to get the endDate.
       java.sql.Timestamp getTimestamp(String columnLabel);  
    Because our DB column doesn't have timezone, what timezone will be used in the returned Timestamp (it extends date type) in the above method (still remember my previous statement "Java's date type represents the specified number of milliseconds since the standard base time known as "the epoch", namely January 1, 1970, 00:00:00 GMT. ")?
    Astute readers may recall there is any similar method in ResultSet that allows you to provide a timezone through a Calendar:
       java.sql.Timestamp getTimestamp(int columnIndex, Calendar cal) ;
    This method is exactly to handle a DB column without a timezone so that you can interpret the DB date times in a specified timezone based on your business logic.

    The Java document for the first method doesn't say what time zone is associated with the returned timestamp value. This is unfortunate and different JDBC drivers may do different things. Later I will present what I found based on my testings.
    Actually Java's date doesn't care about any time zone per se; it only remembers the the number of milliseconds since the "epoch". It is Java's calendar that incorporates a timezone (and also a locale).
    Remember the preferred way to create a date object is to create a calendar first then call its getTime() to return the date. Calendar's getTime() creates a date by calculating the number of milliseconds since the "epoch".
    On the other hand, you can assign a date value to a calendar whose time zone may be different from the original time zone that created the date.
    For example, you created hour 1 in EST(GMT-5), then you assign this time to a calendar whose time zone is CST(GMT-6). The assigned calendar will returns a date whose hour is 0.
    We can also infer that in order to return the same hour, the 2 time zones assigned to the 2 calendars must be the same.

    Secondly, Java has different Calendar creation call syntax.
    They behave differently for different timezones on Mar 14,2010(or any other DST beginning dates) even they all refer to the same region.
    Take the New York region for example. Suppose our code runs in New York, the following 3 calls all return time zones in the New York region:
      Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("GMT-5:00")); //EST. No DST support      (1)
      Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("GMT-4:00")); //EDT. No DST support      (2)
      Calendar cal = Calendar.getInstance(); //default to "America/New_York". Supports DST  (3)

    With (1), you can set the calendar's hour to 1,2 and 3 etc, even hour 2 doesn't exist and it actually corresponds to hour 3 in EDT.
    With (2), you can also set the calendar's hour to 1,2 and 3 etc, even neither hour 1 or 2 exists and they actually correspond to hour 0 and 1 in EST, respectively.
    With (3), you can set the calender to all 24 hours except 2. Specifically hour 1 corresponds to hour 1 in EST and hour 3 and later correspond to hours in EDT. When you try to set hour 2, Java actually changes to hour 3 in EDT because call syntax (3) supports DST and hour 2 doesn't exist.
    (It is easy to understand if you just think the time zones in call syntax (1) and (2) are some first-class time zones and the time zone in (3) can be either GMT-5 or GMT-4 depending on the hour).

    Finally we connect the dots together and shows you why line (4) got hour 1. 
    When Hibernate uses the getTimestamp() without a calendar, the Oracle JDBC driver uses GMT-5 for hour 1 and GMT-4 for other hours to create a calendar and eventually returns a timestamp (I am not sure whether this is standard practice). So the hour 2 in the endDate is kept as hour 2 which actually corresponds to hour 1 in EST.
    Because Line (1) is the Calendar call syntax (3) which supports DST, line (4) returns the actually hour 1 in EST.

    Actually our application needs to return whatever hour MISO sent us without any DST offsetting. Based on the above analysis of Date and Calendar, line (1) must specify the same timezone as the one used to create the endDate by the JDBC driver. Unfortunately this is a guess game if you DB column doesn't have timezone information.

    There are 2 solutions.
    One is to design a DB column with time zone information.
    The other is to extend Hibernate's TimestampType by explicitly specifying a GMT timezone such as your local timezone based on the raw GMT offset (Such Timezones ignore DST schedules).

    Lastly the toString() from a Date object is based on your local timezone which may confuse you when your intended timezone is different.

    Wednesday, August 4, 2010

    hibernate.jdbc.batch_versioned_data can't be set to TRUE for Oracle JDBC driver

    Duo to Oracle's popularity, we originally assumed it should be safe to turn on this flag for Oracle JDBC drivers until we saw a Unit testing exception.
    Basically the Unit test tried to update a POJO in an optimistic way. It should have failed because the same POJO was just updated by another user before it. However Hibernate (3.3) just silently returned without resulting in any database update.
    I traced the code to Hibernate's method checkBatched() in class Expectations$BasicExpections. The rowCounts Oracle returned turns out to be always -2 (Statement.SUCCESS_NO_INFO) for all its version 9i,10g and 11g JDBC drivers.

    This returned value "-2" was finally verified by Oracle's JDBC Developer's Guide titled "Update Counts in the Oracle Implementation of Standard Batching" in Chapter 23 "Performance Extensions".
    Basically it says:
    • For a prepared statement batch, it is not possible to know the number of rows affected in the database by each individual statement in the batch. Therefore, all array elements have a value of -2. According to the JDBC 2.0 specification, a value of -2 indicates that the operation was successful but the number of rows affected is unknown.
    • For a generic statement batch, the array contains the actual update counts indicating the number of rows affected by each operation. The actual update counts can be provided only in the case of generic statements in the Oracle implementation of standard batching.
    • For a callable statement batch, the server always returns the value 1 as the update count, irrespective of the number rows affected by each operation.
    The different returned values for different statements can be explained by the following Oracle implementation details:

    In Oracle JDBC applications, update batching is intended for use with prepared statements that are being processed repeatedly with different sets of bind values.
    The Oracle implementation of standard update batching does not implement true batching for generic statements and callable statements. Even though Oracle JDBC supports the use of standard batching for Statement and CallableStatement objects, you are unlikely to see performance improvement.

    But I still have a hard time to believe that the back-end powerful Oracle database even couldn't know  the number of rows affected by each preparedStatement in a batch.

    I am equally disappointed with the way Hibernate is handling Statement.SUCCESS_NO_INFO(-2).
    This is how it happens: Hibernate uses preparedStatement for batch updates. Because the where clause in the update sql used a staled version number, Oracle just didn't update anything and returned successfully. Finally Hibernate still interprets Statement.SUCCESS_NO_INFO(-2) just as an successful update instead of throwing any optimistic exception. As you know this is not acceptable.

    I know Hibernate is in a dilemma in this case because when Oracle returns Statement.SUCCESS_NO_INFO(-2) it either means some rows were updated or no row was udpated.
    But I still like Hibernate to throw some exception to remind users of the updating ambiguity.

    The good news is batch inserting is still safe as because returning Statement.SUCCESS_NO_INFO(-2) must mean a row was successfully inserted into the database.
    So you may have to create a separate datasource for batch inserting only.