Showing posts with label JDBC. Show all posts
Showing posts with label JDBC. Show all posts

Thursday, October 14, 2010

Asynchronous (non-blocking) Execution in JDBC, Hibernate or Spring?

There is no so called asynchronous execution support in JDBC mainly because you want to wait for the result of your DML or DDL most of the time or because there is too much complexity involved between the back-end database and the front end JDBC driver. 
Some database vendors do provide such support in their native drives. For example Oracle supports non-blocking calls in its native OCI driver. Unfortunately it is based on polling instead of callback or interrupt.
Neither Hibernate or Spring supports this feature.

But sometimes you do need such a feature. For example some business logic is still implemented using legacy Oracle PL/SQL stored procedures and they run pretty long. The front-end UI doesn't want to wait for its finish and it just needs to check the running result later in a database logging table into which the store procedure will write the execution status.
In other cases your front-end application really cares about low latency and doesn't care too much about how individual DML is executed. So you just fire a DML into the database and forget the running status.

Nothing can stop you from making asynchronous DB calls using multi-threading in your application. (Actually even Oracle recommends to use multi-thread instead of polling OCI for efficiency).
However you must think about how to handle transaction and connection (or Hibernate Session) in threads.
Before continuing, let's assume we are only handling local transaction instead of JTA.

1. JDBC
It is straightforward. You just create another thread (DB thread hereafter) from the calling thread to make the actual JDBC call.
If such a call is frequent, you call use ThreadPoolExecutor to reduce thread's creation and destroy overhead.

2. Hibernate
You usually use session context policy "thread" for Hibernate to automatically handle your session and transaction.
With this policy, you get one session and transaction per thread. When you commit the transaction, Hibernate automatically closes the session.
Again you need to create a DB thread for the actual stored procedure call.

Some developer may be wondering whether the new DB thread inherits its parent calling thread's session and transaction.
This is an important question. First of all, you usually don't want to share the same transaction between the calling thread and its spawned DB thread because you want to return immediately from the calling thread and if both threads share the same session and transaction, the calling thread can't commit the transaction and long running transaction should be avoided.
Secondly Hibernate's "thread" policy doesn't support such inheritance because if you look at Hibernate's corresponding ThreadLocalSessionContext, it is using ThreadLocal class instead of InheritableThreadLocal.

Here is a sample code in the DB thread:
// Non-managed environment and "thread" policy is in place
// gets a session first
Session sess = factory.getCurrentSession();
Transaction tx = null;
try {
  tx = sess.beginTransaction();

  // call the long running DB stored procedure

  //Hibernate automatically closes the session 
  tx.commit();
}
catch (RuntimeException e) {
  if (tx != null) tx.rollback();
  throw e;
}

3.Spring's Declarative Transaction

Let's suppose your stored procedure call is included in method:
  @Transactional(readOnly=false)
  public void callDBStoredProcedure();

The calling thread has the following method to call the above method asynchronously using Spring's TaskExecutor:
  @Transactional(readOnly=false)
  public void asynchCallDBStoredProcedure() {
        //creates a DB thread pool
        this.taskExecutor.execute(new Runnable() {
            @Override
            public void run() {
                //call callDBStoredProcedure()
            }
        });
  }

You usually configure Spring's HibernateTransactionManager and the default proxy mode (aspectj is another mode) for declarative transactions. This class binds a transaction and a Hibernate session to each thread and doesn't Inheritance either just like Hibernate's "thread" policy.

Where you put the above method callDBStoredProcedure() makes a huge difference.
If you put the method in the same class as the calling thread, the declared transaction for callDBStoredProcedure() doesn't take place because in the proxy mode only external or remote method calls coming in through the AOP proxy (an object created by the AOP framework in order to implement the transaction aspect. This object supports your calling thread's class by composing an instance of your calling thread class) will be intercepted. This meas that "self-invocation", i.e. a method within the target object (the composed instance of your calling thread class in the AOP proxy) calling some other method of the target object, won't lead to an actual transaction at runtime even if the invoked method is marked with @Transactional!

So you must put callDBStoredProcedure() in a different class as a Spring's bean so that the DB thread in method asynchCallDBStoredProcedure() can load that bean's AOP proxy and call callDBStoredProcedure() through that proxy. 

Wednesday, August 4, 2010

hibernate.jdbc.batch_versioned_data can't be set to TRUE for Oracle JDBC driver

Duo to Oracle's popularity, we originally assumed it should be safe to turn on this flag for Oracle JDBC drivers until we saw a Unit testing exception.
Basically the Unit test tried to update a POJO in an optimistic way. It should have failed because the same POJO was just updated by another user before it. However Hibernate (3.3) just silently returned without resulting in any database update.
I traced the code to Hibernate's method checkBatched() in class Expectations$BasicExpections. The rowCounts Oracle returned turns out to be always -2 (Statement.SUCCESS_NO_INFO) for all its version 9i,10g and 11g JDBC drivers.

This returned value "-2" was finally verified by Oracle's JDBC Developer's Guide titled "Update Counts in the Oracle Implementation of Standard Batching" in Chapter 23 "Performance Extensions".
Basically it says:
  • For a prepared statement batch, it is not possible to know the number of rows affected in the database by each individual statement in the batch. Therefore, all array elements have a value of -2. According to the JDBC 2.0 specification, a value of -2 indicates that the operation was successful but the number of rows affected is unknown.
  • For a generic statement batch, the array contains the actual update counts indicating the number of rows affected by each operation. The actual update counts can be provided only in the case of generic statements in the Oracle implementation of standard batching.
  • For a callable statement batch, the server always returns the value 1 as the update count, irrespective of the number rows affected by each operation.
The different returned values for different statements can be explained by the following Oracle implementation details:

In Oracle JDBC applications, update batching is intended for use with prepared statements that are being processed repeatedly with different sets of bind values.
The Oracle implementation of standard update batching does not implement true batching for generic statements and callable statements. Even though Oracle JDBC supports the use of standard batching for Statement and CallableStatement objects, you are unlikely to see performance improvement.

But I still have a hard time to believe that the back-end powerful Oracle database even couldn't know  the number of rows affected by each preparedStatement in a batch.

I am equally disappointed with the way Hibernate is handling Statement.SUCCESS_NO_INFO(-2).
This is how it happens: Hibernate uses preparedStatement for batch updates. Because the where clause in the update sql used a staled version number, Oracle just didn't update anything and returned successfully. Finally Hibernate still interprets Statement.SUCCESS_NO_INFO(-2) just as an successful update instead of throwing any optimistic exception. As you know this is not acceptable.

I know Hibernate is in a dilemma in this case because when Oracle returns Statement.SUCCESS_NO_INFO(-2) it either means some rows were updated or no row was udpated.
But I still like Hibernate to throw some exception to remind users of the updating ambiguity.

The good news is batch inserting is still safe as because returning Statement.SUCCESS_NO_INFO(-2) must mean a row was successfully inserted into the database.
So you may have to create a separate datasource for batch inserting only.