Commercial-Off-The-Shell Enterprise Real Time Computing (COTS ERTC) -- Part 2: Hardware requirements

Because hardware is at the bottom of a COTS ERTC application stack, we usually focus on throughput instead of latency in order to reduce higher level latency based on our discussion in Section 1.2 in part 1. But we still want to reduce hardware jitter.
Because hardware contributes the least to your COTS ERTC application's latency and jitter compared to other parts, you usually put less effort into hardware for your application jitter optimization.

In this part, I will talk about the latency incurred by computer hardware. I will also talk about how to reduce latency and jitters using such high-performance co-processors as GPU and FPGA.

1. Latency by Computer Hardware
1.1 Interrupt Latency
System timer, power management and such IOs as keyboard, mouse, disk and NIC use interrupts to notify the processor of the need for attention.
Software can also generate an interrupt to notify the processor of the need for a change in execution.

All modern hardware and OS use interrupts to implement thread preemption as shown in Figure 2.

                                            Figure 2 Thread Preemption Through Interrupt

Whenever there is a higher level interrupt, the OS scheduler preempts the low priority thread with the high priority thread. This preemption takes time i.e. "Interval Response Time" in Figure 2. This white paper shows a typical 25us interval response time on modern hardware.
RTC needs to bound this "Interval Response Time" which consists of several components as shown in Figure 2. Only "Interrupt Latency" relates to hardware. All others relate to device driver and OS and will be covered in the next topic.

Interrupt latency is the time required for the interrupt to propagate through the hardware from its source to the processor's interrupt pin.Because it is hardware line level latency, it is usually negligible compared to other components on modern hardware platforms.

Because most interrupts come from IO devices which usually have small time-sensitive data buffer, Interrupts have the highest scheduling priorities meaning they can interrupt any application and OS kernel threads including your RT ones. This has two implications.On one hand for a high priority event such as market data feed from an external source, its interrupt of all low priority threads and switching to the corresponding RT thread is really what you want.
On the other hand your really don't want your RT thread to be interrupted by such unnecessary IOs as keyboard or mouse. But your RT thread was still interrupted by such IOs.
However if the interrupt latency (more precisely, the interval response time) is bounded, your RT application thread behavior in both scenarios is still predicable.

As mentioned in the previous topic, COTS processors are usually designed for high throughput at the cost of high interrupt latency.
However this "high interrupt latency" is only significant for tight HRT. For SRT,
it is negligible compared to larger latencies by other parts in the stack and also thanks to the constant performance improvement in modern computer hardware.

1.2 System Management Interrupt (SMI)
Only X86 processors have those SMI's. When they happens, they suspend the CPU's normal execution and switch it into system management mode (SMM).
Because they usually run inside BIOS firmware, they are transparent to OS and you can't intercept them.  They are the highest priority interrupt in the system even higher than the NMI. Because they can last for hundreds of microseconds, they can cause unacceptable jitter for RTC, especially for HRT.

However they are not that bad if you OS supports ACPI because ACPI took the ownership of power management from SMM. Both Windows and Linux support ACPI (so you can configure your RT application thread higher than the OS ACPI thread).

This resource also lists other measures to mitigate SMI's impact.

IBM also optimized its LS21(Model 7971) and HS21 XM (Model 7995) to reduce SMI jitter for non-fatal SMI by deferring non-fatal SMI's functions to low-priority OS threads (When such a fatal SMI as a memory or chipset error happens, its function can't be deferred because the function has to be used to fix the error).

1.3 DMA Bus Mastering
This happens on the currently pervasive PCI bus. When a device uses the DMA, other devices that want to use the DMA have to wait until the previous device is done, which can cause many micro-seconds jitter based on this resource.

1.4 Max Performance vs Max Power Saving
ACPI can put CPU and other devices into some low-power-consumption or even power-off state after a period of inactivity, it will cause jitter when they have to be woken up.

ACPI support at least two extreme power schemes.
One is "Max Performance" which keeps CPU and other devices active all the time.
The other is "Max Power Saving" which power off CPU and other devices.
The "Max Performance" is preferred for RTC.

1.5 High Resolution Timer
The default system timer usually has a resolution of 10ms on most platforms. Although you can usually lower it to 1ms by configuring your OS, a high-resolution system timer can result in too many interrupt overheads which will severally lower throughput.

However even 1ms is still too coarse for RTC because RTC needs high resolution timer for its periodic or one-time task accurate scheduling and nano-sleep function (after all RTC applications have time or deadline constraints). At least microsecond resolution is required for RTC to keep jitter lower.
A hardware-based timer is needed also because it can generate interrupts as needed even you need a nano-second resolution e.g. the "dynamic tick" or even tickless implementation on Linux RT.

UltraSPARC systems can provide nanosecond timers while most modern X86 systems can only provide as good as microsecond times.

1.6 Cache
Because processors are orders of magnitude faster than the main memory e.g. modern processors have nanosecond execution time per instruction while main memory has microsecond access latency, caches are used to bridge the access latency gap. However this unavoidably creates jitter.

For example if your RTC application can completely fit into L1 cache or L2 cache, its latency will be very low and predicable. Otherwise the main memory access will cause jitter. But this is probably still acceptable to SRT or loose HRT based on your business rules.

1.7 Instruction Level Parallelism
Modern processors use such instruction level parallelism as out-of-order execution, pipeline and supercalar to improve your thread (CPU) throughput. However this again unavoidably create unpredictable temporal behavior for your thread instructions. 
But because they are at instruction level, they are probably still acceptable to SRT or loose HRT based on your business rules.

1.8  Multi-Processor / Core
Because higher processor clock rate leads to lower latency, the traditional approach of increasing processor performance is increasing clock rate based on Moore's Law. However such an approach eventually ran into power consumption and expensive cooling issues.
Multi-processor systems are one solution that can definitely improve throughput. But the interconnect between processors unavoidably introduce additional latency and jitter.
Multi-core processors may be a even better solution for RTC that can achieve both high throughput and low latency thanks to the shared high-speed bus among cores and the shared memory model.

A very important application of multi-processor / core in RTC is the so called "CPU shielding" (it also has few other names such as cpu binding or interrupt binding or thread binding or fine-grained processor control). "CPU shielding" is implemented in OS.

Here are some examples.
You can bind low priority interrupts to one CPU and dedicate another CPU to your RTC thread.
If you have multiple RTC threads that need to run simultaneously, you also have to have CPUs.
In case of Java, you need to bind at least one CPU to the concurrent GC or RTGC or the background JIT compiler so that it can truly run concurrently with your RT threads.

1.9 NUMA
The multiple CPUs mentioned in 1.8 usually have equal access to the shared main memory. This unfortunately also causes contentions when several processors attempt to address the same memory besides the benefits.

NUMA attempts to address this problem by providing separate memory for each processor. For problems involving spread data (common for servers and similar applications), NUMA can improve the performance over a single shared memory by a factor of roughly the number of processors (or separate memory banks).
However if not all data ends up confined to a single task, which means that more than one processor may require the same data, NUMA has to move data between memory banks, which causes jitter.


Both current X86 and UltraSPARC processors can provide good functions in the above areas. They can be used as the underlying platforms for COTS ERTC applications.

2. Co-Processors
The shift to multi-processor / core processors forces application developers to adopt a parallel programming model to exploit CPU performance, which proves to be very error prone.
Traditionally HPC uses clusters consisting of COTS multi-core servers to run data and compute intensive applications. In order to cut down such complex computation to RTC level, hundreds, even thousands of COTS multi-core servers along with high-bandwidth interconnect need to be deployed, which not only creates maintenance headache, but also consumes quite a lot of power. Even the interconnect usually has low latency, it still incurs latency.

CPU-based systems augmented with hardware accelerators as co-processors are emerging as a even better solution to the Moore's Law dilemma. This has opened up opportunities for accelerators like Graphics Processing Units (GPUs), FPGAs, and other accelerator technologies to advance HPC to previously unattainable RTC levels.


2.1 General-Purpose Computing on Graphics Processing Units (GPGPU)
Because traditional CPU design focuses on general purpose (both high volume and low volume; both management task and ALU processing etc), the number of cores and the vector size in SIMD are both small.
Because GPU's specialization in 2D or 3D graphics rendering acceleration using its highly pipeline parallel structures, it can have hundreds of  processor cores each of which can handle hundreds of independent threads. Its SIMD's vector is much longer than CPU's. Also the internal interconnect among cores has much higher bandwidth than the external interconnect in traditional clusters.

GPGPU is the technique of using a GPU to perform computation in applications traditionally handled by the CPU, which is made possible by the addition of programmable pipeline stages - shaders and higher precision arithmetic to the rendering pipelines, which allows software developers to use stream processing on non-graphics data.

A modern GPGPU itself is a cluster of hundreds of cores capable of handling tens of thousands of threads, which can be hundreds of times faster than a transitional cluster made of hundreds of processors.

OpenCL is an GPU programming framework that is supposed by all major GPU vendors. It supports both task and data level parallelism. For data level parallelism, users only need to partition data properly and are not responsible to manage threads, which is much less error-prone compared to the multi-core parallel programming.

GPU interfaces with a computer using PCIe, which can cause latency problem for high-volume and data intensive computing. This is why GPGPU recommend to use more threads to hide latency.

2.2 Field-programmable gate array (FPGA)
FPGA is an integrated circuit designed to be configured by the customer or designer after manufacturing — hence "field-programmable".
Because it uses highly paralleled hardware to implement your application logic traditionally implemented in software along with CPU or GPU,  it provides line speed latency, which is even shorter than GPU.
The FPGA architecture provides the flexibility to create a massive array of application-specific ALUs that enable both instruction and data-level parallelism.
Because data flows between operators, there are no inefficiencies like processor cache misses; FPGA data can be streamed between operators.

FPGA interfaces with a computer using either PCIe or processor bus such as Intel's FSB or QPI or AMD's HyperTransport. The later option makes FPGA just like another processor, which doesn't have cache coherency issue and also enjoys high bandwidth and low latency.

Although the C-to-FGPA compilation toolkit from Implulse enables a developer to use C instead of HDL to design application logic, the learning curve is still high. The developer still needs to know some basic hardware design knowledge and also compilation parallel skills such as loop unrolling and instruction pipeline.

Commercial-Off-The-Shell Enterprise Real Time Computing (COTS ERTC) -- Part 1: Introduction

This five-part series covers how to use real time computing (RTC) to process enterprise workloads using commercial off the shell (COTS) hardware and software based on my experience and researches.
By "enterprise workloads" and "commercial off the shell", I mean the discussion is not about the traditional RTC tailored to a few specific application tasks for bare metal or embedded devices. So you know how COTS ERTC came out.

An enterprise application stack typically consists of, counting from the bottom, the underlying hardware (micro-processors and / or co-processors) and OS platforms,  the network,  the program language (I will focus on Java) and the application itself.
Here are the five parts for this series:

• Part 1: Introduction
• Part 2: Hardware (Microprocessor and Coprocessor) requirements
• Part 3: Operating System (OS) requirements 
• Part 4: Network requirements 
• Part 5: Java requirements

In this introduction, I will clarify the following importance concepts: throughput vs latency; hard real time (HRT) vs soft real time (SRT); proprietary vs COTS.

1. Throughput and Latency
An application's performance can have quite a few metrics. However throughput and latency (or response time) are the two most important aspects for most enterprise workloads and for most developers.
Throughput is the amount of useful work accomplished over a period of time.
Latency is the time needed to accomplish some amount of useful work.

In my above definitions, I use such fuzzy words as "a period" and "some amount". You should quantify them based on your business logic. They are mathematically inverse to each other.

By "useful work", it means there are also overheads that can't be avoided completely during the period. Usually the more interruptions such as sending users feedback during the period, the more overheads. So in order to have a better throughput, the application needs to have few interruptions or longer latency.
On the other hard, in order to have lower latency, you have to process less amount of work before sending out a response, which means lower throughput. Be careful not to lower the amount of work too much otherwise you may well end up nothing but too frequently sending out responses, which is practically not doing any useful work.
The relationship between throughput and latency can also be shown in Figure 1 where you can see lower latency (more overhead) means lower throughput.

Figure 1 useful work and overhead over a period

1.1 Is it possible to improve both throughput and lower latency at the same time?
It depends.
It is not possible if you don't put more resources into your application.
It is very achievable if you put more resources into your application such as upgrading your hardware or deploying more hardware or employing more efficient algorithms.

Here is a theoretical example.
Suppose your application needs to process 1,000,000 transactions. The user needs to get a feedback after some number of transactions have been processed. Each transaction processing takes 1ms and each feedback incurs 10ms overhead.
You are asked to calculate the application's throughput over a period of 1s.
Basically you have two configurations.
(1)The user doesn't need low latency or fast response
This needs your application to send out feedback every a larger number of transactions. If the batch number is 100, you can get the following result by simple calculation:
  throughput: 909 transactions
  latency: 100ms

(2)The user needs to have low latency or fast response.
This needs your application to send out feedback every a smaller number of transactions. If the batch number is 10, you can get the following result by simple calculation:
  throughput: 500 transactions
  latency: 10ms

From the above numbers, you know lower latency means lower throughput and this becomes more obvious when the per-feedback overhead becomes larger and larger.
How do you keep the 10ms latency in (2) while still enjoying the 909 transaction throughput in (1)? You have to make your per-transaction processing more efficient and / or lower the per-feedback overhead.
Suppose you can't lower the per-feedback overhead but you do know a better algorithm or be able to upgrade your hardware to cut down the per-transaction processing time. By simple calculation, you know you per-transaction processing time has to be 0.55ms or better instead of the original 1ms.

Here is a real example about JVM's garbage collection (GC).
In this case your application threads are doing useful work while the GC just incurs overhead. The typical Stop-The-World (STW) GC causes application pause time which is the main contributor to latency. The more memory to collect, the more pause time, the lower throughput.
In order to reduce pause time, the GC has to collect memory more frequently or concurrently, which unfortunately incurs more overheads such as thread context switching and bookkeeping.
If you want very low pause time, the GC may have to collect memory way too frequently, which is practically equal to STW.


In order to have both good throughput and latency, modern JVMs employ more advanced GC algorithms such as parallel GC, concurrent GC and dynamic adjustment based on adaptive algorithms and also require multi-core processors.

1.2 End user level latency relies on underlying throughput
In the above first example, if the user defines latency as the time taken to process all 1,000,000 transactions, what strategy will the underlying system employ to get a better latency?
Obviously the underlying system can process 1,000,000 transaction faster using throughput than latency (the throughput and latency here are of the underlying system, not the end user level).

In the above second example, the latency consists of your application thread time plus the GC pause time in the application window. Let's focus on the GC pause time. Obviously in order to minimize the GC pause time, the GC should use throughput collectors such as STW parallel collector instead of latency collectors such as concurrent Mark Sweep collector.

This last point is very important because it help you understand the following points:

• Improving infrastructure level throughput can not only improve overall system throughput but also provide users with low latency;
• Higher level low latency doesn't means the underlying systems must also provide low latency. This is why people focus more on High Performance Computing (HPC) than on RTC (see Section 3 for details);
• You should also not be surprised when you hear a vendor boasting of his product capable of low latency and high throughput.

2. Hard Real Time (HRT) and Soft Real Time (SRT)
The widely recognized RTC definition comes from Donald Gillies in RTC FAQ: 

"A real-time system is one in which the correctness of the computations not 

only depends upon the logical correctness of the computation but also upon 

the time at which the result is produced. If the timing constraints of the  

system are not met, system failure is said to have occurred."

In other words, RTC must be deterministic to guarantee it performs within the required time frame or deadline. It is not about throughput, it is about latency. It is not about how fast the response is, it is about its latency being predicable.
No system can provide absolutely constant latency. The variance in latency is called Jitter.

Practically RTC can be classified as either HRT or SRT:
An HRT system must meet all its deadlines or its jitter is always bounded; otherwise it has catastrophic consequences.
An SRT system can still function correctly if it misses its deadlines occasionally or its jitter sometimes overshoot the boundary. The occasional misses usually happen in worst case scenarios.

2.1 Predictability or determinism is subjective and driven by your business requirements
It really depends on your business requirements how to setup a boundary, how tight the boundary is and whether the boundary can be broken occasionally.

For example, suppose your mean latency is 10ms and the jitter is 5ms. If you boundary from 5ms to 15ms is acceptable, you get an HRT system.
However if the above boundary is not acceptable and a new tighter boundary such as from 8ms to 12ms is required, you can still get an SRT system if your business really don't need HRT and you can predict the jitter only occasionally overshoots the boundary.

2.2 It still matters how fast your response is
Although RTC is not about how fast the response is, RTC systems usually have low latency and are very fast. Actually lower latency also makes your system more predicable because the lower latency usually means less jitter.

For example, if the options pricing in financial services takes several hours, even days, nobody will think it is real-time pricing and it has to be used as offline pricing.
Also probably nobody in the financial services industry would think a 1s latency is real-time and wants to use it in his/her high frequency trading.
However on the other hand, very low latency is much harder and more expensive to achieve or even impossible without using proprietary hardware and software.

Hereafter we will use "tight HRT" to mean HRT with very low latency requirement and "loose HRT" to mean HRT with lenient latency requirement.
Although they are again very fuzzy and depend on your business requirements, they will not affect our discussion.

2.3 RTC is not for everyone
Based on the above analysis in section 1, we know RTC achieves its low latency at the cost of poor throughput or requires more resources to achieve both lower latency and high throughput. So RTC is just not for everyone and should be targeted at those applications that need it in your enterprise. Also if SRT can meet your needs, don't use HRT because HRT has much higher requirements on all parts in the application stack than SRT does.

2.4 RTC is still strange to most developers.
Because RTC can trace its root from embedded devices in the tel-comm, auto and other control industries, most enterprise developers are not familiar with it.
Another reason is most hardware and OS platforms and application designs deployed in enterprises focus on throughput. Their algorithms are optimized for average cases instead of the worst cases.  But HRT needs to reduce jitter in wost cases so that a predicable boundary can be defined.
Recalling 1.2, you should further understand why HRT is much harder to implement than SRT.

3. Proprietary and COTS
Most developers heard of RTC from such RTOS as QNX and VxWorks. They are only used in embedded systems which have little memory, limited external connection, are tailored to a few specific tasks, and are limited to a small user base and fewer vendors. So they are proprietary niche products not good for enterprise applications.
But those traditional RTC systems really shine and are unbeatable when there is a need for very predicable timing behavior or HRT. Actually theoretically only HRT is RTC.

In the past several decades, both hardware and software have experienced rapid advancements in both functionality and performance, and many have been commoditized. Such COTS systems as X86 microprocessors, GPGPU / FPGA, Linux, Windows, Java and C# have become more powerful both on throughput and latency while still being cheap and easy to use.
Unfortunately those general purpose COTS products are traditionally designed with throughput in mind. However due to their huge deployments, large user base and wide industry support, RTC on top of COTS or COTS ERTC is very attractive.
The most important signature of a COTS ERTC system is RTC functions are added without changing existing COTS system's functions so that a COTS ERTC system can support both traditional and RTC enterprise workloads.

A very bright spot in COTS ERTC is to implement an SRT system. Compared to traditional proprietary RTC or HRT, It is easy to build and can handle most of ERTC workloads while still maintaining good throughput. Such an SRT system is so appealing that it should be the first choice for an enterprise.

Another bright spot is High Performance Computing (HPC) is often employed to make high-volume computing tasks qualified as RTC based on COTS ERTC clusters or GPU/FGPA (see the above Section 1.2). 

Using traditional single processor / core, such complex tasks as options valuation and risk analytics in financial services often take many hours even days. HPC can cut down the computation time to sub-second or even less on average, which basically qualify them as SRT.

When vendors say they can provide financial services industries with real-time products, they really usually mean SRT based on COTS ERTC instead of HRT.
Here are more examples:

• "best execution" and transparency regulated in the US's RegNMS and European's MiFID.
  RTC can definitely reduce inconsistent trading times that can result in server fines.
• FTSE 100 Index which is calculated in real time and published every sub-second. (Remember not long ago it was 15s, which is not RTC due to long latency)
• Reuters Data Feed Direct (RDF-D) is an ultra-low latency real-time feed handler that has an average latency (so SRT) of less than one millisecond during periods of high traffic.
• Algorithmic Trading, especially High Frequency Trading (HFT) needs to execute a trade in low milliseconds or even sub-millisecond in order to beat competitors

Because RTC achieves low latency at the cost of throughput, it is very import for those COTS ERTC to keep the original functionality while providing additional RTC. After all RTC is not for everyone and an enterprise probably needs both RTC and non-RTC functionality to co-exist.
Fortunately such COTS ERTC building components are commercially available.

4. Prioritized Approach
The overall application latency is the cumulative delays of all parts in the application stack. If any part creates jitter, your application will experience jitter.
So in order to develop a COTS ERTC system, all parts in the application stack should ideally support RTC. However you should prioritize latency and jitter analyses.
For example if your Java GC has a 500ms jitter while the OS scheduling has a much smaller 5µs jitter, the GC jitter tuning should take precedence because any scheduling jitter improvement is negligible.

Since the bottom parts usually have lower latency and create less jitter than the top ones, the top parts usually have higher requirements on latency and jitter than the bottom parts, especially for HRT.
For example if you just need SRT, you can directly tackle your Java and application logic without putting too much effort on modern fast hardware or OS.

The following parts in this series will analyze latency and jitters for all parts in the application stack and find ways to reduce them.

A New Step for My Wife

Today is the first day that my wife go to New-York Presbyterian Hospital. I really don't need to say anything about the high reputation of this prestigious hospital exception her No 6 overall ranking in the nation by U.S. News and World Report in its latest America's Best Hospitals Report.

My wife will receive a three year program one of which are researches and the rest of which are clinical trainings. She will be a medical physicist after three years.
If things are going well, she will have a great chance to leave in that hospital.

TimesTen -- an Oracle Database Saviour for Performance-Critical Applications?

Performance-critical applications such as those used in trading and telecoms often need low latency in sub-milliseconds or even microseconds and high throughput.
If such an application uses a disk-based database for persistence (or the system of record), you probably will have to miss your low latency requirement because disk-based accesses incur latency in many milliseconds and often seconds.

In order to achieve low latency and high throughput, there should be at least 3 requirements to be met:

1. Uses memory-based accesses. The access rate for the commonly wildly used computer DRAM is in microseconds;
2. Collocates your application logic with your data. The traditional multi-tiered approaches incurs network IO latency.
3. A fast and predictable JIT compiler if you use Java. Oracle's JRockit is a good choice.

I know 2 products which can meet the above requirements and worth a try.
One is to adopt Gigaspaces' space-based programming paradigm.
The other is to employ an in-memory database (IMDB) such as Oracle's TimesTen along with your traditional database such as Oracle.

I will focus on TimesTen here and compare it with Gigaspaces.
As mentioned above, using Oracle database alone in a performance-critical application is not practical. Accordingly Oracle just doesn't have the same popularity in such applications as in others.
Fortunately if we use Oracle along with its cache companion - TimesTen, it is a very attractive option.

TimesTen is an in-memory relational database from Oracle. Because it caches all data in memory and has no disk access, TimesTen can achieve breakthrough performance such as 5-microsecond reads and 15-microsecond updates based on its white paper "Using Oracle In-Memory Database Cache to Accelerate the Oracle Database".
TimesTen is targeted to run in your application tier, close to applications, and optionally in process with applications. A TimesTen database may be used as the database of record, and/or as a cache to an
Oracle database.

As a cache to Oracle database, TimesTen can be embedded into your application tier and only cache the performance-critical subset of your Oracle database while still leaving your Oracle database for non-performance-critical applications. This architecture is shown in Figure 3 in the above white paper:

In the above figure, you can only cache the subset of an Oracle database that needs real-time processing such as stock trading data. The cached data can be either read-only, read-mostly or read-write. In case of data modification, the "Cache Agent" will take care of the data synchronization between TimeTen and Oracle.
You can also scale out your application tier horizontally by deploying several TimeTimes together to form a so called cache grid.

Here are my TimesTen comparisons to Gigaspaces.
Commons to Both:

• cache data along with your application logic;
• supports multiple topologies such as partitioned, replicated or both;
• supports ACID properties often provided by traditional relational databases; 
• supports asynchronous operations for better performance such as transaction write-behind and asynchronous replication;
• besides as a cache bus, they can also be used as a message bus.

Pros and Cons on TimesTen and Gigaspaces:

• Gigaspaces can cache all your data in a computing cloud such as Amazon's EC2 while TimesTen only practically caches subset of your whole enterprise data;
• Gigaspaces has more flexible deployment topologies than TimesTen. For example, Gigaspaces can adjust your deployment based on your SLA while TimesTen's deployment tends to be static;
• Gigaspaces supports application level optimistic locking while TimeTen only provides 2 traditional database's isolation levels - read-commit and serializable;
• Gigaspaces intrinsically supports parallel processing such as map-reduce paradigm through its master-worker pattern and thread executor framework while using TimesTen you have to build up such functions by yourself.  For example, if you want to do a search in your TimesTen cache grid (suppose you partitioned your data in the grid), you have to code by yourself to submit tasks to grid memebers and finally aggregate the each individual searching result;
  
• It is easier to adopt TimesTen than Gigaspaces because TimesTen intrinsically supports the popular JDBC API while Gigaspaces's intrinsic JavaSpaces API is still not that popular; 
• TimesTen is budget and deployment friendly because it only needs to cache a subset of your whole enterprise data. Most often a large amount of your enterprise data don't need real-time accesses and accordingly caching them may be wasting your valuable resources.
• TimesTen has closer integration with Oracle than Gigaspaces does. For example, TimesTen supports the same flexible locks as in Oracle and materialized views;
• TimesTen seems to have a bit better performance. For example, TimesTen can achieve 5-microsecond reads and 15-microsecond updates while Gigaspaces can achieve sub-millisecond latency at best. This may be because TimesTen was implemented by C/C++ and Gigaspaces was implemented Java (JVM has pause).

    

Daylight Saving Time (DST) and Timezone Handling in Java, JDBC and Oracle

The following problem has been puzzling me for 2 days.
Because MISO (Midwest ISO. A power market for several Mid West regions) doesn't support DST, it sent us data at hour 2 on Mar 14,2010 which is the DST beginning date (the hour 2 is supposed to be skipped if the ISO supports DST).

The date values in question are stored in an Oracle column called endDate which is of Oracle's date type without time zone information. In other words, you interpret such a date type's components (year,month,day,hour,minute,second and millisecond) in your local timezone.
If your data are across timezone, you should use Oracle's "timestamp with timezone" or "timestamp with local timezone". Such a date type has an additional timezone component based on which you interpret other components.

This additional timezone component is the key to understand the difference between Oracle's date type and Java's date type.
Because Java's date type represents the specified number of milliseconds since the standard base time known as "the epoch", namely January 1, 1970, 00:00:00 GMT. (Don't miss this GMT timezone!)
When you try to load an Oracle date value without a timezone into a Java date object, you will face difficulty without using any timezone in Java.

We used Hibernate's TimestampType to map this DB column to a Java Date type.
I used the following code to extract the hour in the endDate:
    Calendar cal = Calendar.getInstance();    //(1)

  int he;                                   //(2)
       
  cal.setTime(getEndDate());                //(3)
  he = cal.get(Calendar.HOUR_OF_DAY);       //(4)

The code is running in east coast which is currently in DST (it is August 2010). The hour in line (4) returns 1 for the hour 2 in question and the getEndDate()'s toString() also shows hour 1 instead of hour 2 or hour 3.

Getting to know why this happened is quite involving and confusing mainly due to the DST switch. We explain it in three steps.

First we need to know how Hibernate and JDBC driver retrieves you endDate value to a Java date object.
Hibernate's TimestampType just calls the following ResultSet's method to get the endDate.

   java.sql.Timestamp getTimestamp(String columnLabel);  

Because our DB column doesn't have timezone, what timezone will be used in the returned Timestamp (it extends date type) in the above method (still remember my previous statement "Java's date type represents the specified number of milliseconds since the standard base time known as "the epoch", namely January 1, 1970, 00:00:00 GMT. ")?
Astute readers may recall there is any similar method in ResultSet that allows you to provide a timezone through a Calendar:

   java.sql.Timestamp getTimestamp(int columnIndex, Calendar cal) ;

This method is exactly to handle a DB column without a timezone so that you can interpret the DB date times in a specified timezone based on your business logic.

The Java document for the first method doesn't say what time zone is associated with the returned timestamp value. This is unfortunate and different JDBC drivers may do different things. Later I will present what I found based on my testings.
Actually Java's date doesn't care about any time zone per se; it only remembers the the number of milliseconds since the "epoch". It is Java's calendar that incorporates a timezone (and also a locale).
Remember the preferred way to create a date object is to create a calendar first then call its getTime() to return the date. Calendar's getTime() creates a date by calculating the number of milliseconds since the "epoch".
On the other hand, you can assign a date value to a calendar whose time zone may be different from the original time zone that created the date.
For example, you created hour 1 in EST(GMT-5), then you assign this time to a calendar whose time zone is CST(GMT-6). The assigned calendar will returns a date whose hour is 0.
We can also infer that in order to return the same hour, the 2 time zones assigned to the 2 calendars must be the same.

Secondly, Java has different Calendar creation call syntax.
They behave differently for different timezones on Mar 14,2010(or any other DST beginning dates) even they all refer to the same region.
Take the New York region for example. Suppose our code runs in New York, the following 3 calls all return time zones in the New York region:
  Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("GMT-5:00")); //EST. No DST support      (1)
  Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("GMT-4:00")); //EDT. No DST support      (2)
  Calendar cal = Calendar.getInstance(); //default to "America/New_York". Supports DST  (3)

With (1), you can set the calendar's hour to 1,2 and 3 etc, even hour 2 doesn't exist and it actually corresponds to hour 3 in EDT.
With (2), you can also set the calendar's hour to 1,2 and 3 etc, even neither hour 1 or 2 exists and they actually correspond to hour 0 and 1 in EST, respectively.
With (3), you can set the calender to all 24 hours except 2. Specifically hour 1 corresponds to hour 1 in EST and hour 3 and later correspond to hours in EDT. When you try to set hour 2, Java actually changes to hour 3 in EDT because call syntax (3) supports DST and hour 2 doesn't exist.
(It is easy to understand if you just think the time zones in call syntax (1) and (2) are some first-class time zones and the time zone in (3) can be either GMT-5 or GMT-4 depending on the hour).

Finally we connect the dots together and shows you why line (4) got hour 1. 
When Hibernate uses the getTimestamp() without a calendar, the Oracle JDBC driver uses GMT-5 for hour 1 and GMT-4 for other hours to create a calendar and eventually returns a timestamp (I am not sure whether this is standard practice). So the hour 2 in the endDate is kept as hour 2 which actually corresponds to hour 1 in EST.
Because Line (1) is the Calendar call syntax (3) which supports DST, line (4) returns the actually hour 1 in EST.

Actually our application needs to return whatever hour MISO sent us without any DST offsetting. Based on the above analysis of Date and Calendar, line (1) must specify the same timezone as the one used to create the endDate by the JDBC driver. Unfortunately this is a guess game if you DB column doesn't have timezone information.

There are 2 solutions.
One is to design a DB column with time zone information.
The other is to extend Hibernate's TimestampType by explicitly specifying a GMT timezone such as your local timezone based on the raw GMT offset (Such Timezones ignore DST schedules).

Lastly the toString() from a Date object is based on your local timezone which may confuse you when your intended timezone is different.