My Recent Interview Experience in the Financial Industry -- Computer Science Fundamentals

A base salary around $150k seems to be the watershed between a regular job that can be easily landed and a better paid job that needs a lot of efforts.
If you want to get better compensations, it is mandatory that you master computer science fundamentals. Most of financial firms I dealt with just didn't ask me any financial domain knowledge. Nor did they ask so-called populate technical skills such as Ajax, j2ee web development, Java Swing or JDBC etc. Instead, they just focused on fundamentals such as thread, socket, data structures and algorithms. Especially they wanted to make sure you are good at language-agnostic commonly-used data structures and algorithms.

I used to think that only hedge funds, prop trading firms, and high-profile technology firms such as Google and Amazon asked data structures and algorithms. But now it seems that other firms are just following suit.
For thread, it is not enough to only know how to create them. You also need to know thread safety, thread pooling, memory model etc.
For socket, it is not enough to only know TCP and UDP. You also need to know the Select paradigm and how to fine tune performance and latency.
For data structures, you should be very familiar with collection, set, list, queue, hash and tree.
For algorithms, you should know very well complexity analysis (the Big-O notation),  all common sorting, median and order statistics, iterative and recursive, divide and conquer, exhaustive, back trace and dynamic programming.

There is a very well-known book "Introduction to Algorithms" from the MIT press. If you was not touch this book in school, you need to learn it by yourself unless you don't want to work for good companies.

If you have worked for many years, you feel it is not quite fare to be asked so many algorithms because new graduates usually have fresh memory on algorithms than you. But, if you think you are a smart guy who is willing to accept challenges, you should welcome any interview questions.

Speaking challenges, many firms also ask you some brain teaser questions. Basically the interviews wanted to test your analysis skills and logic reasoning capacities. Since most developers don't often face such problems at work, they feel it more difficult to work on them than on regular programming issues.
The good news is many brain teasers are relatively easier if you go through some training. A good source is the "Brain Teaser" link under my "INTERVIEW WEB SITES" section on the left.

Floating-Point Number Equality Comparison in Oracle, Sybase / SQL Server, Java and JMS

As you know that computer can't store all floating-point numbers exactly primarily due to memory constraint. A double value such as 0.1 is actually represented as 0.1000000000000000055511151231257827021181583404541015625 in computer memory. Due to rounding errors, your equality comparison on floating-point numbers will probably fail.
In this topic I will show how SQL and Java store exact floating-point numbers (or just decimal numbers), and how floating-point literals are represented.

SQL99 uses FLOAT, REAL and DOUBLE PRECISION for approximate floating-point numbers, and NUMERIC(p,s) / DECIMAL (p,s) for exact floating-point numbers. For floating-point literals, SQL99 stores them as approximate floating-point types if they use the E notation (scientific notation); otherwise as the exact floating-point type if the literals meet the precision and scale of the exact types.
Because the NUMERIC(p,s) / DECIMAL (p,s) stores values without loss of precision for your decimal numbers, it should be used if you want to compare the equality between a floating-point column (in DB) or variable (in Java) to a decimal literal such as 0.1.
Different database vendors may have a bit different variants from the above specification.

Oracle 10g uses NUMBER(p,s) for both floating-point types in addition to its BINARY_FLOAT, BINARY_DOUBLE and FLOAT(p) for approximate types. Make sure you always specify the precision (p) for exact types.
If a floating-point is sufficed by f  or F (for BINARY_FLOAT), or d or D (for BINARY_DOUBLE), it is of approximate types; otherwise of the exact NUMBER(p,s) type.

Sybase ESA 15 and MS SQl Server 2008 both comply with the above SQL99 standard very well. And both also have MONEY(SMALLMONEY) for exact float-point numbers.

Java has BigDecimal to store decimal values exactly. For example new BigDecimal("0.1")  stores 0.1 exactly.

JMS's message selector is a string whose syntax is based on a subset of SQL92. Unfortunately JMS doesn't support exact floating-point numbers and restricts its exact numeric literal only to those without a decimal.

Key Concepts to Understand Spring's Consistent Transaction Handling

Class FooBarService shows simplified pseudo code for different transaction handling paradigms using JDBC,Hibernate and JTA.
Class FooBarServiceSpring shows simplified pseudo code for consistent transaction handling using Spring.

Obviously Spring's transaction handling is quite different from JDBC and Hibernate; but is closer to JTA. However, by no mean does Spring only support JTA. Instead, it is only a matter of configuring different PlatformTransactionManager (PTM hereafter) in your application context in order to support different transaction handling paradigms.  For example, Spring's DataSourceTransactionManager, HibernateTransactionManager and JTATransactionManager are for JDBC, Hibernate and JTA, respectively.

This discussion can help readers understand some internals of Spring's  PTM implementations on heterogeneous native transaction platforms.

//different transaction paradigms using JDBC,Hibernate and JTA

class FooBarService {

  //local transaction paradigm using JDBC

  public void foo() {

    //creates a session with the backend database

    Connection conn = dataSource.getConnection();

    //starts a new local transaction.

    conn.setAutoCommit(false);

    updates-data-through-this-conn

    //commits the local transaction

    conn.commit();

    //closes the session

    conn.close();

  }

  //local transaction paradigm using Hibernate

  public void bar() {

    //creates a session with the backend database

    Session session = sessionFactory.openSession();

    //starts a new local transaction.

    Transaction tx = session.beginTransaction();

    updates-data-through-this-session

    //commits the local transaction

    tx.commit();

    //closes the session

    session.close();

  }

  //Global transaction paradigm using JTA

  public void fooBar() {

    userTransaction.begin();

    //creates a session with the backend database1

    Connection conn1 = dataSource1.getConnection();

    updates-data-through-this-conn1

    //creates another session with the backend database2

    Connection conn2 = dataSource2.getConnection();

    updates-data-through-this-conn1

    //closes the two sessions

    conn1.close();

    conn2.close();

    userTransaction.commit();

  }

}

//consistent transaction handling paradigms using Spring

class FooBarServiceSpring {

  @Transactional

  public void foo() {
    //updates data through some data source connection

  }
 

  @Transactional(propagation=Propagation.REQUIRES_NEW)

  public void bar() {
    //updates data through some data source connection

  }

  @Transactional

  public void fooBar() {

    foo();

    bar();

  }

}

1. Key Transaction Abstraction

The following 3 interfaces are key to understand Spring's transaction handling:

• TransactionDefinition
  It allows you to define transaction requirements (isolation level, propagation behavior, timeout and read-only status) before your transaction is started.
  In FooBarServiceSpring, you specify your TransactionDefinition in @Transactional.
• TransactionStatus
  Once your transaction is started, this interface allows you to query its current status including rollback flag and new vs existing flag, and to mark it to rollback only (for example when you encounter an exception).
  Spring's default TransactionStatus implementation also includes the underlying native transaction such as a connection for JDBC, a session for Hibernate and a user transaction for JTA.
  In FooBarServiceSpring, @Transactional allows you to specify your rollback rules. For other status information, you either implicitly knows(such as IsNewTransactio and IsComplete) or can retrieve from the underlying native transaction bound to the thread (Spring binds JDBC connection and Hibernate Session to the thread; JTA's UserTransaction is also bound to a standard JNDI name). Please note that Spring doesn't bind TransactionStatus to the thread because PTM can pass it around in your AOP proxied methods.
• PlatformTransactionManager
  It wires together the above 2 interface and works as a coordinate overall.
  Specifically its getTransaction() method takes your TransactionDefinition and creates a TransactionStatus representing either a new or existing transaction.  Its commit() and rollback() methods take the returned TransactionStatus and commit and rollback the target transaction, respectively.
  It allows you to demarcate transactions as a singleton because it can get different transactions bound on different threads.
  In FooBarServiceSpring, @Transactional uses whatever transaction manager you configured in your application context such as DataSourceTransactionManager for JDBC.

When we say Spring transaction, we really mean TransactionDefinition and TransactionStatus. The PTM is more like a JTA UserTransaction / TransactionManager. But once again, it handles all types of native transaction API's behind the scene.

2. Physical Transaction and Logical Transaction

FooBarServiceSpring's foobar() calls foo() and bar(). Suppose there is no transaction before you call foobar(). So foobar() creates a new physical transaction while foo() starts a nested logical transaction due to its default propagation being REQUIRED; the logic transaction ends when foo() returns. 

But bar() suspends the current transaction and creates a new physical transaction due to its propagation being REQUIRED_NEW. The new transaction ends and the suspended transaction is resumed when bar() exits.

Because each (physical or logic) transaction always has a begin and end, we sometimes use transaction scope.

For data persistence, only the outer physical transaction can commit or rollback. But on each individual inner logic transaction (method) level, you always specify your TransactionStatus and can mark transaction to rollback . However the embed native transaction in TransactionStatus is either new (if it is a physical one) or existing (if it participates in an existing one).
You can also turn on the "validateExistingTransaction" flag in PTM so that the inner logic transaction will reject participation if its TransactionStatus is not compatible to the outer TransactionStatus.

3. Resource, Connection, Session, Transaction and Resource Manager (RM)

Resource can mean any valuable data; but in PlatformTransactionManager it is either a connection (for JDBC and JMS etc) or session (for Hibernate and JMS etc).

Both connection and session represent a communication link with the some backend resource manager (RM for short). In the database world, both termscan be used interchangeably. Although Hibernate uses session, it uses a connection behind the scene.
 

Transaction represents a unit of work. We mentioned in the above Section 1 that Spring's abstract transaction covers an underlying native transaction.

For JDBC and Hibernate, you create local transactions using connections / sessions and because they have one-to-one relationship, you can sometimes interchange the 2 terms. In other words, if the inner logic transaction participants in the outer physical transaction, it must reuse the the same connection / session.
For JTA, you creates global transactions using transaction managers; multiple connections can be enlisted in global transactions as transaction branches. One thing that is very different from JDBC and Hibernate is that you can conduct DML changes using different connections to the same backend RM instance in the same global transaction (This is because when a connection is enlisted, XID is created to represent this branch both in the JTA and the RM. Even you use a different connection, the RM can still identify the same transaction using the passed-in XID).

RM managers resources (transaction) on behalf of you. It is commonly used in JTA/XA world. For most of us, databases are common RM.

4. Transaction and Session -- Which Comes First?

In the foo() and bar() of FooBarService, you first create a connection / session, then starts a transaction. Before you close the connection / session, you first commit / rollback the transaction.
But in the foobar() of FooBarService, you first starts a transaction, then create a connection / session. Before you commit / rollback the transaction, you first close the connection / session.
In FooBarServiceSpring, all you have is just @transactional and you are not allowed to open or close any connection / session explicitly. So how can Spring hide the difference between JDBC, Hibernate and JTA and whois taking care of connections / session anyway?
Data access experience tells us that we always need a connection / session and its lifecycle methods (open, close etc) are just boilerplate and resource will leak if you forget to call the close method (actually this happens very often). But we must define transaction requirements by ourselves. For example, only you know that the bar() in FooBarServiceSpring requires REQUIRE_NEW propagation based on your business rules.

So Spring is taking care of connections / sessions behind the scene. You just need to specify you transaction requirement using @transactional. The next section tell you more details. 

5. Transaction and Session -- Resource Synchronization with Transaction
Because you always explicitly specify your transaction requirements, Spring synchronizes the needed resource (connection or session) with the transaction at both transaction start time and ending time or at the transaction scope boundaries.
 
For JDBC and Hibernate, Spring binds the connection / session (hence also local transaction) to the thread so that it is always available on the execution call stack. 
Spring PTM automatically either retrieves an existing connection / session from the thread (if an outer transaction is already there) or creates a new connection / session and binds it to the thread before starting a transaction.
When transaction is right before being committed or rolled back, Spring PTM also automatically closes the connection / session and unbinds it from the thread.


For JTA, Spring gets the UserTransaction from the standard JNDI name, so the global transaction is also available on the execution call stack.
When you retrieves a connection / session, it is automatically enlisted in the global transaction (Spring doesn't help here; the underlying XADataSource usually does the magic).

When the JTA transaction is right before being committed or rolled back, Spring PTM once again automatically closes the connection / session .

6. Nested Transaction and Transaction Suspend / Resume

• For nested transaction, both  the outer and inner transaction are live in your thread while when a transaction is suspended, your thread work on another new transaction;
• Nested transactions are dependent i.e. only the outer transaction is physical and can commit the change; but you can rollback the inner logic transaction.
  The suspended transaction and the new transaction are two physical transactions and are independent i.e. they can both commit or rollback independently;
• Suspending tranasction is deadlock-prone because the backend RM still holds the resource locks on the suspended transaction even your JTA has switched to another new transaction. You should be very careful even your application also locks some resources in the suspended transaction that are needed by the new transaction;
• Nested transactions are only supported by JDBC with the savepoint feature. Because XA doesn't support nested transactions, neither JTA can (This seems to be ironic because most databases support savepoint);
• Suspending a JDBC transaction in Spring means first unbinds it from the thread, then creates a new connection and binds to the thread. For JTA, Spring just delegates the suspend/resume to the underlying UserTransaction in addition to its own bookkeeping for resource synchronizations.

7. Propagation.SUPPORTS and No Transaction

No transaction means your data changes are committed on each individual SQL statement level (it is autocomit for JDBC) instead of grouping several SQL statements in a transaction.

Propagation.SUPPORTS also commits data changes on each individual SQL statement level if there is no existing transaction. But if you set your TransactionSynchronization in PTM to ALWAYS, it also supports resource synchronization on an empty transaction scope (empty transaction scope means no underlying native transaction).

To make it concrete, the same JDBC connection or Hibernate Session will bind to the thread inside the empty transaction scope for reuse in the execution call stack until the scope ends.  Be aware that the connection is of course in autocommit mode.

Please remember that if you empty transaction scope has active resource synchronization, DON'T nested Propagation.REQUIRED or Propagation.REQUIRED_NEW in it because resources synchronizations for the 2 transaction scopes will conflict (unfortunately you have to read the PMT source code in order  to get to the bottom).

Dynamically Creates Quartz Scheduler using Spring

 Here are  my requirements: I need to dynamically create quartz cron triggers and add them to my scheduler. All my triggers schedule the same job but with trigger specific job mapping data.

Before I show how to dynamically create quartz schedulers using Spring, let's quickly review Sping's static configuration. The following excerpts come from "Chapter 23 Scheduling and Tread Pooling" in Spring 2.5.6 reference document and shows how to statically create a quartz scheduler:

<bean name="exampleJob" class="org.springframework.scheduling.quartz.JobDetailBean">
   <property name="jobClass" value="example.ExampleJob"/>
</bean>

<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
   <property name="jobDetail" ref="exampleJob" />
   <!-- run every morning at 6 AM -->
   <property name="cronExpression" value="0 0 6 * * ?" />
</bean>

<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
   <property name="triggers">
      <list>
         <ref bean="cronTrigger" />
      </list>
   </property>
</bean>

Spring's JobDetailBean extends quartz's JobDetail and provides the following added functions:

1. Setups sensible defaults in its life-cycle init method afterPropertiesSet() e.g. set the job's name to the bean name and set the job's group name to "DEFAULT" if they are empty (Remember that quartz scheduler uniquely identifies a job by its name and group);
2. Supports any job class that implements the Runnable interface besides quartz'a regular Job interface;
3. If your actual Job class extends Spring's QuartzJobBean, Spring also automatically injects the job map data to the job's properties;

Spring's CronTriggerBean extends quartz CronTrigger and provides the following added functions:

1. Setups sensible defaults in its life-cycle init method afterPropertiesSet() e.g. set the trigger's name to the bean name and set the trigger's group name to "DEFAULT" if they are empty (Remember that quartz scheduler uniquely identifies a trigger by its name and group).
2. Associates the trigger to the provided's JobDetail in afterPropertiesSet();
3. Set the trigger's timezone to the default one and start time to the current time afterPropertiesSet();
4. Sets trigger specific job map data through the Java's Map interface;

Spring's SchedulerFactorBean is a quartz Scheduler factory bean and wires jobs and triggers together. It implements many functions behind the scene that you have to do by yourself  if you directly use quartz API:

1. Overrides default quartz properties e.g. thread pool parameters and jmx export;
2. Registers triggers;
3. Registers jobs. If you already assigned a job in your Spring trigger bean, you don't need this;
4. Sets the JobFactory to the default Spring's AdaptableJobFactory if it is not set by you. AdaptableJobFactory support Item 2 in the above JobDetailBean's added functions. AdaptableJobFactory's sublass SpringBeanJobFactory also supports the same function as mentioned in Item 3 in the above JobDetailBean's added function.
5. Schedules your jobs with quartz's scheduler;

All the above functions are implemented in the Spring' init method afterPropertiesSet().

Here are the steps to dynamically configure quartz scheduler using Spring to meet my above requirements.

1. Still configures my JobDetailBean, and my actual Job also extends Spring's QuartzJobBean in order to take advantage of the added function;
2. Still configures my CronTriggerBean using Spring  in order to take advantage of the added function. But its scope should be "prototype" so that I can dynamically retrieve a new trigger bean instance and set its cronExpression and its job map data.
   Here is the new trigger configuration:
   
   <bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean" scope="prototype">
   <property name="jobDetail" ref="exampleJob"/>
   </bean>
   
   Here is the pseudo code:
   CronTrigger ct = (CronTrigger)appCtx.getBean("cronTrigger");
   ct.setName(ct.getName() + some-distinguisher);
   ct.getJobDataMap().put(some-key, some-value);
   ct.setCronExpression(some-exp); 
   
   
3. Still configures my SchedulerFactorBean. But I need to add my above triggers to the scheduler. Because I create the above trigger after SchedulerFactorBean's afterPropertiesSet(), the trigger's associated JobDetail is not added to the scheduler. But I can explicitly register the JobDetail:
   
   <bean id="scheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
      <property name="jobDetails">
         <list>
           <ref bean="exampleJob"/>
         </list>
      </property>
      <property name="triggers"> <!-- will dynamically add more triggers -->
         <list/>
      </property>
      <property name="autoStartup" value="true"/>
      <property name="quartzProperties">
         <props>
           <prop key="org.quartz.scheduler.jmx.export">true</prop>
           <prop key="org.quartz.threadPool.threadCount">2</prop>
         </props>
      </property>
   </bean>
   Here is the pseudo code:
   Scheduler sched = (Scheduler)appCtx.getBean("scheduler");
   sched.scheduleJob(ct);

Will class literals (some-named-type.class) cause class initialization?

Suppose you have the following classes:

class Foo {
    static int cls_var = 10;
    static {
        System.out.println("cls_var:" + cls_var);
    }
   
    int inst_var = 20;
}

class Bar {
    public static void main(String arg[]) {
        System.out.println(Foo.class.getName());
    }
}

Will the class literal in bold blue cause Foo to be initialized (assign cls_var to 10 and run the following initializer)?
On the console you will only see "Foo"; you will not see "cls_var:10" as you may think you should see.

In order to find why, you need to know that JVM subject a class to the following 3 processes in order:

1. Loading 
2. Linking: Verification, Preparation, and Resolution   
3. Initialization

The class literal evaluates to the Class object for the named type (or for void) as defined by the defining class loader of the class of the current instance, and this information is accommodated by the above Step 1.

Foo's static variable initialization and the executor of static initializer are done in Step 3. Step 3 can be lazily executed by JVM after Step 1, which means Step 3 may not immediately follow Step 1.

This is why you will only see "Foo" output on the console. For more details, please see the linked JVM spec and Java language spec.