Tuesday, December 25, 2012

Technical Evalutions for a Real Time Searching System Part 1 -- Overview

I recently conducted a technical evaluations for a real time search system (RTSS hereafter). It primarily focuses on mature open source frameworks. Many technologies involved are quite new and very interesting.
I present my ideas in a 4-part series:
  • Part 1 -- Overview
  • Part 2 -- Solutions Based on SQL and NoSQL DB (H2 and MongoDB)
  • Part 3 -- Solutions Based on Index Search Engines (Lucene Solar and Sphinx);
  • Part 4 -- Solutions Based on XML DB (MarkLogic and Berkeley DB XML)

1 Main Business Requirements for RTSS

  • Supports auto-complete and spell checkers on customized list of names;
  • Supports advanced searching and browsing on many combinations of attributes;
  • Support ranking / relevance;
  • Supports different types of clients (Java/C/C#) by being service-oriented;
  • Supports huge user based by being reliable and scalable;

 2 Architecture Layers

The RTSS can be divided into 3 logic layers. From front-end to back-end, they are as follows:
  • The UI layer that includes web applications hosted on Jetty/Tomcat/JBoss, and non-web applications such as Java Swing and c/C++;
  • The service or business layer that provides results to the UI layer for auto-complete and spell-checker and all kinds of search requests either in a language neutral format such as XML, Fast Infoset, JSON or such descriptive binary protocol as Google Protocol Buffers;
  • The DAO layer that provides the service layer with search results. Data stores (SQL, NoSQL, index search engines and XML DB) will also be discussed along with DAO.
The 3 layers may be hosted on different servers for flexibility and scalability. However, they are usually combined together if possible for better performance. Data stores are usually on a separate server. However, in-memory DB can be embedded into the service layer.
In the following discussion, the lines or words in bold blue are viable options for final selection.

No comments:

Post a Comment