|
Complex Event Processing: Leveraging Intelligence From Massive Amounts of Data |
|
|
High-performance CEP is the continuous processing, analysis, and response to high-volume, high-speed data streams in real-time. It can be used to identify and analyze cause-and-effect relationships among events. And because it does all of this in real-time, it enables instant and informed responses to critical intelligence in real-time. Four different approaches are typically considered for high-performance CEP:
- Custom-Coded Applications - typically written in Java, C++ or other similar programming languages. Custom applications had been the leading solution for many organizations until the recent advent of off-the-shelf CEP. However, custom coding is limited by long development cycles of 6-9 months or more, the high cost of maintenance, and inflexibility.
- Rules Engines - for watching input data-streams for conditions of interest that match its rules - typically condition/action pairs, usually expressed using “if-then” notation. The rules are typically written in a proprietary language and when a condition of a rule is matched, the rule is said to “fire” a corresponding action, alert or output to external applications.
- In-Memory Databases - to reduce the latency delays of traditional database management systems by eliminating disk I/O, caching, and transaction logs. But they still incur latency penalties and require custom-coding to build CEP applications.
- Stream Processing Software - handles streaming data and performs queries and computations on incoming messages as they fly by - without storing them. Stream processing uses specialized primitives and constructs, such as time-windows, to express stream-oriented processing logic.
Of all of the commercial offerings, high-performance CEP using stream processing software offers the lowest latency, highest throughput, most flexible programming model, and fastest time-to-value.
|
