The latest IT bandwagon that vendors are jumping all over is complex event processing, which promises to help firms make rapid decisions on streaming data (tick prices, for example)that is "constantly changing".
It is lumped into the broad category of 'business intelligence,' which is seeing a lot of activity of late with Oracle announcing its $3.3 billion acquisition of business intelligence (BI) vendor Hyperion, which it will combine with its own BI software to provide customers with tools for collecting and analysing information about their business.
What is the big deal about BI and complex event processing? Every firm has reams of customer, logistical and transactional data stored in silos, but this data is meaningless unless firms can glean some form of intelligence from it and use that to enhance customer service levels or gain a competitive advantage.
Event processing (EP) has found a natural home in the capital markets, where according to Brad Bailey, a senior analyst at Aite Group, "[it] has made an impressive showing in the algorithmic and strategy trading areas, and EP solutions are now migrating to other areas of the capital markets, such as data monitoring, compliance, Transaction Cost Analysis (TCA), risk management, proprietary data derivations, market making, and others."
A number of vendors are now claiming that event processing software is a 'must-have' component of firms' preparations for MiFID particularly when it comes to intelligent order routing and demonstrating best execution
Yet, making sense of what vendors are really offering in the event processing (EP) space is fraught with difficulty as not all of the firms in this space are specialists, they have merely added on BI or EP tools to existing data management applications.
Phil Howard, research director at Bloor Group cautions that "misleading claims" are being made about performance in the CEP space. There are claims and counterclaims about the speed and performance of various EP applications. There also appear to be some deep-rooted philosophical differences between vendors, which is only adding to the confusion for firms contemplating event processing for the first time.
Streambase Systems, designed its own language (StreamSQL) which adds time and event-based windows to standard SQL (Structured Query Language)to support live time queries on event streams. SQL has been used for many years to access and manipulate database systems.
Barry Morris, chairman and CEO of Streambase, says there is no other candidate than SQL, which is more widely used and understood than some of the proprietary technologies and languages competitors are developing.
Critics of SQL say it is not up to the job of performing real-time queries on streaming data, yet Morris maintains that it hasn't encountered any problem it hasn't been able to solve using StreamSQL. "StreamSQL allows firms to access static data that is on disk or in-memory in the same way that they access real-time data," he says. He claims that an "old fashioned" 'rules-engine' type approach, which some of its competitors offer, does not know how to handle static data.
Friday, March 02, 2007
Subscribe to:
Post Comments (Atom)
6 comments:
Wow - I didn't realize that rules engines prevented one from utilizing 'static data.' I better have an immediate meeting with my engineering team to make sure we covered that one...
Colin Clark
CTO - Kaskad
I would like to clarify any potential confusion in the published comments from StreamBase. Space limitations may have prevented publishing the complete story. StreamSQL allows firms to process real-time event streams at speeds of 10s of thousands to hundreds of thousands of messages/second or more. StreamSQL also allows firms to access stored data using the same language and paradigm as is used for analyzing real-time data. The syntax and semantics are designed to be identical to standard SQL, except for the addition of time or event-based windows and other attributes necessary for real-time streams. Major IT industry players are also now lining up behind such a SQL-based standard for stream processing. Rules engines do not typically use a language based on the SQL standard, thus their proprietary language for querying and analyzing real-time data is often different than that which would be used for querying SQL-compliant databases. StreamSQL offers one standards-based language, approach, and paradigm for querying both real-time streams and stored data.
We appreciate the highlighting of StreamBase's unique standards-based approach to complex event processing and welcome additional comments.
Bill Hobbib
VP of Marketing
StreamBase Systems
Thanks for your comments and clarification Bill on StreamSQL. I did try and make the distinction between well-established standards like SQL and the proprietary ones developed by other vendors but perhaps that was not clear. Interestingly however, I have read that a number of players don't buy into the SQL approach.
Until you can buy a book on a language, it's not even close to being a standard. Vendors in this space who use rules engines are basing their approach on standards too; like C++ and/or Java.
There seems to be little support for a SQL based standard for ESP (in terms of companies offering this approach). And if the standard is only supported by a couple of implementations, then the standard isn't worth much (anyone have a video on beta out there?).
Colin's right, there's no support for SQL as a standard in the event processing space, and we (at Progress Apama) are disappointed that Anita didn't take a broader view of this issue. StreamBase issued a press release about "braod support" for SQL-based standards 6 months ago and, since then, no other vendor has supported it, no customers have supported it, and no standards activity has spawned as a result. Furthermore, event processing has nothing, syntactically or semantically, to do with SQL, as the marketing people from StreamBase claim.
So, although the claim that SQL is the hammer which would apply to any software nail is an intriguing, but specious argument.
That said, what Anita covers very nicely is that the adoption of event processing is happening. Colin from Kaskad can cite the Boston Stock Exchange as proof; Progress Apama can cite firms like Deutsche Bank, ABN Amro, JP Morgan, Finames, Koscom, and HG Trading.
All these firms, and many others, have been public about their use of non-SQL approaches that leverages defacto standards such as Java as the best way to attack this challenge.
- Mark Palmer
General Manager, Progress Apama
I must point out this is not the first time I have written about the different approaches to CEP. In previous posts I have spoken about SQL and how a number of firms do not support it in the complex event processing space.
With my post on Streambase's comments I was merely presenting an alternative viewpoint to those earlier postings, so I disagree with the comment that I haven't taken a broader view. The broader view from an outsider looking in is that these key philosophical differences between CEP vendors over SQL or non-SQL may just end up confusing firms contemplating this nascent technology for the first time. And as with a lot of things in the financial services industry, it comes down to a debate about standards. Are firms going to want to implement technologies that support proprietary standards or more importantly, as firms jockey for position in this space, what is going to be the predominant standard?
Post a Comment