Thursday, March 15, 2007

The SQL debate

Having opened a veritable 'can of worms' with my post dated the 2 March entitled, Event Processing gets 'complex', my curiosity regarding the debate surrounding the use of Structured Query Language (SQL) in some complex event processing (CEP) applications, was further aroused.

The debate that post generated can more or less be summed up as the pro-SQL vs non-SQL camp, with CEP vendors like StreamBase Systems putting its case for an SQL-based standard for querying real-time event streams and stored data, while other vendors such as Kaskad and Progress Apama stated that there was little support for a SQL-based standard for event stream processing (in terms of companies offering this approach).

That may well be the case, but if so many CEP vendors do not support SQL, it made me wonder why some were sticking to their guns and implementing SQL-based languages anyway? I was interested to read an article penned by John Morrell of complex processing software provider, Coral8, reproduced on the Complex Event Processing web site.

Coral8 has developed an SQL-based Continuous Computation Language. In the article Morrell writes, "This [SQL] provides a familiar programming environment, speeding the creation of event processing applications." He later adds that as SQL is a 'pervasive' language, it lowers the learning curve for developers of CEP applications.

So there you have it. The highly competitive CEP space is dominated by vendors pushing different approaches, and after all choice is a good thing. But is there a right or wrong approach, and can SQL in CEP applications be discounted altogether if it does do what its proponents say it does in terms of speeding up development?

5 comments:

Anonymous said...

Greetings,

If you are interested in this subject, I'd appreciate your comments on my recent blog post on
CEP and SQL: The Top Five Myths.

Regards,

Mark Tsimelzon
CTO, Coral8

FinancialTech Insider said...

Thanks. I found your post interesting having been chastised by non-SQL CEP vendors for even daring to mention SQL in the same breath as CEP.The non-SQL vendors say that most customers have opted not to support SQL by virtue of choosing them. I imagine though that some customers must be choosing vendors like yourself otherwise why would you be in business. Is SQL raised by your customers in the RFP process?

Anonymous said...

Yes. Customers mention SQL in RFPs all the time, and, more important, it's the number one reason they tell us our product is easy to use.

The ease of use is critical to our business model, as we are a startup and don't have a huge sales force. The Coral8 Engine is available on our web site, and a good fraction of our customers download it, learn it, build the prototype, and only then contact us for the first time about the license.

When we ask them how they manage to build a non-trivial prototype without any presentation, training, or support, the answer we get most often is "What, you think we are stupid here? It's just SQL!"

We feel going with an SQL-based language was by far the best decision our company ever made.

Mark Tsimelzon
CTO, Coral8

Ashwin Jayaprakash said...

Hello,
I'm working on an SQL/RDBMS based Event Processing Engine called StreamCruncher (http://www.streamcruncher.com) that is currently Free (as in Beer). I've found that SQL as a language, with additional Event Processing constructs lends itself very well to describing Stream Processing/Correlation Queries. I'm quite certain that SQL based Syntax will be especially appealing to Developers who will eventually be using Event Stream Processing in their Projects when it becomes more widely used/accepted. The adoption would certainly be quite easy because we've been using Databases for so long now.

Ashwin (http://www.streamcruncher.com).

Mark said...

Hi Anita - I hope you haven't felt "chastised" by Apama on this standards issue. The issue we have is in some vendors continuing to polarize this young industry by pushing SQL as the "only way" to do CEP, when clearly there are many good approaches.

We want to ensure that the industry considers the range of CEP approaches that our clients care about, which includes language-based (the most common), graphical high-level tools (designed for a different type of user than SQL-based tools), CEP rules, and Java extensions for CEP. Yes, SQL is a technically feasable approach to CEP, but customers will ultimately decide what the best approach is as they support vendors whose techniques match their requirements. For more detail on this position read this post in the Apama blog