Web Software Architecture and Engineering – Life on the Bleeding Edge

In case you’ve been hiding under a rock, you know that the Verity search engine has its limitations. Its nice, will serve many functions, but once you need to go beyond a certain record limit or hit performance snags, you quickly start looking for alternatives. (And thats just Windows) I’ve been working with Lucene off and on for 3 years. I’ve never quite gotten it to work with ColdFusion quite right. My company has finally started hitting both performance and record limit issues, so we need an alternative. I was working with Solr as well, and SolColdFusion is promising as well. But raw integration with Lucene, that’s another thing.
So when Ray Camden decided to give it a go, I was quick to encourage him. And finally getting a chance to evaluate his code, I’ll be posting on what I find.
First things first. We’ve had integration with Lucene in the past. There was Lindex, CFLucene, and others. Lately, Wayne Graham started work on integrating Solr with ColdFusion. In case you haven’t heard, Solr is a project on top of Lucene that allows you to hit Lucene commands via HTTP. Officially, “Solr is an open source enterprise search server based on the Lucene Java search library, with XML/HTTP and JSON APIs, hit highlighting, faceted search, caching, replication, and a web administration interface. It runs in a Java servlet container such as Tomcat.” Having that sort of API makes it easier to interact with Lucene. And finally, we have Ray’s work on Lucene in a project named Seeker.
So what’s next? Well, I’ve asked Adobe to simply include Lucene/Solr as alternative to Verity natively. But until that happens (and if it ever does), I’ll be playing with Ray’s and Wayne’s code further and posting my thoughts here.

Advertisements

Comments on: "ColdFusion & Lucene / Solr" (9)

  1. Great!
    I have a question – may not be directly related to your post. Is it possible in anyways to search lucene indexes using Action Script 3.0. I am looking at an offline solution, which has no access to internet. The index files can be pulled in directly from internet when online.. or pulled into a DB or…

  2. @Vipin:

    I am sure there is, but I am not directly aware of any. What you may want to do is look at the Lucene sub-projects, where they try to solve issues like this and provide unique methods of accessing the Lucene project.

    I’m assuming you are talking about having a way for Flash (via AS 3) to talk to Lucene files.

  3. @Sami..

    Yes, I am talking about Flex (AIR Application) to talk to lucene files. I am looking for an offline solution for the search. I can ofcourse import the lucene terms, scores, and docids to a database (SQLite) and search on it. This will be slower than what could be possible with a direct search in lucene files. An AS3 port of lucene library may be a solution. I will look for the sub-projects, but as an experienced community member, if you can shed some light on this (now or as you come to know…) that will be much helpful…

  4. @Vipin:

    One possible solution: http://www.blog.lessrain.com/as3-java-socket-connections-to-ports-below-1024/. You may be able to have AS3 talk to Solr with its HTTP API. Solr is easy to start up with its Jetty engine. I would start here first. Another possibility, and one I may explore in the future, is to run Railo in the background, and have your Flash talk to a CFC, which works with Lucene inside Railo. As you may know, Railo supports Lucene as the default search engine and not Verity. Either way, since AS3 is HTTP friendly, you may be able to use the technologies as a bridge to Lucene. At least, this is where my mind goes first. Not an expert on AS3 myself.

  5. Hi Sami,

    Rob Munn and I have been encouraging Adobe to include Lucene (and especially Solr) as well. Rob Munn’s actually done a bunch of CF integration with Solr in the past, and I believe he has some of the code available on his blog.

    Regardless, from my point of view, the parametric search capabilities are the sweet spot for the Solr capabilities. And, if Adobe gets to the point where they can replace Verity with Lucene/Solr, it would surely cut down on the OEM costs within the product as they would no longer have to pay license fees to the Verity folks.

    Verity is fine in its own right, but the lack of real parametric search capabilities combined with the limits on the index size keep it a limited option for many.

  6. Thanks…I will start from here….

  7. @Rob,

    I wasn’t able to find anything on Rob Munn’s blog. But yeah, as long term insiders we can continue to push. I’ll keep blogging about this most definitely.

  8. Hi Sami,

    Ooops, I should have mentioned that Rob actually has 2 blogs. The Solr entries are here:

    http://blog.emergentpath.com/index.cfm/Enterprise-Search

  9. i would love to read more from you on this

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: