RSS feed
<< The extra dot in dot-com | Home | Implementing decorator templates with Apache Velocity >>

Lucene gets a performance boost

Following the release of Lucene 2.3.0 on January 24th, I decided to upgrade an application I'm developing. The change log promised several improvements in indexing performance.

My application uses Lucene to index moderately complex information about events. Testing the application with Lucene 2.2.0 I found that a full reindex of 80.000 events stabilized on 164 seconds after a few rounds to make sure the JVM and caches were warmed up.

After upgrading to 2.3.0 and switching IndexWriter.setMaxBufferedDocs() to the new IndexWriter.setRAMBufferSizeMB() I started a reindex again to measure the difference.

So how long did it take to reindex 80.000 events in Lucene 2.3.0?

35 seconds

That's 470 percent faster!

So thanks to the Lucene team for the great work with this release. These guys continue to impress me.

Tags :
Responses[2] RSS feed for responses to this blog entry
Posted by Eirik Bjørsnøs on February 15, 2008 12:00:13 AM CET #


Reply

Re: Lucene gets a performance boost

Comment from Mihai Campean on February 15, 2008 9:46:43 AM CET #
This sounds really impressive, can you please give a bit more details about the project and hardware you use, just so we can get a better idea of the conditions of the improvements.

Thanks,
Mihai
Reply

Re: Lucene gets a performance boost

Comment from Eirik Bjørsnøs on February 15, 2008 1:44:06 PM CET #

Hi Mihai,

The tests were run on my MacBook Pro. This machine has a 2GHz Intel Core Duo CPU and 2GB RAM.

I used Apples Java 1.5 on OSX 10.4.11. My test includes optimizing the index and then refreshing of caches after the reindex so the raw improvements in Lucene should be even better than what my test shows.

If you want details on the project itself, please contact me via email.


Add a comment Send a TrackBack