.net – Storing words with apostrophe in Lucene index

lucenelucene.net

I've a company field in Lucene Index.
One of the company names indexed is : Moody's

When user types in any of the following keywords,I want this company to come up in search results.
1.Moo
2.Mood
3.Moodys
4.Moody's

How should I store this index in Lucene and what type of Lucene Query should I use to get this behaviour?

Thanks.

Best Answer

Based on your clarifications, I want to divide your question into two, and answer each in turn:

  1. How do I index words with apostrophes as equivalent to similar words without an apostrophe? e.g. mapping Moodys and Moody's to the same index term.
  2. How do I implement auto-complete search in Lucene - i.e. given an index, find documents using word prefixes, e.g. map Moo to Moodys ?

1 is relatively easy - Use a StandardToeknizer to create a token combining the apostrophe and s with the previous word, then a StandardFilter to remove the apostrophe and s. This will convert Moody's to Moody. A StandardAnalyzer does this and much more (lowercasing and stop word removal), which may be more than you need. Using a stemmer should take both Moodys and Moody to the same token. Try SnowBallFilter for this.

2 is harder: Lucene's PrefixQuery, to which Alan alluded, will only work when the company name is the first word in a field. You need something like the answer to this question about auto-complete in Lucene.

Related Topic