C# – Howto perform a ‘contains’ search rather than ‘starts with’ using Lucene.Net

clucenelucene.netsearch

We use Lucene.NET to implement a full text search on a clients website. The search itself works already but we now want to implement a modification.

Currently all terms get appended a * which leads Lucene to perform what I would classify as a StartsWith search.

In the future we would like to have a search that performs something like a Contains rather than a StartsWith.

We use

Lucene.Net 2.9.2.2
StandardAnalyzer
default QueryParser

Samples:

(Title:Orch*) matches: Orchestra

but:

(Title:rch*) does not match: Orchestra

We want the first and the second one to both match Orchestra.

Basically I want the exact opposite of what was asked in this question, I'm not sure why for this person Lucene performed a Contains and rather than a StartsWith by default:
Why is this Lucene query a "contains" instead of a "startsWith"?

How can we make this happen?
I have the feeling it has something to do with the Analyzer but I'm not sure.

Best Answer

First off, I assume you're using StandardAnalyzer, or something similar. Your linked question fail to understand that you search for terms, and his case a* will match "Fleet Africa" because it's tokenized into "fleet" and "africa".

You need to call QueryParser.SetAllowLeadingWildcard(true) to be able to write queries like field:*value*. Are you actually changing the string that's passed to QueryParser?

You could parse the query as usual, and then implement a QueryVisitor that rewrites all TermQuery into WildcardQuery. That way you still support phrase searches.

I see no good things in rewriting queries into prefix- or wildcard-queries. There is very little shared between an orc, or a chest, and an Orchestra, but both words will match. Instead, hook up your customer with an analyzer that supports stemming, synonyms, and provide a spell correction feature to fix simple searching mistakes.

Related Solutions

C# – Trouble searching for acronyms in Lucene.NET

The StandardAnalyzer uses the StandardTokenizer which tokenizes 'N.A.S.A.' as 'nasa', but won't do this to 'N.A.S.A'. That's why your original query matches both the input 'N.A.S.A' which are processed into 'nasa', and the input 'nasa' which matches the already tokenized value. This also explains why 'N.A.S.A' wont match anything since the index only contains the token 'nasa'.

This can be seen when outputting the value from the token stream directly.

public static void Main(string[] args) {
    var analyzer = new StandardAnalyzer(Version.LUCENE_30);
    var stream = analyzer.TokenStream("f", new StringReader("N.A.S.A. N.A.S.A"));

    var termAttr = stream.GetAttribute<ITermAttribute>();
    while (stream.IncrementToken()) {
        Console.WriteLine(termAttr.Term);
    }

    Console.ReadLine();
}

Outputs:

nasa
n.a.s.a

You would probably need to write a custom analyzer to handle this scenario. One solution would be to keep the original token so n.a* would work, but you would also need to build a better detection of acronyms.

Java – How to perform a lucene query containing special character using QueryParser

I am not sure about this , but I guess you need to escape - with \ . As per the Lucene docs.

The "-" or prohibit operator excludes documents that contain the term after the "-" symbol.

Again ,

Lucene supports escaping special characters that are part of the query syntax. The current list special characters are

+ - && || ! ( ) { } [ ] ^ " ~ * ? : \ /

To escape these character use the \ before the character.

Also remember, some characters you'll need to escape twice if they have special meaning in Java.

Best Answer

Related Solutions

C# – Trouble searching for acronyms in Lucene.NET

Java – How to perform a lucene query containing special character using QueryParser

Related Topic