Solr Search Query API

Version 16.1 by Admin on 2021/03/16 18:04

cogQuery module for the Solr index. Allows performing Solr queries using XWiki's Query API
TypeJAR
Category
Developed by

XWiki Development Team

Rating
1 Votes
LicenseGNU Lesser General Public License 2.1
Bundled With

XWiki Standard

Installable with the Extension Manager

Description

The Solr Search Query API is exposed using the Query Module API.

API Usage

Performing the search

From Scripting

Example 1

Searching for all pages in all wikis, with content 'text' somewhere and with an XWiki.TagClass xobject with the tags property having a value of News:

Before 9.0, this API require programming right.

{{include reference="Main.SolrSearchMacros"/}}

{{velocity}}
#set ($queryStatement = 'text AND property.XWiki.TagClass.tags:News')
#set ($query = $services.query.createQuery($queryStatement, 'solr'))

## Filter result based on current user right (true by default)
#set ($discard = $query.checkCurrentUser(true))
## Only receive 10 results
#set ($discard = $query.setLimit(10).setOffset(0))
## Sort based on score in descendant order
#set ($discard = $query.bindValue('sort', "score desc"))
## Filter the results to get only documents with locale fr or en and which are not hidden
#set ($discard = $query.bindValue('fq', '{!q.op=AND} type:DOCUMENT locales:(en OR fr) hidden:false'))

#set ($searchResponse = $query.execute()[0])
#foreach ($searchResult in $searchResponse.results)
 #displaySearchResult($searchResult)
#end
{{/velocity}}

In the example above $searchResponse is an instance of QueryResponse. You should read the Solr schema design to understand what you can search for using this API.

Example 2

Searching for all pages in all wikis having an xobject of type XWiki.XWikiGlobalRights:

{{velocity}}
#set ($queryStatement = 'object:XWiki.XWikiGlobalRights')
#set ($query = $services.query.createQuery($queryStatement, 'solr'))
#set ($discard = $query.bindValue('sort', "score desc"))
#set ($discard = $query.bindValue('fq', 'type:DOCUMENT'))
#set ($searchResponse = $query.execute()[0])
#foreach ($searchResult in $searchResponse.results)
  * [[${searchResult.wiki}:${searchResult.fullname}>>${searchResult.wiki}:${searchResult.fullname}]]
#end
{{/velocity}}

From Java

Make sure your Maven module's pom.xml has a dependency on:

<dependency>
 <groupId>org.xwiki.platform</groupId>
 <artifactId>xwiki-platform-search-solr-query</artifactId>
 <version>(version of xwiki you wish to use)</version>
</dependency>

Then, you could do something along these lines:

// Declare the dependency injection in the code
@Inject
private org.xwiki.query.QueryManager queryManager;
...
// Use it to create a query
org.xwiki.query.Query query = queryManager.creatQuery("your SOLR query statement", "solr");
// Bind more values to the query
query.bindValue("whateverYouWantToAdd", value);
// Execute the query
org.apache.solr.client.solrj.response.QueryResponse response = query.execute().get(0);
// Use the response to get the information you need.
response.getResults();
// Etc

Manipulating the results

The search results are of type SolrDocument which provides APIs to retrieve the values that have been added to the Solr index. Check the Solr Schema documentation for the available index fields. Note that in order to be able to retrieve the value of a Solr index field, two conditions must be met:

  • the field must be stored (i.e. the field definition must have stored="true" in the schema.xml file)
  • the field must be selected by the fl (field list) query parameter

Each search result has an associated entity. Most of the time it is a Document, but it can also be an Attachment, Object or Object Property. This is the entity that has been indexed. One common operation is to determine which is the entity associated with a search result. We provide an EntityReferenceResolver<SolrDocument> for this, that you can use to extract an EntityReference from a search result.

Note that if some information is available on the search result (e.g. document title, file size, etc.) then it's much faster to access it from there rather than loading the associated entity and getting the same information from the entity.

From Scripting

If the type field is specified in the search result then you can use:

#set ($searchResultEntityReference = $services.solr.resolve($searchResult))

If the type field is not specified in the search result (e.g. because the search query selects only results of a given type) then you can use:

#set ($attachmentReference = $services.solr.resolve($searchResult, 'ATTACHMENT', $NULL))

You can also use a dedicated resolver like this:

#set ($documentReference = $services.solr.resolveDocument($searchResult))

From Java

/**
 * Used to resolve any type of entity reference.
 */

@Inject
private EntityReferenceResolver<SolrDocument> solrEntityReferenceResolver;

/**
 * A dedicated resolver.
 */

@Inject
private DocumentReferenceResolver<SolrDocument> solrDocumentReferenceResolver;

Standard Search Parameters

When we execute a search query we can specify configuration parameters that influence the search results. Let's see what are the most common query parameters:

ParameterDescription
qDefines a query using standard query syntax. This parameter is mandatory
q.opSpecifies the default operator for query expressions. Possible values are "AND" or "OR"
qfQuery fields: specifies the fields in the index on which to perform the query by default. Of course these have to be indexed fields. E.g. qf="title^10.0 doccontent^2.0"
flField list: the list of fields that should be included in each search result. Of course, these must be stored fields. E.g. fl="title,doccontent"
sortSpecifies the sort field. E.g. sort="title_sort asc". Search results are sorted by relevance score by default.
fqFilter query: additional constraints to satisfy besides the query
facetWhether to activate faceting or not
facet.fieldSpecifies a faceting field. E.g face.field="creator"
hlWhether to activate highlighting or not
hl.flHighlighting field list: the list of fields for which you want highlighting

See the Solr common query parameters and the Extended DisMax query parser parameters documentation for other, less used, parameters.

Custom Search Parameters

NameDescriptionVersion
xwiki.multilingualFields

The list of multilingual fields that will be expanded in the search query. This way, a user can write a query on the "title" field and all the "title_<language>" variations of the field will be used in the query. The list of languages for which a field is expanded is taken from the 'xwiki.supportedLocales' query parameter (see below).

Default value: title, doccontent, doccontentraw, comment, objcontent, propertyvalue, attcontent, property.*, object.*

5.3+
xwiki.supportedLocales

The list of supported locales. This is used to expand the fields specified by xwiki.multilingualFields in search queries.

Default value:

  • 6.4M1+, the list of supported locales configured for the current wiki (where you execute the search)
  • < 6.4M1, empty list (which means only the ROOT locale is used for expanding multilingual fields in search queries)
5.3+
xwiki.typedDynamicFields

The list of typed (non-string) dynamic fields that will be expanded in the search query. The names of these fields are suffixed with the name of their data type (e.g. *_int is a dynamic integer field). This allows users to write a query like this:

property.Blog.BlogPostClass.publishDate:[NOW-1MONTH TO NOW]

which will be expanded into:

property.Blog.BlogPostClass.publishDate_boolean:[NOW-1MONTH TO NOW] OR
property.Blog.BlogPostClass.publishDate_int:[NOW-1MONTH TO NOW] OR
...
property.Blog.BlogPostClass.publishDate_date:[NOW-1MONTH TO NOW]

Default value: property.*

5.3+
xwiki.dynamicFieldTypes

See the dynamic field definitions in schema.xml.

Default value: boolean, int, long, float, double, string, date

5.3+

You can see more parameters in solrconfig.xml. Search parameters are set using the Query Module API:

$query.bindValue('parameterName', $parameterValue)

Faceting on Object Properties (5.3+)

You can enable a facet on an XObject property using the Query Module API:

#set ($discard = $query.bindValue('facet.field', ['someOtherField', 'property.Test.TestClass.staticList1_string']))

where:

  • Test.TestClass is the class name
  • staticList1 is the property name
  • 'string' suffix means the property was indexed/stored verbatim (without being analysed). See the Solr Schema for details.

Then you can trigger the search facet with a search query like this:

object:Test.TestClass

You'll have to customize the facet display to make it look nicer, but it should work nevertheless without any changes to the search UI.

Sorting on Object Properties (5.3+)

You can sort the document search results based on a property value using the Query Module API:

#set ($discard = $query.bindValue('sort', "property.Test.TestClass.staticList1_sortString asc"))

Test.TestCass is the class name and staticList1 is the property name. The 'sortString' suffix is the dynamic type that is used for sorting. See the Solr Schema for details.

Note that Solr doesn't support sorting on multivalued fields. The documentation says:

 Sorting can be done on the "score" of the document, or on any multiValued="false" indexed="true" field provided that field is either non-tokenized (ie: has no Analyzer) or uses an Analyzer that only produces a single Term (ie: uses the KeywordTokenizer).

If you try to sort on a multivalued field you'll get:

Caused by: org.apache.solr.common.SolrException: can not sort on multivalued field: property.Test.TestClass.staticList1_string
    at org.apache.solr.schema.SchemaField.checkSortability(SchemaField.java:155)

That's why we use dedicated 'sortXXX' fields that are single valued. The consequence is that only the last value of a property is used for sorting (you can have multiple values either because the property supports multiple selection or because there are multiple objects of the same type on the indexed document).

Prerequisites & Installation Instructions

We recommend using the Extension Manager to install this extension (Make sure that the text "Installable with the Extension Manager" is displayed at the top right location on this page to know if this extension can be installed with the Extension Manager).

You can also use the manual method which involves dropping the JAR file and all its dependencies into the WEB-INF/lib folder and restarting XWiki.

Dependencies

Dependencies for this extension (org.xwiki.platform:xwiki-platform-search-solr-query 13.1):

Get Connected