Index for the LLM Application (BETA)

Last modified by Michael Hamann on 2024/06/06 10:09

cogKnowledge index providing search capabilities to the LLM Application.
TypeXAR
CategoryApplication
Developed by

Matéo Munoz, Ludovic Dubost, Michael Hamann, Paul Pantiru

Rating
0 Votes
LicenseGNU Lesser General Public License 2.1
Compatibility

16.2.0 and above

Installable with the Extension Manager

Description

The knowledge index provides a way to configure and manage an index of collections of documents that can provide additional context to chats in the LLM Application. A technique called Retrieval Augmented Generation (RAG) is used to provide context to the chat completion. Context chunks that are semantically similar to the chat message are retrieved from the index and provided to the completion model. This allows the completion model to use knowledge from the index to generate more factual completions.

There are two ways to manage collections and the documents in them: through the UI that is provided in this extension or through the REST API. Right now, the extension primarily supports indexing external content, but in the future, it will also support indexing existing content that is already in the wiki.

Managing Collections

The AI application provides a simple interface for managing collections and their documents. When opening the AI application as admin, a link is provided at the bottom to open the Collections overview.

Collections overview, showing two collections in a table

Here you can add a new collection or edit or delete an existing one. In the view of a collection, you can edit all properties, see the documents in the collection and add new documents. The name of the page of a document is a random value that is derived from the name of the collection. It doesn't use the name directly in order to support long document names, e.g., coming from an external application that might not be valid document names in XWiki. Renaming documents isn't supported at the moment as the primary use case is indexing external content. In a future version, it will be possible to index existing content of the wiki instead of managing the content separately.

View of a single collection with various properties and two documents in a table below it

The following properties can be edited:

  • Title: The display name of the collection
  • Embedding Model: The reference to the embedding model that can be configured in the models section of the AI application. This model is used to generate embeddings of parts (chunks) of the documents in the collection in order to support similarity search. By embedding the chat message in the same way, the similarity of the chat message to the documents in the collection can be calculated and the most relevant documents can be found.
  • Chunking Method: The method used to split the documents into chunks. Currently, only the maxChars method is supported, which splits the document into chunks of a maximum size with an overlap.
  • Chunking Max Size: The maximum size of a chunk in characters
  • Chunking Overlap Offset: The overlap of chunks in characters
  • Allow Guests: If guests can query this collection (as part of a chat completion)
  • Query Groups: The list of groups that are allowed to query this collection (as part of a chat completion). This is only used when guests aren't allowed to query the collection.
  • Rights Check Method: The method used for checking access rights of individual documents during queries. Supported values: public, external. See also the Authorization section below.
  • URL of the external rights check method: The URL that is used to check rights for all found documents when the external rights check method is selected. This property is only displayed when the external rights check method is selected.

Managing Documents

In the view of a collection, you can see the documents in the collection and add new documents.

View of a single document "WAISE Intro"

You can configure the following properties of documents:

  • Title: The title of the document
  • Language: The language of the document (currently not used)
  • URL: The URL of the document, used to display a link to the original resource when the document is used as context in a chat
  • Mime Type: The mime type of the document, currently not used, could be used in the future to use a chunking method that is specific to the mime type
  • Content: The content of the document that is indexed

Additionally, you can attach files to a document and they will also be indexed if the content can be extracted by Apache Tika.

Authorization

At query time, regular XWiki access rights aren't checked. Instead, access to the indexed can be controlled at two levels:

  • The collection: It can be controlled which groups can query a collection in the configuration of the collection. A collection can also be allowed for guests, in this case no check is performed.
  • The document: On every collection, a method for checking rights for individual documents can be configured. After retrieving relevant chunks of context information, this rights checking method is asked for authorization for every retrieved document.

By default, two rights check methods are provided:

  • Public: this method just allows access to all documents. It is best suited when all users who have access to the collection should also be allowed to access all documents in it.
  • External: this method queries an external API via HTTP to check which documents can be accessed. It supports configuring a URL that is contacted on every query.

External Authorization Checks

On every query, a POST request with the following contents is submitted to the configured URL:

If you need to pass additional parameters like the collection name you can specify them as URL parameters.

An example request for a single document for the standard admin user could look like this:

{"document_ids":["Performance"],"xwiki_username":"xwiki:XWiki.Admin"}

The response needs to be an object mapping document IDs to true or false, meaning that the user has or hasn't access to the respective document. To grant access to the "Performance" document in the example, the endpoint would need to reply {"Performance":true}. Any missing values or errors are interpreted as no access.

Prerequisites & Installation Instructions

We recommend using the Extension Manager to install this extension (Make sure that the text "Installable with the Extension Manager" is displayed at the top right location on this page to know if this extension can be installed with the Extension Manager). Note that installing Extensions when being offline is currently not supported and you'd need to use some complex manual method.

You can also use the following manual method, which is useful if this extension cannot be installed with the Extension Manager or if you're using an old version of XWiki that doesn't have the Extension Manager:

  1. Log in the wiki with a user having Administration rights
  2. Go to the Administration page and select the Import category
  3. Follow the on-screen instructions to upload the downloaded XAR
  4. Click on the uploaded XAR and follow the instructions
  5. You'll also need to install all dependent Extensions that are not already installed in your wiki

Dependencies

Dependencies for this extension (org.xwiki.contrib.llm:application-ai-llm-index-ui 0.3.1):

Get Connected