Mail Archive Admin Guide

Last modified by Jeremie Bousquet on 2012/09/14 13:11

Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed in [extensions:Extension.Mail Archive Admin Guide]. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details.

Installation and initial set-up

Installation is straightforward using the extension manager, initial set-up is a bit more complex but necessary.

Once installed, proceed to page MailArchive.Admin to start initial set-up.

If you see the following :

MA_Admin_1_MissingPrerequisites.png

... it means the dependend Tabs Macro was not installed. Please install it as recommended before continuing.

You should see the following:

MA_Admin_2.png

The first thing to do is to define an administrator profile for the Mail-archive.

What is a mail-archive administrator profile ?
It is a specific xwiki user that has full admin rights on the mail-archive. It is this user that will create all pages when mails are loaded into the archive. It is a good practice to create a specific user for that, but you can also use an already existing admin profile.

If you want to have a specific profile for that (recommended), proceed as follows then go back to the Admin page:

  • Create a new wiki profile using XWiki administration. You can name it "MailArchiveAdmin" or whatever pleases you
  • Assign edit rights to this user on spaces MailArchive, MailArchiveCode, MailArchiveItems and MailArchivePrefs.
  • Leave only view rights to all other users to the same spaces (use XWikiAllGroup).

If you want to use an existing admin profile, do the same steps above with it.

To administrate and configure the mail archive, you only need to be logged with a user that has edit rights on MailArchivePrefs space. You could use the specific profile defined, or any profile with admin rights.

Once you have a mail archive admin profile, go back to the mail-archive admin page, and select "Global Parameters" tab:

MA_Admin_3_GP_LoadingUser.png

In the "Loading User" field, choose the profile you set-up previously, then proceed to the bottom of the page and click on "SAVE".

You should see the page updated as follows:

MA_Admin_4_AdminProfileSet.png

Follow the indications and click on "Install" link in the Summary pane.

In the Install page that shows, click on "Confirm assigning..." link to set programming rights to the configured admin profile, and add translations bundle in wiki preferences, then go back to admin page.

If you decide later to use another profile as mail archive admin, re-do all previous steps with this new profile.

Adding the translation bundle to xwiki prefs should be done by this step, though it does not work yet. So please add manually "MailArchiveCode.Translations" document to the list of internationalization bundle in XWiki preferences.

Now the application is installed but will remain empty until we configure it and start loading emails ...

Configuration

The configuration consists of a set of parameters that let you define the behaviour of the archive (tabs Global Parameters, Enterprise Features and Advanced Parameters of Admin page). It consists also of defining what will be the content of the archive (the Mailing-Lists tab), some sources for loading emails (the Servers tab) and how content will be organized (the Mail Types tab).

Default parameters should be ok most of the time, so we'll see them later, but defining servers and lists is mandatory.

Defining new source server connections

You need to create one or more Servers to define sources to load emails from. A Server hosts the configuration to connect to a mail server and account, as well as a targeted folder. It is possible to target different folders on same email server account.

Suppose for example that we want to load emails from the XWIKI and JENKINS folders from my gmail account below:

MA_Admin_Server_2_google.png

Note: the mail archive uses javamail under the hood, so possible mail servers connections available are the ones of javamail. The application has been tested against gmail and exchange servers, others should work but if you have special needs / find issues please create a JIRA issue in contrib project.

Here, the XWIKI folder contains emails from mailing-lists xwiki devs and xwiki users, while JENKINS folder contains mails from the Jenkins users list (of course, define you must previously have registered to the needed mailing-lists, and it's recommended to put them in specific folders). The mail archive app is capable of distinguishing lists stored in the same folder though it's practical to use several folders for your own sake.

Back to the Admin page, go to Server tab:

MA_Admin_Server_1.png

It's quite empty ... For now. Click on the "Add..." button:

MA_Admin_Server_3_addserver.png

Description:

  • Id: a unique ID you define for this server connection and folder. This is used under the hood and for some advanced features we'll see later, but remember one thing: you should never update the id of an existing server. If you really need to, create a new server with new id and optionally remove the old one.
  • Hostname: here, "imap.gmail.com"
  • Port: here, 993
  • Protocol: gmail is "imaps"
  • User/Password: your credentials for mail server connection
  • Folder: targeted directory containing emails to load. Notes: mails loading does not scan sub-folders recursively. To target sub-folders, merely use '/' as separator. Here we will first enter "XWIKI" to target xwiki lists.
  • Additional Javamail Properties: you can specify additional properties that will be used for connection with javamail to this server. These properties can target javamail API and/or the specific server you connect to. See Javamail api doc for a description. By the way, the mail archive automatically adds some specific properties needed for gmail when you target it, so you shouldn't have to care about this.
  • State: a Server can be enabled or disabled. Enabled means only that this Server will be taken into account by automatic loading job. It also means that you can always manually load emails even from a Disabled Server.

Enter connection and folder information then click on "save" button:

MA_Admin_Server_4_addserver.png

You see the Server we defined has successfully been added. You may note also the "Status" that is "unknown". It's time to check that we can successfully connect to this server ...

To do that, open the Server page by clicking on its Id in the first column:

MA_Admin_Server_5_testserver.png

Then click on "Test Connection" button (upper right):

MA_Admin_Server_TS_UnknownHost.png

Connection failed, because gmail hostname is not "imaps.gmail.com" but "imap.gmail.com". Let's edit the page to correct this, then test again:

MA_Admin_Server_6_TestOk.png

Notice that this check also shows the number of available messages. This is in fact the number of unread message in the targeted folder. By default the archive only bothers with unread messages, we'll see later that it's possible to load already read messages if needed.

Now is time to get back to Admin page, Server tab, and add a new Server with exactly same configuration, except that this time targeted folder will be "JENKINS".

As a result we get this list of servers:

MA_Admin_Server_7_ServersList.png

The number in "Status" column are the number of unread emails available from this source (at the time connection was tested - this is not automatically refreshed). Negative numbers indicate that last connection test was a failure.

In case of failures, when testing connection, the app tries to provide to you some indications about the problem. Anyway, you can check the Trouble-shooting section to help you connect successfully.
(Particularly, with Exchange servers in enterprise environment, you might need to setup things about certificates...)

Defining Mailing-Lists

The mailing-lists are assigned only during email loading phase, meaning that you must specify them before loading related emails. If you add new mailing-lists, existing emails won't be assigned even if they match the pattern specified.

Now is time to define our 3 mailing-lists: xwiki devs and users, and jenkins users. In fact, we will register the 3 mailing-lists email addresses: [email protected], [email protected] and [email protected].

Proceed to "Mailing-Lists" tab then click on "Add" button:

MA_Admin_Lists_1_add.png

Description:

  • Display Name: the name says it all... here we choose "XWiki Users List"
  • Pattern: the email to match, first we enter "[email protected]"
  • Tag: a tag that will be applied to all created pages that belong to this mailing-list, here we choose "XWiki Users"
  • Color: a color that will nicely differentiate this list from others in some mail archive screens. Enter this as you would for a CSS color, click indicated link for more details. Here we will enter "darkblue"
  • List Id: this is optional, and not managed for now so don't bother

Click on "Save" button to create the new list, then repeat these steps twice to add the xwiki devs and jenkins users lists.

We eventually get this:

MA_Admin_Lists_2_add.png

The (nice) colors will be used when displaying statistics about the archive, and in some other places.

Defining mail types

The mail types are assigned only during email loading phase, meaning that you must specify them before loading related emails. If you add new mail types, existing emails won't be assigned even if they match the pattern specified.

Mail types are an optional but powerful feature. They allow you to categorize loaded emails. Mail types are also independant of mailing-lists.
For example for XWiki lists, you could specify mail types for Announcements, Proposals, XWiki Farm Requests, ...
A mail can belong to more than one type, if it matches more than one.

Mail types are defined by matching specific parts of emails (subject, from, to, body...) against regular expressions, allowing for precise matching.

But let's proceed to the Mail Types tab in Admin page:

MA_Admin_Types_1.png

Mmm... There's already something this time.
What you see is in fact a built-in type - do not remove it.

Mail Types assignment rules:

  • If a mail does not match any other type, it is assigned the default "Message" type
  • If a mail matches at least one other type, it is assigned only this matched type
  • If a mail matches several types, it is assigned all these types (but again, not the default one)

So, let's add a mail type for wiki announcements. Usually, these mails contain "[ANN]" in the subject. If you look carefully, you'll see that sometimes these mails contain "[Announcement]" or "[Ann]" or some variations.
So first we have to define the regular expression that will match what we want from the subject.

I propose to use this:

(?mi)^.*\[ANN.*\].*$

If you need more details about regular expressions please check Javadoc about Pattern class...

  • (?mi) : flags to match multi-line content, and ignore case
  • ^ $ : match begin and end of line respectively
  • .* : match any character any number of time
  • \] \[ : escape special characters

So let's click on "Add..." button to create our new type:

MA_Admin_Types_2_add.png

Description:

  • Name: a unique ID for this type (not that despite of the "name" it's more an "ID")
  • Display Name: the display name ...
  • Icon: like the color for mailing-lists, but types are marked by an icon. Enter the name of an icon from XWiki Icon Set. For an announcement, we will use a "bell"
  • Patterns List: a "pattern" is the association of specific field(s) (subject, body ...) and of a regular expression to match. Here we check the regular expression defined above against the subject. You can match the same regular expression against different fields. You can also use buttons to add or remove patterns, allowing to match different regular expressions.

For now only "AND" logical operator is possible between different patterns, but if needed "OR" could be added later, and so also priorities (parenthesis).

After adding the types for Announcements and XWiki Farms Requests:

MA_Admin_Types_3_typeslist.png

We highly recommend to think, prepare and create all needed types before starting to download emails, because this information won't be easily updated afterwards...

Now we can take care of other parameters available.

Summary View

If you go back to the Summary View, you'll see that it gives you an overview of the mail archive:

MA_Admin_Summary.png

Here of course there are not topics nor email as we did not start loading anything.

This view also shows that you could add some optional extensions to your wiki, that could be activated to improve the mail archive. The app will work without these extensions though, and even if present you'll need to activate them to be used by the mail archive.

Global Parameters

MA_Admin_GP.png

Enterprise Features

MA_Admin_EF.png

We highly recommend to leave the "Use local store" option checked.
With this option, a backup copy of all (successfully) loaded emails is kept in a local store on server filesystem. Later, a feature will allow to reload emails from this store, for example to manage mail archive migrations, or to re-apply a new type or a new mailing-list to the whole set of emails.
Be aware that this will consume disk space on server filesystem though, so be really careful on production platforms that there is enough space (complete emails are stored, including attachments if any).

Tags:
    

Get Connected