XWiki Concerto

A research project for a peer-to-peer replicated wiki

Description

XWiki Concerto aims at evolving the open-source XWiki engine towards a P2P architecture supporting mobility, offline work and replication of content across a large number of peers. This research work will allow XWiki to support mobile workers in their daily collaborative activities, using an efficient and secure solution supporting replication and synchronization of the content, and to run on a P2P architecture that allows for scalability.

In this page you will find all the information and documentation to setup and start using a XWiki Concerto network.

Watch XWiki Concerto in action:

Prerequisites & Installation Instructions

Installation

XWiki Concerto is distributed using a self-installing JAR. After downloading it and execute the installer with the following command line

If you're installing XWiki Concerto on a server without any graphical display, you may perform an automated installation using the XML configuration file  http://www.github.com/xwiki-contrib/concerto/tree/master/concerto-installer/src/main/resources/install.xml by running the following command after having customized the installation path in the XML file:

Configuration

Before starting to use XWiki Concerto, XWoot, its peer-to-peer and replication engine, must be configured properly or "bootstrapped". The configuration is done only once per installation and the settings are being restored if XWoot is restarted. 

A wizard-like configuration process has been set up to help you trough the bootstrap process.

Bootstrap page

• XWiki Endpoint: This is the location of the XWiki installation you wish to be replicated. Actually, it is the endpoint of the XML-RCP protocol used by XWoot to communicate with the XWiki. If XWoot and XWiki are not on the same machine, replace "localhost:8080" with the "host:port" where XWiki is installed. If you have installed Concerto by using the provided installer, you can leave this to the default version.

• XWiki User: This is required in order to communicate with XWiki to retrieve modifications or to apply modifications received from others. It needs to be a privileged (usually Admin) user that can apply the modifications to all of the pages of the XWiki.

• XWiki Password: The password for the provider user.

• Data Serialization Folder: An accessible folder on drive where to store XWoot's data. This data is made up of network settings, group settings and, finally, XWoot's internal model of the replicated XWiki's contents. If you, later on, wish to reset XWoot to it's initial state, you can delete this folder. However, you will then need to reconfigure XWoot completely.

• XWoot Server Name: This is the name of your XWoot peer. This name will be visible to others in their neighbour list, once you join their group. It is recommended, but not required, to keep this name as unique as possible. Each peer has an unique ID used by XWoot's components but this is not visible or relevant to the end user.

Network bootstrap

What is a Network?

XWiki Concerto uses JXTA as the P2P network platform.

In short terms, a network is a collection of groups, peers and super-peers.

A super peer can be in charge of 2 things:

• Route communication for a network (RendezVous peer or just RDV). It has 2 major roles:
  • It can act as a network entry-point. Any peer willing to join the network of this peer can specify it as RDV Seed.
  • Keep track of the current groups in a network. Any peer that wishes to discover the existing groups in the network or who wishes to create a new group will forward the request to this peer.

• Relay communication for Firewalled/NAT-ed peers (Relay peer). This peer needs to be directly accessible from anywhere, meaning that it needs to have a real IP address or at least have port forwarding enabled.
  • It's role is to ensure connectivity throughout the network. Any peer that is behind a firewall and can not be directly accessed from the outside will add this peer as a Relay Seed. A Relay super-peer will forward any message from one peer to another, passing firewalls and NATs that would normally prevent the sender's message from reaching its destination.

Here's a picture to better understand relays, how they work and why they are so important for a network:

A network can also be seen as a controlled environment deployed by an organization to allow various departments to replicate their XWikis.

Imagine the scenario where a company creates a network and adds a couple of RDV and Relay peers. If no such network existed, every department that wishes to replicate it's wiki with it's department members would need to create its own network and deal on its own with Firewalls/NATs and generaly handle and assign super-peers on its own.

If the company did in fact create such a network, all they have to do is to enter the network parameters received from the network admin and they can now easily create a group and share an XWiki in that group. They don't have to worry about network logistics any more.

Creating a network

In order to create a new network, you just have to press the "Create network" button.

Also, make sure that you are using at least one communication method (see below), you are using an unused port for it and that you are accepting incoming communications on it. This is required because you will be acting as both RDV and Relay peer for your newly created network and this ultimately means that you will be a server for peers willing to join your network.
Joining a network

To join a network you must know at least one network entry-point (the address of a RDV peer of that network).

Here you have 2 options:

• Use JXTA public network. Because XWiki concerto uses JXTA, it can also use the public JXTA network provided by the JXTA team, more as a testing ground rather than a stable environment. It is recommended to use the XWiki Concerto network instead or a custom one. However, if the previous two options are not available for whatever reason, it can be used to quickly share an XWiki but there are no guarantees of any kind.

• Use custom network. It is normal for an organisation to wish to handle their own network (see above). Here you can manually specify the network parameters of a network you wish to join.

Network parameters

These parameters are:

• RDV SeedingUris: This is a comma separated list of addresses of a services that provide a list of RDV peers available in the network to use as entry-points. It is more convenient for an network admin to use such a service because he can then add/remove RDV peers and clients will start using them automatically. Example: http://www.myOrganisation.com/concertoNetwork/rdvs.php, http://www.myOrganisation.com/concertoNetwork/alternativeRdvs.php

• Relay SeedingUris: The same as above only that it applies for Relay peers. Note: You only need this if you know your computer is not reachable from the outside and it requires the service of a Relay. Example: http://network.myOrganisation.com/relays.php, http://network.myOrganisation.com/alternativeRelays.php

• RDV Seeds: If there is no SeedingUri available to get RDVs from, you can manually specify a comma separated list of RDVs to contact in order to join the network. You also need to specify the communication methods and port to use to reach them. Example: tcp://rdv1.myOrganisation.com:9701, http://rdv2.myOrganisation.com:9700

• Relay Seeds: The same as for RDV Seeds, only that it applies for Relays. Again, you need this if you are firewalled or NAT-ed. Example: tcp://relay1.myOrganisation.com:9701, http://relay2.myOrganisation.com:9700

• As explained above, you can also become a RDV or a Relay (super-peer) for the network you wish to join. You can do so by using the 2 checkboxes that follow. Please note that you also need to be reachable from the outside and you need to have incoming connections enabled.

A last thing to note about joining networks is that, if you need to use a Relay and have provided some relays in the inputs above, you must *disable incoming connections* when joining the network. If you do not do so, you will not be able to talk to other peers because you will not be properly using the available Relays.

Advanced settings

Here is where you set to use the communication method I have been talking about in the paragraphs above. They are quite self-explanatory.

The external IP setting is something you might need if you are normally NAT-ed but you have port forwarding enabled by your ISP. This can come in very handy because instead of using a relay, you can actually accept incoming connections and receive messages directly. You just have to provide the external IP for which you have port forwarding enabled and that will be used as well, besides your local addresses, to tell others how to reach you.

Use only external IP for all communication. This is when you want to be reached only trough the external IP and not trough your local addresses. You should use this when you know you are not accessible in any other way.

Bootstrap Group

What is a group?

Once you have joined a network, you will want to start replicating an XWiki.

In order to differentiate between replicated XWikis, groups are introduced. For each replicated XWiki there is a group whose members share the same XWiki data.

Groups can also be password protected to restrict access to copy or alternation of the replicated XWiki's data. The secure distribution of the group's password to trusted peers is the group creator's task.

In order for group members to communicate to each-other, there is a need of group RDVs (just like network RDVs). Once you have joined a group, you will automatically try to connect to an existing RDV from that group. If no such peer node is found, then you will automatically be promoted to RDV in order to enable group communication. The peer that created the group will automatically be RDV for the group.

The number of RDVs in a group is automatically managed by XWoot, by promoting and demoting peers depending on the number of peers currently inside a group. 

Creating a group

Creating a group is pretty straight forward.

You just have to specify a group name and an optional description.

If you wish to password protect your group, you can do so by checking the "Is private group" and then entering the desired group password.

Joining a group

To join a group, you have to select the desired group from the list of existing ones on the current network.

You can refresh the list of current groups by clicking on "Refresh List" button.

The Route communication for this group option makes this peer directly join the group as RDV peer, without waiting to contact other RDV peers inside the group. This can cause unwanted effects like not being able to contact peers in the group for an initial period of a couple of minutes. (This option could be removed in future versions)

Joining a group can take up to 2 minutes if no RDV peer is found in the group.

State Management

A state represents all the content of a replicated XWiki.

This concept derives from the fact that XWoot and XWiki do not share the same data on an implementation level and, before being able to start to function properly, XWoot needs to have its data initialized.

On a bit more detailed level, a state contain all the content present on the replicated XWiki but in a format on which XWoot can apply its merging algorithms and can keep track of and apply patches containing changes of that content.

Computing a state

In order to be able to start the replication of an XWiki, a state needs to be computed on the XWoot peer that created the group.

If the group was created by this peer, the button "Compute new state" will be available.

When you compute a new state, XWoot synchronizes with the XWiki for the first time.

Asking a state from the group

After joining a group you need to get the state of the XWiki that is being replicated inside that group.

You can do so by pressing the "Ask state from group" button.

After getting the state, XWoot will make it its own and synchronize with your local XWiki by replacing all your XWiki's content with the content described in the received state thus making your XWiki synchronized with the group's XWiki. 

Importing a state

A third option is to import a state file in zip format you received from a different source, other than asking it from the group.

Note: You need to provide the state file for the group you are joining and no other. This means you must not change the name of the zip file.

Tutorial

In this section we provide a tutorial setting up two replicated wikis that can be used to test XWiki Concerto capabilities.

In order to have a test setup, install XWiki Concerto on two different directories on the same host.

At the end you will have two directories, let's call them xwc1 and xwc2, on your disk. We will use these installations in order to setup two replicated wikis to play with.

Configuring the installations

First on all you have to launch the two XWiki Concertos by using the start_xwiki.sh command. You must specify the port number for the second instance otherwise a conflict would happen:

Now you have two instances running. You can check it by opening a browser at the following addresses: http://localhost:8080/xwootApp and http://localhost:8181/xwootApp

Configuring the first instance on http://localhost:8080/xwootApp

• The first screen you will see it's the "Bootstrap". You can leave all the parameters as they are (maybe you might want to change the Data serialization folder to a more suitable directory)

• Then you will have the "Network bootstrap" screen. Since no networks exist, you can click on the "Create" button in order to create a new network

• The "Group bootstrap" will appear on your screen. Since we are creating a new group, here you must configure the "Create group" part. Pick up a group name and fill the respective fields. If you want the group to be private, click on the "This is a private group" and fill the password fields with this the password you have chosen. Once you are done click on the "Create" button.

• At this point you must compute a state for your XWiki. Click on the "Compute new state" and wait that the process is over (it can take several minutes)

• Once the state is computed you will see the page with the (empty) neighbors list. XWiki Concerto is up and running and is waiting for discovering/accepting other peers for replicating content.

Configuring the second instance on http://localhost:8181/xwootApp

• In the "Bootstrap" screen make sure that you change the port number from 8080 to 8181 in order to correctly associate the replication engine to the right XWiki. You have also to specify a different directory from the one you specified earlier where storing data files. Finally pick up a different

• In the "Network bootstrap" screen, this time, you have to configure what are the Rendez-vous and Relay peers that will provide seeds for the P2P network. Use for both the "RDV Seeds" and "Relay Seeds" the following address: "tcp://localhost:9701". Make sure that you selected the "Use a custom network" radio button. You must also change the port numbers for in the advanced settings, otherwise they will clash with the ones of the first instance. Once you are done, click on the "Join" button.

• This time you will have to join an existing group. Click on the "Refresh List" and you will see the group you created in the dropdown combo box. Type the password that you defined for this group and then click the "Join" button.

• In the "State management" page, click "Ask state from group" in order to import the initial state and wait for the synchronization to terminate (it can take several minutes)

• Once the process is done you will see the page with the neighbors list and the name of the first instance listed in the panel. If you go to the http://localhost:8080/xwootApp/synchronize.do address, you will see the second instance's name listed in the neighbors list as well.

Now you have successfully configured the two instances for replicating their content.

Testing the replication

• Go to the first instance WebHome and login as Administrator (login:Admin, password:admin). Change the page and save it.

• Automatic replication happens every 60 seconds, so changes should be propagated in less than a minute.

• If you open the Main.WebHome page (http://localhost:8181/xwiki/bin/view/Main/) on the second instance you will see the synchronized content.