Diff Module

Last modified by Michael Hamann on 2023/07/26 16:06

cogA set of diff/merge APIs
TypeJAR
Category
Developed by

XWiki Development Team

Rating
0 Votes
LicenseGNU Lesser General Public License 2.1
Bundled With

XWiki Standard

Compatibility

Since 4.1 Milestone 2

Description

Since there is no real standard API for it and that most of the existing library are not very active enough or very tied to various EDIs we decided to write our own API for XWiki ecosystem which would be independent from the actual implementation.

List Diff & Merge

Base API module

Expose generic diff and merge component implementing org.xwiki.commons.diff.DiffManager role.

The main idea of this API is that it's not tied to any type so you have to first transform the datas you want to diff into lists and the diff will be a diff between thoses two lists.

For example when you want to diff text you can choose if you want to to a line based diff, a word based diff, a character based diff, etc.

public interface DiffManager
{
   /**
     * Produce a diff between the two provided versions.
     *
     * @param <E> the type of compared elements
     * @param previous the previous version of the content to compare
     * @param next the next version of the content to compare
     * @param configuration the configuration of the diff behavior
     * @return the result of the diff
     * @throws DiffException error when executing the diff
     */

   <E> DiffResult<E> diff(List<E> previous, List<E> next, DiffConfiguration<E> configuration) throws DiffException;

   /**
     * Execute a 3-way merge on provided versions.
     * If a conflict is detected during the merge, no error is triggered and the returned {@link MergeResult} object
     * always has a result (see {@link MergeResult#getMerged()}): the conflict is automatically fixed with a fallback
     * defined by the {@link MergeConfiguration}.
     *
     * If the {@link MergeConfiguration} instance contains some {@link ConflictDecision}
     * (see {@link MergeConfiguration#setConflictDecisionList(List)}), then those decisions are taken into account
     * to solve the conflict. The decision linked to the conflict is retrieved and applied, unless the decision state
     * is still {@link org.xwiki.diff.ConflictDecision.DecisionType#UNDECIDED}. When a decision is used to solve a
     * conflict, the conflict is not recorded in the {@link MergeResult}.
     *
     * If the decision is {@link org.xwiki.diff.ConflictDecision.DecisionType#UNDECIDED}, or if no decision is available
     * for this conflict, then the fallback version defined in the {@link MergeConfiguration}
     * (see {@link MergeConfiguration#setFallbackOnConflict(MergeConfiguration.Version)}) is used to fix the conflict,
     * but in that case the conflict is recorded in the returned {@link MergeResult}.
     *
     * Finally the configuration parameter accepts a null value: in that case, the fallback version is always the
     * current version.
     *
     * @param <E> the type of compared elements
     * @param commonAncestor the common ancestor of the two versions of the content to compare
     * @param next the next version of the content to compare
     * @param current the current version of the content to compare
     * @param configuration the configuration of the merge behavior
     * @return the result of the merge
     * @throws MergeException error when executing the merge
     */

   <E> MergeResult<E> merge(List<E> commonAncestor, List<E> next, List<E> current, MergeConfiguration<E> configuration)
       throws MergeException;
}

Merge Configuration

The merge configuration class is mainly used to setup the fallback version of the merge in case of conflict. As specified in the javadoc, the merge operation always return a MergeResult which contains a merged value, even if they were conflicts during the operation. The possible fallback versions, are current, next or previous, current being the default version to fallback if nothing is specified. 

Starting with XWiki 11.7RC1 the MergeConfiguration and MergeResult classes get new APIs to retrieve precisely the conflicts information, and to be able to take decisions on a conflict by conflict basis. The decisions specified on the MergeConfiguration to solve a conflict have always the priority over the fallback configuration.

Display API module

This module helps you display a DiffResult obtained with the base API in various formats. Currently there are two diff formats supported.

Inline diff

An inline diff is made of a list of chunks, each marked as added, removed or unmodified. For instance, if changes are computed at word level, you can have this inline diff:

The quicksick brown fox jumpsstumbles over the lazycrazy mad dog.

At character level the diff looks a bit different:

the qusick brown fox

In this case the first chunk is "the ", an unmodified chunk, made of 4 characters and the second chunk is "qu", a removed chunk, made of 2 characters. An inline diff can be displayed either as you've seen above, mixing added and removed chunks in one line, or it can be displayed on two lines, one showing the removed chunks and the other the added chunks:

The quick brown fox jumps over the lazy dog.
The sick brown fox stumbles over the crazy mad dog.

Unified diff

An unified diff consists in a list of blocks, each block grouping changes that are close to each other. The distance between two changes in a block is less than 2 * context size, where context size represents the number of unmodified elements to include before and after a change in order to place that change in context.

If the elements can be split in sub-elements, i.e. if a splitter is provided through the configuration, then the unified diff displays also the changes inside the modified elements using the inline format.

If changes are computed at the line level in a text, i.e. the elements that are compared to produce the diff are lines of text, and a word splitter is provided through configuration then the following is a block from a unified diff:

@@ -81,5 +85,5 @@
 first line of context
 another unmodified line
-this line <del>has been removed</del>
+this line <ins>replaced the previous line</ins>
 close the block with unmodified lines
 last line of context

Script service

Expose more script oriented APIs of the two previous modules. It also adds helpers like String based APIs etc.

## Compute the differences between two lists of elements (e.g. lines of text).
#set ($diffResult = $services.diff.diff($previous, $next, $configuration))

## Display the differences between two texts in the unified format.
#foreach ($block in $services.diff.display.unified($previous, $next))
  $block
#end

Conflicts Display

Starting with XWiki 11.7RC1 the display diff API also allows to display information about conflicts.
The APIs to build unified diff takes as input a list of conflict elements that have been computed during a merge: some conflict information are then available in the resulting unified diff blocks, and can be used to present conflicts and possible decisions inside an UI presenting unified diff.

XML Diff

This module provides an API to compute and display the changes between 2 XML trees (DOM instances). It was introduced in XWiki 11.6RC1.

Computing the Changes

To compute the changes you need to use the XMLDiff component.

@Inject
private XMLDiff xmlDiff;

@Inject
private XMLDiffConfiguration config;

...

Document left = parseXML("...");
Document right = parseXML("...");
Map<Node, Patch<?>> patches = this.xmlDiff.diff(left, right, this.config);

The default implementation uses the list diff API to compute the changes between child nodes, attributes and text content. The algorithm is quite simple:

if leftNode and rightNode are "similar"
  if leftNode has value (true for text, comment or attribute nodes)
    if leftNode's value is different than rightNode's value
      compute the changes using the splitter indicated in the configuration
      (character splitter is used for text nodes by default, but there is also a word splitter available)
  else
    if leftNode has attributes (true for elements)
      compute the difference between attributes (added, removed, modified)
    compute the difference between child nodes (lists of nodes), matching nodes that are "very similar"
    compute (recursively) the changes between leftNode's children that are "very similar" to rightNode's children
else
  add change delta

Where "similar" and "very similar" are defined as:

similar = same nodeType, nodeName, localName, namespaceURI, prefix
very similar = similar and the percent of text changes (Levenshtein distance / max length) is less than 60%

The similarity threshold (0.6 by default) can be changed from the configuration.

As indicated in the algorithm described above, you can also change from the configuration the text splitter used for a specific node type. And, of course, you can implement your own splitter component.

@Inject
@Named("myCustomSplitter")
private StringSplitter myCustomSplitter;

@Inject
private XMLDiffConfiguration config;

...

((DefaultXMLDiffConfiguration) this.config).setSplitterForNodeType(Node.TEXT_NODE, this.myCustomSplitter);

Displaying the Changes

The component responsible for computing and displaying the changes is XMLDiffManager. There's no generic (default) implementation provided at the moment. The provided implementation is dedicated to displaying changes between HTML documents.

@Inject
@Named("html/unified")
private XMLDiffManager unifiedHTMLDiffManager;

@Inject
@Named("html")
private XMLDiffConfiguration config;

...

String previousHTML = "...";
String nextHTML = "...";
String diff = this.unifiedHTMLDiffManager.diff(previousHTML, nextHTML, this.config)

Be careful if you use this with some content generated from an XWikiDocument: if for some reasons (e.g. a missing script right and a macro needing it) the content generates a stacktrace, then the stacktrace will contain the line of the code where the content has been generated. Then if you generated the code for previousHTML and the code for nextHTML in two different lines, you will obtain a difference in the stacktraces that you probably don't expect to see. So for such scenario it's generally useful to encapsulate both rendering on the same line like this: this.unifiedHTMLDiffManager.diff(getRenderingContent(previousDoc), getRenderingContent(nextDoc), this.config).

You can control from the configuration which XMLDiffFilters are applied on the XML documents before and after computing the changes. You can also implement your own filters, e.g. to remove irrelevant changes, or to ignore parts of the XML documents while computing the changes. The default configuration applies a filter to mark context (unmodified) nodes after the changes are computed.

@Inject
@Named("myCustomFilter")
private XMLDiffFilter myCustomFilter;

@Inject
@Named("html")
private XMLDiffConfiguration config;

...

this.config.getFilters().add(this.myCustomFilter);

Script Service

Here's how you can compute and display the changes from a Velocity script:

{{velocity}}
{{html clean="false"}}
#set ($discard = $xwiki.ssfx.use('uicomponents/viewers/diff.css', true))
#set ($discard = $xwiki.jsfx.use('uicomponents/viewers/diff.js'))
#set ($previousHTML = '<p>one two three</p>')
#set ($nextHTML = '<p>one 2 three</p>')
<div class="html-diff">
  #set ($htmlDiff = $services.diff.html.unified($previousHTML, $nextHTML))
  #if ($htmlDiff == '')
    No changes.
  #elseif ("$!htmlDiff" == '')
    Failed to compute the changes.
  #else
    $htmlDiff
  #end
</div>
{{/html}}
{{/velocity}}

You can configure the diff by passing a third argument:

#set ($config = $services.diff.html.defaultConfiguration)
#set ($discard = $config.setSimilarityThreshold(0.6))
#set ($discard = $config.addFilter('hintOfMyCustomFilter'))
## Change the splitter used for text nodes from 'character' to 'word'.
#set ($discard = $config.setSplitterForNodeType(3, 'word'))
#set ($htmlDiff = $services.diff.html.unified($previousHTML, $nextHTML, $config))

The default configuration applies a filter that embeds images into the HTML before computing the changes, in order to compare the image data and not the image location.

Starting with XWiki 12.5RC1 you can also have a button to toggle show / hide the context nodes (unmodified content):

{{velocity}}
{{html clean="false"}}
...
<div class="changes-body">
  <div class="changes-actions active">
    <a href="#toggleRenderedDiffContext" class="html-diff-context-toggle html-diff-context-toggle-show hidden">
      <span class="html-diff-context-show">
        $services.icon.renderHTML('plus-square')
        $escapetool.xml($services.localization.render('web.history.changes.showContext'))
      </span>
      <span class="html-diff-context-hide">
        $services.icon.renderHTML('minus-square')
        $escapetool.xml($services.localization.render('web.history.changes.hideContext'))
      </span>
    </a>
  </div>
  <div class="html-diff">
    ...
  </div>
</div>
{{/html}}
{{/velocity}}

Configuration of Image Embedding

When comparing HTML documents, by default XWiki embeds images as data URIs to compare their contents instead of their URLs. Images are cached to avoid repeatedly downloading the same URLs.

14.10.5+, 15.5.1+, 15.6+ 

The embedding of images can be configured through xwiki.properties. It is possible to disable this feature, to limit the maximum size of the images to be embedded or the timeout to use when downloading images to avoid waiting too long when the server that hosts the image is unresponsive. The available options are:

#-# [Since 14.10.15, 15.5.1, 15.6]
#-# If the compared documents contain images, they can be embedded as data URI to compare the images themselves
#-# instead of their URLs. For this, images are downloaded via HTTP and embedded as data URI. Images are only
#-# downloaded from trusted domains when enabled above. Still, this can be a security and stability risk as
#-# downloading images can easily increase the server load. If this option is set to "false", no images are
#-# downloaded by the diff and images are compared by URL instead.
#-# Default is "true".
# diff.xml.dataURI.enabled = true

#-# [Since 14.10.15, 15.5.1, 15.6]
#-# Configure the maximum size in bytes of an image to be embedded into the XML diff output as data URI.
#-# Default is 1MB.
# diff.xml.dataURI.maximumContentSize = 1048576

#-# [Since 14.10.15, 15.5.1, 15.6]
#-# Configure the timeout in seconds for downloading an image via HTTP to embed it into the XML diff output as data URI.
#-# Default is 10 seconds.
# diff.xml.dataURI.httpTimeout = 10

Tags:
    

Get Connected