General Actions:
Filesystem Attachment Porter
Last modified by Caleb James DeLisle on 2012/05/07
 | Tool for porting attachments from the database to the flesystem store |
| Type | Snippet |
| Developed by | Caleb James DeLisle |
| License | GNU Lesser General Public License 2.1 |
| Compatibility | version 2.0: 3.4+ |
Table of contents
Description
With the new filesystem based attachment storage engine, you can save attachments without placing load on the database. You can also save very large (over 1 gigabyte) attachments. However all of your existing attachments are still in the database and schemes such as checking the filesystem and then the database or automatically porting the attachments out were judged to be complex and magical, both attributes which in storage we want to avoid, the answer is to use a script to move all of the attachments under the administrator's supervision.
When running the script, it will walk you through the process but a preview of the steps are as follows:
Step 1: Switch to Filesystem attachments.
Edit your xwiki.cfg file by modifying the attachment store lines to read as follows:
xwiki.store.attachment.hint = file
xwiki.store.attachment.versioning.hint = file
xwiki.store.attachment.recyclebin.hint = file
xwiki.store.attachment.versioning.hint = file
xwiki.store.attachment.recyclebin.hint = file
Also make sure they are not commented out.
The storage section of my xwiki.cfg file reads as follows:
#---------------------------------------
# Storage
#
#-# Role hints that differentiate implementations of the various storage components. To add a new implementation for one
#-# of the storages, implement the appropriate interface and declare it in a components.xml file (using a role-hint other
#-# than 'default') and put its hint here.
#
#-# The main (documents) storage.
# xwiki.store.main.hint=default
#-# The attachment storage.
xwiki.store.attachment.hint=file
#-# The document versioning storage.
# xwiki.store.versioning.hint=default
#-# The attachment versioning storage. Use 'void' to disable attachment versioning.
xwiki.store.attachment.versioning.hint=file
#-# The document recycle bin storage.
# xwiki.store.recyclebin.hint=default
#-# The attachment recycle bin storage.
xwiki.store.attachment.recyclebin.hint=file
#-# Whether the document recycle bin feature is activated or not
# xwiki.recyclebin=1
#-# Whether the attachment recycle bin feature is activated or not
# Storage
#
#-# Role hints that differentiate implementations of the various storage components. To add a new implementation for one
#-# of the storages, implement the appropriate interface and declare it in a components.xml file (using a role-hint other
#-# than 'default') and put its hint here.
#
#-# The main (documents) storage.
# xwiki.store.main.hint=default
#-# The attachment storage.
xwiki.store.attachment.hint=file
#-# The document versioning storage.
# xwiki.store.versioning.hint=default
#-# The attachment versioning storage. Use 'void' to disable attachment versioning.
xwiki.store.attachment.versioning.hint=file
#-# The document recycle bin storage.
# xwiki.store.recyclebin.hint=default
#-# The attachment recycle bin storage.
xwiki.store.attachment.recyclebin.hint=file
#-# Whether the document recycle bin feature is activated or not
# xwiki.recyclebin=1
#-# Whether the attachment recycle bin feature is activated or not
Step 2: Add a new directory to your backup routine.
Since data is going to be stored on the filesystem, you need to make the directory where it is stored is backed up regularly.
Step 3: Copy attachments from database to filesystem.
Now you are ready to copy the data over from your database to the filesystem. It is prudent to leave the attachments in the database since in most situations the attachment data is not bothersome just sitting in the database (The only risk of attachments left in the database is that they will bloat the size of the database files). As such, this script contains no facility to delete entries from the database.
If anything goes wrong in this function, it will fail with an error message and you should get the stack trace, keep it to confuse and humiliate the developer with. No harm should be done since this only loads from the database and only saves to the filesystem.
Checking for corruption is recommended using the "dry run" option, you can also enable "verbose" to watch the list of attachments be printed to the screen as they are processed.
Once you click the "start" button, it will do the work in 20 second chunks and print the results of each chunk. It is doing something.
Step 4: Make sure everything is working.
Check to make sure your attachments are still there, if an attachment is broken, it will appear to be there but on opening it will tell you the attachment does not exist. If something goes terribly wrong with filesystem attachment store, you may have your old attachment system back simply by changing the lines in the xwiki.cfg file and restarting the container. HOWEVER: This will not preserve attachments which were uploaded after switching to filesystem attachments so as users edit the system you will become locked in to filesystem attachments unless a script is written to do the inverse of this one.
Despite being experimental, the filesystem attachment storage is quite stable and the risk of actually losing something so that it is unrecoverable is very remote.
NOTE: This must be run seperately in each subwiki.
And that's all it is.
Prerequisites & Installation Instructions
Copy the code snippet to a page and save it.
- You must run this as a user with programming permission.
- Since it is not intended to be secure, it should be removed or disabled after use.
Release Notes
Release notes for version 2.0:
Handles large wikis much better, doing work in 20 second chunks.
Javascript front end prints attachments as they are processed.
Errors are explained better and stack traces are printed to the window rather than the log.
Dry run option to scan for corruption without saving files.